Find the oldest file

I had a problem at work the other day. In a project we have a service that needs to handle files on a first come, first served basis. In effect it scans a folder for files and find the file with the oldest last modified stamp, then it processes the file.

This seems a simple enough task, but once when the service was down for a long time the number of files waiting to be processed was huge (in the several hundred thousand range). Scanning a folder with that many files in C# and then determining the oldest one takes quite a long time.

So I took a look at three different methods for solving this simple task.

1. Using the DirectoryInfo.GetFiles function
2. Using the Directoty.GetFiles and then Directory.GetLastWriteTime on each file
3. Using plain old DOS dir command

For 50.000 files the first two options takes about 7 seconds each on my development machine. While using dir takes only 0.7 seconds. Thats 10 times the performance difference.

This is how I managed to tweak dir to do the work:

static string ScanFilesUsingDir(string filePath)
{
   string cmd = @"c:\Windows\System32\cmd.exe";
   string args = @"/C \"dir " + filePath + " /B /O:D /T:W\"";

   Process proc = new System.Diagnostics.Process();
   proc.EnableRaisingEvents = false;
   proc.StartInfo.CreateNoWindow = false;
   proc.StartInfo.FileName = cmd;
   proc.StartInfo.Arguments = args;
   proc.StartInfo.RedirectStandardOutput = true;
   proc.StartInfo.UseShellExecute = false;
   proc.Start();

   // Display returned console information if any
   StreamReader dirOutput = proc.StandardOutput;
   string oldestFile = dirOutput.ReadLine();
   dirOutput.Close();

   proc.Kill();
   return oldestFile;
}

Oldest file found: f9e41b18-efa8-49ee-9a51-a89f7ef514ee.tmp
Activity ‘Using DirectgoryInfo’ took 00:00:07.0810000

Oldest file found: f9e41b18-efa8-49ee-9a51-a89f7ef514ee.tmp
Activity ‘Using FileInfo’ took 00:00:07.2710000

Oldest file found: f9e41b18-efa8-49ee-9a51-a89f7ef514ee.tmp
Activity ‘Using Shell Dir’ took 00:00:00.7030000

Just goes to show that POD (Plain Old Dos) still is good for something

Complete source code: FileGenerator.rar