Console application to search for files that contain a match for a regex I created a small console application to recursively search a folder and all subfolders for files that contain a match for a regex.

The command line arguments are the path you're looking to search followed by the regex (in .NET regex format).

I break this up into two threads: the main thread loops through the files and puts file data (their content, path, and name) on a concurrent queue. The second thread processes the queue, looking for regex matches.

Would using a Parallel.ForEach loop be better here? The main issue I had with this is that the queue is being modified while we're looping over it, but a Parallel.ForEach loop assumes that you already have a complete collection. The two possibilities I came up with this:


Just read the content of the files, and then do a Parallel.ForEach loop to do the regex.
Process it on the background thread like I'm doing until we're done reading the file contents, then do a Parallel.ForEach loop over any remaining items once we're done.


What I have here works but is slower than I'd like. I'd definitely like feedback on the multithreading I'm doing here, especially if one of the possibilities I mentioned above would be better than what I'm doing (or if there's another possibility that I don't mention above).

namespace SearchByRegex
{
class Program
{
    // I should probably just move this out to its own file
    private class FilePair
    {
        public string Content { get; set; }

        public string FilePath { get; set; }

        public string FileName { get; set; }
    }

    private static ConcurrentQueue&lt;FilePair&gt; fileInformation = new ConcurrentQueue&lt;FilePair&gt;();

    static void Main(string[] args)
    {
        Console.WriteLine("Working...");

        // Command lines are a path followed by the regular expression
        var regex = new Regex(args[1], RegexOptions.Multiline);

        // Process
        Thread thread = new Thread(PrintInformationOnMatches);
        thread.Start(regex);

        SearchDirectory(args[0]);            

        while (fileInformation.Any())
        {
            Thread.Sleep(1000);
        }

        thread.Abort();

        Console.WriteLine("Press any key to exit");
        Console.ReadLine();
    }

    /// &lt;summary&gt;
    /// Print information on all files whose content matches the &lt;paramref name="regex"/&gt;
    /// &lt;/summary&gt;
    /// &lt;param name="regex"&gt;Regex we're looking for&lt;/param&gt;
    private static void PrintInformationOnMatches(object regex)
    {
        var castRegex = (Regex)regex;
        while (true)
        {
            // Process the entire queue
            while (fileInformation.Any())
            {
                FilePair pair;
                fileInformation.TryDequeue(out pair);

                if (castRegex.IsMatch(pair.Content))
                {
                    Console.WriteLine($"File name: {pair.FileName}. Path: {pair.FilePath}.");
                }
            }

            // Wait for more stuff on the queue
            Thread.Sleep(1000);
        }
    }

    /// &lt;summary&gt;
    /// Get the contents of the file and put them on the queue
    /// &lt;/summary&gt;
    /// &lt;param name="path"&gt;Path we are recursively searching&lt;/param&gt;
    private static void SearchDirectory(string path)
    {
        string[] files = Directory.GetFiles(path);

        //Console.WriteLine("Path: " + path);
        foreach (string file in files)
        {
            fileInformation.Enqueue(new FilePair { Content = File.ReadAllText(file), FilePath = path, FileName = file });
        }

        foreach (string directory in Directory.GetDirectories(path))
        {
            SearchDirectory(directory);
        }
    }
}
}
private static void SearchDirectory(string path)



This method does not need to be recursive. You could use the Directory.EnumerateFiles method


Directory.EnumerateFiles Method (String, String, SearchOption)



where


  One of the (SearchOption) enumeration values that specifies whether the search operation should include only the current directory or should include all subdirectories.


I don't know how your regex looks like but for simple pattern with wildcards you can use the second parameter 


  The search string to match against the names of files in path. This parameter can contain a combination of valid literal path and wildcard (* and ?) characters (see Remarks), but doesn't support regular expressions.




As far as the multithreading is concered you should definitely use the new TPL (Parallel.ForEach) instead of creating your own threads.