Geek asks and answers

How to do a recursive search for folders and files using producer / consumer queues?

I would like to search the directory first, and then the files inside it, for the keyword.

I know that I need two classes: the Producer class and the Consumer class, but I don’t know how to search using the C # producers / consumers queue?

public class Program
{
    private static void Main()
    {
        Queue<File> searchFile = new Queue<File>();
        Queue<Directory> searchDirectory = new Queue<Directory>();

        new Thread(searchDirectory).Start();

        for (int i = 0; i < 3; i++)
            new Thread(searchFile).Start();
    }
}
+3
source share
4 answers

Initial issues:

  • You declare 2 variables of different types, using the same variable name with the same scope.
  • You do not want to run threaded Directory searches and the other one in the file.

2 , - IO. , IO ( ), .

, (, ). , .

+1

-, Directory , . DirectoryInfo. -, , DirectoryInfo. .

, -. BlockingCollection, . -, .

public class Searcher
{
  private BlockingCollection<DirectoryInfo> m_Queue = new BlockingCollection<DirectoryInfo>();

  public Searcher()
  {
    for (int i = 0; i < NUMBER_OF_THREADS; i++)
    {
      var thread = new Thread(Run);
      thread.IsBackground = true;
      thread.Start();
    }
  }

  public void Search(DirectoryInfo root)
  {
    m_Queue.Add(root);
  }

  private void Run()
  {
    while (true)
    {
      // Wait for an item to appear in the queue.
      DirectoryInfo root = m_Queue.Take();

      // Add each child directory to the queue. This is the recursive part.
      foreach (DirectoryInfo child in root.GetDirectories())
      {
        m_Queue.Add(child);
      }

      // Now we can enumerate each file in the directory.
      foreach (FileInfo child in root.GetFiles())
      {
        // Add your search logic here.
      }
    }
  }
}

, , , , , , , .

+1

, , IO, . , ( ), . :

    class Program
{
    static void Main(string[] args)
    {
        ConcurrentQueue<DirectoryInfo> concurrentQueue = new ConcurrentQueue<DirectoryInfo>();
        GetAllDirectories(new DirectoryInfo(@"C:\local\oracle"), concurrentQueue);
        Action action = () =>{
            const string toFind = "ora";
            DirectoryInfo info;
            while(concurrentQueue.TryDequeue(out info))
            {
                FindInFile(toFind, info);
            }
        };

        Parallel.Invoke(action, action, action, action);
        Console.WriteLine("total found " + _counter);
        Console.ReadKey();
    }
    static int _counter = 0;
    static void FindInFile(string textToFind,DirectoryInfo dirInfo)
    {
        var files =dirInfo.GetFiles();
        foreach(FileInfo file in files)
        {
            using (StreamReader reader = new StreamReader(file.FullName))
            {
                string content = reader.ReadToEnd();

                Match match = Regex.Match(content, textToFind, RegexOptions.Multiline);

                if(match.Success)
                {
                    Interlocked.Increment(ref _counter);
                    Console.WriteLine(file.FullName + " found " + match.Captures.Count);

                    foreach(var t in match.Captures)
                    {
                        Console.WriteLine("-------------> char index" + match.Index);
                    }
                }
            }
        }
    }

    internal static void GetAllDirectories(DirectoryInfo root, ConcurrentQueue<DirectoryInfo> values)
    {
        foreach (var di in root.GetDirectories())
        {
            GetAllDirectories(di, values);
            values.Enqueue(di);
        }
    }
}
+1

( ). , , , , , - ( , - , , , , !).

. / concurrency, .

-, IAbstract, /, , -.

Also, if you are comfortable, you can also look at the latest Async CTP1 DataFlow data library, which has the latest support for this template using the Parallel Library. Alternatively, you can use BlockingCollectionto implement this template.

Stackoverflow also has questions revolving around your question with some excellent answers. Just search for producer-consumer to read them.

0
source

More articles:


All Articles