I want to know if I understood consumer consumer design correctly using ExecutorService and ArrayBlockingQueue. I understand that there are various ways to implement this project, but, I think, in the end it depends on the problem itself.
The problem I had to deal with is this: I have one manufacturer that reads from a large file (6 GB); it is read line by line and converts each line into an object. It places an object in an ArrayBlockingQueue.
Consumers (several) take an object from ArrayBlockingQueue and store it in the database.
Now, obviously, the producer is much faster than the consumer; It takes a few seconds to convert each line into an object, but for consumers it takes longer.
So ... if I want to speed up this process by doing this: I created 2 classes "ProducerThread" and "ConsumerThread", they share ArrayBlockingQueue. A thread that coordinates between 2 looks like this:
@Override
public void run()
{
try{
ArrayBlockingQueue<Ticket> queue = new ArrayBlockingQueue<Ticket>(40);
ExecutorService threadPool = Executors.newFixedThreadPool(8);
threadPool.execute(new SaleConsumerThread("NEW YORK", queue));
threadPool.execute(new SaleConsumerThread("PARIS", queue));
threadPool.execute(new SaleConsumerThread("TEL AVIV", queue));
threadPool.execute(new SaleConsumerThread("HONG KONG", queue));
threadPool.execute(new SaleConsumerThread("LONDON", queue));
threadPool.execute(new SaleConsumerThread("BERLIN", queue));
threadPool.execute(new SaleConsumerThread("AMSTERDAM", queue));
Future producerStatus = threadPool.submit(new SaleProducerThread(progressBar, file, queue));
producerStatus.get();
threadPool.shutdown();
}catch(Exception exp)
{
exp.printStackTrace();
}
}
My questions:
The result, by the way, is quick (consider that the first version was consistent and it took 3 hours), now it takes ~ 40 minutes (but, perhaps, there are opportunities for improvement).
Thanks for any pointer