Geek asks and answers

Run a loop in bash across multiple cores

I have a shell script that contains the following loop.

i=0  
upperlimit=$verylargevariable  
do  
   complexstuff RunManager file $i  
   i= 'expr $i +1'  
done

This script runs on a quad-core processor and accordingly topuses about 15% of each core in a single iteration of the loop. I would like to distribute it across four kernels, so that each iteration of the loop performs complexstufffour times, one for each core, so the resources will be used more efficiently. We are talking about calculations that currently take several hours, so efficiency here is not just good practice. (The output of each iteration is obviously independent of the previous one.)

PS: A host is a server running Cent-OS, if that helps.

+4
source share
2

Ole Tange ( ), , - :

i=0  
upperlimit=$verylargevariable  
do  
   complexstuff RunManager file $i &
   i= 'expr $i + 1'
   complexstuff RunManager file $i &
   i= 'expr $i + 1'
   complexstuff RunManager file $i &
   i= 'expr $i + 1'
   complexstuff RunManager file $i &
   i= 'expr $i + 1'
   wait
done

, 4 bash, (, , ). 4 , , .

+3

GNU Parallel :

seq $verylargevariable | parallel -j150% complexstuff RunManager file

150% 1,5 , , 15%, 100% 4 .

, : http://www.youtube.com/watch?v=OpaiGYxkSuQ

+10

More articles:


All Articles