I wrote a short python script to handle my large fastq files ranging in size from 5 GB to 35 GB. I am running a script on a Linux server with many cores. The script is not written in parallel and takes about 10 minutes to complete for a single file on average.
If I run the same script in multiple files, for example
$ python my_script.py file1 &
$ python my_script.py file2 &
$ python my_script.py file3 &
using the and sign to cancel the process.
Are these scripts running in parallel and can I save some time?
It seems to me wrong, because I use the top command to check the processor usage, and each of them crashes when I added new runs or should not use somewhere around 100%?
So, if they do not work in parallel, is there a way to get them to run them in parallel?
thanks for answers