Geek asks and answers

Python Multiprocessing: the fastest way to notify an event of all processes?

I am doing a monte carlo simulation with multiple processes using the python multiprocessing library. Processes basically guess a certain object, and if it satisfies a certain condition, it is added to the general list. My calculation is completed if this list meets some condition.

My current code is as follows: (pseudocode without inconsequential details)

mgr = Manager()
ns = mgr.Namespace()
ns.mylist = []
ns.othersharedstuff = x
killsig = mgr.Event()
processes = [ MyProcess(ns, killsig) for _ in range(8) ]
for p in processes: p.start()
for p in processes: p.join()
get data from ns.mylist()

def MyProcess.run(self):
    localdata = y
    while not killsig.is_set():
        x = guessObject()
        if x.meetsCondition():
            add x to ns.mylist and put local data into ns()
            if ns.mylist meets condition:
                killsig.set()
    put local data into ns()

When I replace 'while not killsig.is_set ():' with 'while True:', the speed of my simulation increases by about 25%! (except that it certainly does not end)

, ? , , -, process.terminate(), .

+3
1

, ? - :

ns.othersharedstuff = x
killsig = mgr.Event()
processes = [ MyProcess(ns, killsig) for _ in range(8) ]
for p in processes: p.start()
while not killsig.isSet():
    time.sleep(0.01) # 10 milliseconds 
for p in processes: p.terminate()
get data from ns.mylist()

while while true:

+3

More articles:


All Articles