Geek asks and answers

Is it possible to repeat the iteration of a dictionary modified by another thread without involving exceptions?

I have a dictionary updated by one thread, and in another thread I would like to iterate over its values. I usually use locking, but this code is very performance critical, and I want to avoid this, if at all possible.

A feature of my case is that I do not care about the absolute correctness of the iterator; if it skips records that were deleted after the iteration started, or after that adds those that were added later, that’s fine. I only require that it does not cause any "dictionary size changed during iteration".

Given this relaxed restriction on correctness, is there an effective way to iterate a dictionary without using locks?

Note. I know it keys()is thread safe in Python 2.x, but since this behavior has changed in 3.x, I want to avoid it.

+5
source share
3 answers

There is no personal experience with this, but I read it a while ago: http://www.python.org/dev/peps/pep-3106/

These operations are thread safe only if using them in an unsafe thread can throw an exception but not damage the internal representation.

As in Python 2.x, a dict mutation while iterating over it using an iterator has an undefined effect and in most cases raises a RuntimeError exception. (This is similar to the guarantees provided by the Java Collections Framework.)

+4
source

, , :

with lock:
    values = the_dict.values() # python 2
    # values = list(the_dict.values()) # python 3
for value in values:
    # do stuff

catch RuntimeError, , .

[] . . :

while True:
    try:
        values = list(the_dict.values())
        break
    except RuntimeError:
        pass

.

+3

Two things:

  • Reset the keys in the queue and read it safely.
  • High-performance code should probably not use Python threads.
+2
source

More articles:


All Articles