Geek asks and answers

Python threading in web programming

I am facing a potential race condition in a web application:

# get the submissions so far from the cache
submissions = cache.get('user_data')
# add the data from this user to the local dict
submissions[user_id] = submission

# update the cached dict on server
submissions = cache.update('user_data', submissions)

if len(submissions) == some_number:
    ...

The logic is simple: we first extract the shared dictionary stored in the cache of the web server, add the send (delivered by each request to the server) to its local copy, and then update the cached copy, replacing it with an updated local copy. Finally, we do something else if we get a certain amount of data. note that

submissions = cache.update('user_data', submissions)

will return the last copy of the dictionary from the cache, i.e. updated.

Since the server can serve several requests at the same time (each in its own thread), and all these threads access the general dictionary in the cache, as described above, thereby creating potential race conditions.

, - , , , . .

+3
3

, , dict , . , dict. , .

, , -, .

EDIT: ; .

EDIT: Python :

import threading, Queue

Stop = object()

def consumer(real_dict, queue):
    while True:
        try:
            item = queue.get(timeout=100)
            if item == Stop:
                break
            user, submission = item
            real_dict[user] = submission
        except Queue.Empty:
            continue

q = Queue.Queue()
thedict={}

t = threading.Thread(target=consumer, args=(thedict,q,))
t.start()

:

>>> thedict
{}
>>> q.put(('foo', 'bar'))
>>> thedict
{'foo': 'bar'}
>>> q.put(Stop)
>>> q.put(('baz', 'bar'))
>>> thedict
{'foo': 'bar'}
+2

, - . . , , , (, sub ):

Thread A        Thread B        Cache
--------------------------------------------
                                [A]=P, [B]=Q
sub = get()
   [A]=P, [B]=Q
>>>> suspend
                sub = get()
                   [A]=P, [B]=Q
                sub[B] = Y
                   [A]=P, [B]=Y
                update(sub)
                                [A]=P, [B]=Y
                >>>> suspend
sub[A] = X
   [A]=X, [B]=Q
update(sub)
                                [A]=X, [B]=Q         !!!!!!!!

-, , . , A , .

, . , , , , .

+1

, , -, .

, . , , , . , , .

: . , . , , , , .

, , Redis, Redis [1]. , Redis , , memcache ( ).

[1] Note. I just found this link as a result of a quick Google search and did not check it. I do not vouch for its correctness.

+1
source

More articles:


All Articles