Geek asks and answers

How to prevent ndb from put_async () batch call and make it release RPC immediately?

I have a query handler that updates the object, saves it in the data warehouse, then it needs to do some additional work before returning (for example, to sequence the background task and json serialize some results). I want to parallelize this code so that extra work is done while the object is being saved.

Here the code of my handler comes down to:

class FooHandler(webapp2.RequestHandler):
    @ndb.toplevel
    def post(self):
        foo = yield Foo.get_by_id_async(some_id)

        # Do some work with foo

        # Don't yield, as I want to perform the code that follows
        # while foo is being saved to the datastore.
        # I'm in a toplevel, so the handler will not exit as long as
        # this async request is not finished.
        foo.put_async()

        taskqueue.add(...)
        json_result = generate_result()
        self.response.headers["Content-Type"] = "application/json; charset=UTF-8"
        self.response.write(json_result)

However, Appstats shows that RPC datastore.Putis executed in series, after taskqueue.Add:

Appstats screenshot

A little digging through ndb.context.pyshows that the call put_async()ends with adding to AutoBatcherinstead of RPC, which is issued immediately.

, , _put_batcher , toplevel .

, puts , , RPC- , , .

yield foo.put_async(), Appstats, datastore.Put :

2nd Appstats screenshot

, yield put_async() .

ndb.get_context().flush() foo.put_async(), datastore.Put taskqueue.BulkAdd - Appstats.

, : put_async() RPC?

+5
2

. , . , ?

loop - ndb.eventloop.get_event_loop()
while loop.run_idle():
    pass

, ndb/eventloop.py, , - , run0(), RPC. , , :

while loop.current:
    loop.run0()
while loop.run_idle():
    pass

( , , , , , .)

+6

, 100% , :

foo = yield Foo.get_by_id_async(some_id)
future = foo.put_async()
future.done()

ndb , RPC, . foo.put_async(), , ndb ( ) @ndb.toplevel.

future.done() , , .

, :

ndb.get_context().flush()
-2

More articles:


All Articles