Geek asks and answers

Is a C extension possible for multi-threaded Python performance improvements?

I heard about the Python GIL issue, which said that only one Python thread can execute Python bytecode at a time in a multi-core machine. So a multithreaded Python program is not a good idea.

I am wondering if I can write a C extension that creates pthread, which potentially loads the performance of my program. I mean, I am creating a thread in my C extension and hope that it will work in parallel with the main Python thread.

My guess is that the main Python thread will do more with IO; and pthread in my C extension will spend most of the time computing. The main Python thread communicates with the thread in the C extension using a queue (for example, a consumer-producer model).

Is there any difference between multi-threaded with Python extension and C?

+5
source share
3 answers

To answer your original question:

, C GIL, API Python GIL. , Python, GIL . , API C, ctypes C ( pthreads, ), Cython C Python - .

+1

Python C , .

, . . interprocess IO , , C. , , .

+1

If one thread controls the CPU and the other works with IO, I don't see a problem.

A linked IO thread will invoke IO routines that typically issue a GIL in their actions, effectively letting another thread run.

So give the “simple” solution a try and switch only if it really doesn't work the way you want it to.

0
source

More articles:


All Articles