Geek asks and answers

Python dict set min_size

I am parsing hundreds of millions of JSON records and storing the corresponding components from each in dict. The problem is that due to the number of records being processed, python has to increase the size of the hash table several times dict. This results in a lot of data to be rephrased. A huge amount of paraphrasing seems to be worth a lot of time. So I wonder if there is a way to set the minimum size in the hash table dictto minimize the number of resize operations.

I read this about python optimization dictfrom the answer to this question , but cannot find how to change the initial size of the hash table dict. If anyone can help me, I will be very grateful.

thank

+5
source share
1 answer

If you do this:

a = dict.fromkeys(range(n))

it will force the size of the dictionary to place n elements. This is pretty quick after that, but it takes 3 seconds.

+2
source

More articles:


All Articles