Python 3.2 pickle.load causes EOFError randomly

I meet a very strange mistake. I have a cronjob that runs daily: it opens a pickled file, downloads that file, and then processes the data. At the end of the script, it will save the data in this file for processing the next day.

In most cases, this script works fine, but there is some time (I would say twice a month) when opening this file, an EOFError raises:

Traceback (most recent call last):
  File "read.py", line 11, in <module>
    SellerDictionarycf=pickle.load(SellerDict)
EOFError

I firmly believe that no other process works with this file, and I am 100% sure that I will write data, close this file before reading it.

My snipet code to read the file at the beginning of the script:

SellerDict=open('/home/hostadl/SellerDictab','rb')
SellerDictionaryab=pickle.load(SellerDict)
SellerDict.close()

My snipet code to write the file at the end of the script:

SellerDict=open('/home/hostadl/SellerDictab','wb')
pickle.dump(SellerDictionaryab,SellerDict)
SellerDict.flush()
SellerDict.close()

(, EOFError ), :

python3.2 -m pickletools -o test.txt SellerDictab
Traceback (most recent call last):
  File "/usr/local/lib/python3.2/runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/lib/python3.2/runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.2/pickletools.py", line 2403, in <module>
    args.indentlevel, annotate)
  File "/usr/local/lib/python3.2/pickletools.py", line 1963, in dis
    for opcode, arg, pos in genops(pickle):
  File "/usr/local/lib/python3.2/pickletools.py", line 1874, in genops
    arg = opcode.arg.reader(pickle)
  File "/usr/local/lib/python3.2/pickletools.py", line 474, in read_unicodestring4
    "remain" % (n, len(data)))
ValueError: expected 2254 bytes in a unicodestring4, but only 0 remain

test.txt , :

head test.txt
    0: \x80 PROTO      3
    2: }    EMPTY_DICT
    3: q    BINPUT     0
    5: (    MARK
    6: X        BINUNICODE 'bradey4357604769'
   27: q        BINPUT     1
   29: ]        EMPTY_LIST
   30: q        BINPUT     2
   32: ]        EMPTY_LIST
   33: q        BINPUT     3

tail test.txt
18933166: e                APPENDS    (MARK at 18930621)
18933167: ]            EMPTY_LIST
18933168: r            LONG_BINPUT 174342
18933173: (            MARK
18933174: X                BINUNICODE 'HP PhotoSmart C4480 All-in-One Printer'
18933217: r                LONG_BINPUT 174343
18933222: G                BINFLOAT   45.0
18933231: G                BINFLOAT   84104.0
18933240: X                BINUNICODE 'Salt Lake City,  UT '
18933265: r                LONG_BINPUT 174344

( EOFError) :

head testgood.txt
    0: \x80 PROTO      3
    2: }    EMPTY_DICT
    3: q    BINPUT     0
    5: (    MARK
    6: X        BINUNICODE 'bj8016541577'
   23: q        BINPUT     1
   25: ]        EMPTY_LIST
   26: q        BINPUT     2
   28: (        MARK
   29: ]            EMPTY_LIST

tail testgood.txt
16569368: X                BINUNICODE 'Bought this beautiful dinner set but never opened it, I would love to get rod of it pleas tex with an offer...'
16569483: r                LONG_BINPUT 161843
16569488: J                BININT     20177894
16569493: X                BINUNICODE 'antonio8016497082'
16569515: r                LONG_BINPUT 161844
16569520: e                APPENDS    (MARK at 16569283)
16569521: e            APPENDS    (MARK at 16568508)
16569522: u        SETITEMS   (MARK at 16088830)
16569523: .    STOP

(, )? .

+3
1

, ( - ).

( close/flush):

with open('/home/hostadl/SellerDictab','rb') as SellerDict:
    SellerDictionaryab=pickle.load(SellerDict)

with open('/home/hostadl/SellerDictab','wb') as SellerDict:
    pickle.dump(SellerDictionaryab,SellerDict)
0

All Articles