python – Save Numpy Array using Pickle

python – Save Numpy Array using Pickle

You should use numpy.save and numpy.load.

I have no problems using pickle:

In [126]: arr = np.zeros((1000,2))
In [127]: with open(test.pkl,wb) as f:
     ...:     pickle.dump(arr, f)
     ...:     
In [128]: with open(test.pkl,rb) as f:
     ...:     x = pickle.load(f)
     ...:     print(x.shape)
     ...:     
     ...:     
(1000, 2)

pickle and np.save/load have a deep reciprocity. Like I can load this pickle with np.load:

In [129]: np.load(test.pkl).shape
Out[129]: (1000, 2)

If I open the pickle file in the wrong I do get your error:

In [130]: with open(test.pkl,wb) as f:
     ...:     x = pickle.load(f)
     ...:     print(x.shape)
     ...:    
UnsupportedOperation: read

But that shouldnt be surprising – you cant read a freshly opened write file. It will be empty.

np.save/load is the usual pair for writing numpy arrays. But pickle uses save to serialize arrays, and save uses pickle to serialize non-array objects (in the array). Resulting file sizes are similar. Curiously in timings the pickle version is faster.

python – Save Numpy Array using Pickle

Its been a bit but if youre finding this, Pickle completes in a fraction of the time.

with open(filename,wb) as f: pickle.dump(arrayname, f)

with open(filename,rb) as f: arrayname1 = pickle.load(f)

numpy.array_equal(arrayname,arrayname1) #sanity check

On the other hand, by default numpy compress took my 5.2GB down to .4GB and Pickle went to 1.7GB.

Related posts on python :

• macos – Python: ImportError: lxml not found, please install it
• python – LinAlgError: Last 2 dimensions of the array must be square
• python mpl_toolkits installation issue
• python – AttributeError: NoneType object has no attribute format
• python – No pyvenv.cfg file
• sdl – How to suppress console output in Python?
• python – Bad operand type for unary +: ‘str’
• python – ImportError: No module named encodings