Post by Peng YuPost by Robert KernPost by Peng YuI use pickle to dump a long list. But when I load it, I only want to
load the first a few elements in the list. I am wondering if there is
a easy way to do so? Thank you!
Not by pickling the list. However, you can concatenate pickles, so you could
just pickle each item from the list in order to the same file and only
unpickle the first few.
In [1]: import cPickle
In [2]: from cStringIO import StringIO
In [3]: very_long_list = range(10)
In [4]: f = StringIO()
...: cPickle.dump(item, f)
In [6]: f.seek(0,0)
In [7]: cPickle.load(f)
Out[7]: 0
In [8]: cPickle.load(f)
Out[8]: 1
In [9]: cPickle.load(f)
Out[9]: 2
In [10]: cPickle.load(f)
Out[10]: 3
How do I determine if I have loaded all the elements? I use the
following code. I'm wondering if there is any better solution than
this.
###############
import pickle
alist = [1, 2.0, 3, 4+6j]
output=open('serialize_list.output/serialize_list.pkl', 'wb')
pickle.dump(e, output)
output.close()
input=open('serialize_list.output/serialize_list.pkl', 'rb')
e = pickle.load(input)
print e
pass
You could write out an integer with the number of expected elements at the very
beginning.
In [1]: import cPickle
In [2]: alist = [1, 2.0, 3, 4+6j]
In [3]: output = open('foo.pkl', 'wb')
In [4]: cPickle.dump(len(alist), output)
In [5]: for item in alist:
...: cPickle.dump(item, output)
...:
...:
In [6]: output.close()
In [7]: input = open('foo.pkl', 'rb')
In [8]: n = cPickle.load(input)
In [9]: n
Out[9]: 4
In [10]: for i in range(n):
....: print cPickle.load(input)
....:
....:
1
2.0
3
(4+6j)
In [11]: assert input.read(1) == ''
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco