Skip to content

[wishlist] Make Git() serializable (i.e. pickleable) #504

Closed
@yarikoptic

Description

@yarikoptic

I decided to become fancy and use some multiprocessing in our datalad, but quickly was bombarded with

  File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
    raise self._value
PicklingError: Can't pickle <class 'git.cmd.AutoInterrupt'>: attribute lookup git.cmd.AutoInterrupt failed
()

so I looked into cmd.py to see that AutoInterupt is actuall a sub-class of the Git class... so something went hairy here and I have decided to reproduce on a more obvious snippet:

$> python -c 'from git import Git; r = Git("/tmp/123"); import pickle; pickle.dumps(r)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/pickle.py", line 1380, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "/usr/lib/python2.7/copy_reg.py", line 77, in _reduce_ex
    raise TypeError("a class that defines __slots__ without "
TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled

so failed to pickle directly but differently... but overall story is that as long as I have a Git object involved anyhow -- can't pickle my beast, can't use multiprocessing :-/

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions