Description
Code Sample, a copy-pastable example if possible
# Your code here
import portalocker
with portalocker.Lock("/path/to/file.pickle", "rb+", timeout=60) as file:
file.seek(0) # not sure why I had this from before but it shouldn't cause an issue right?
df = pd.read_pickle(file)
# do stuff to df
file.seek(0)
file.truncate() # I think here I was trying to erase the previous file just in case because of the read-mode
file.to_pickle(file)Alternatively, a more concise, self-contained, reproducible example:
import portalocker
import pandas as pd
df = pd.DataFrame({'a':[1,2,3], 'b':[1,2,3]})
path = 'test.pickle'
df.to_pickle(path)
with portalocker.Lock(path, "rb+", timeout=60) as file:
file.seek(0)
df = pd.read_pickle(file)Replace portalocker.Lock with open(path, 'rb+') and you get the same thing.
Problem description
The above used to work. However, having updated to the latest pandas version recently (0.25.1), it now produces an error on the pd.read_pickle, and based on other issues, it will also produce an error on df.to_pickle() if it gets that far. Am I doing it wrong or should I do it differently, or it is just a bug?
The error traceback is:
df = pd.read_pickle(file)
File "/HOME/anaconda3/lib/python3.6/site-packages/pandas/io/pickle.py", line 145, in read_pickle
f, fh = _get_handle(path, "rb", compression=compression, is_text=False)
File "HOME/anaconda3/lib/python3.6/site-packages/pandas/io/common.py", line 392, in _get_handle
raise ValueError(msg)
ValueError: Unrecognized compression type: infer
Expected Output
Output of pd.show_versions()
This didn't work as it couldn't determine the version for pip for some reason (I have pip, I'm not sure what's going on here), but the pandas version I'm running is: 0.25.1
On my other machine, which also has the error:
INSTALLED VERSIONS
commit : None
python : 3.6.8.final.0
python-bits : 64
OS : Linux
OS-release : 4.18.16-041816-generic
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8
pandas : 0.25.1
numpy : 1.14.3
pytz : 2018.4
dateutil : 2.7.3
pip : 19.1.1
setuptools : 39.1.0
Cython : 0.28.2
pytest : 3.5.1
hypothesis : None
sphinx : 1.7.4
blosc : None
feather : None
xlsxwriter : 1.0.4
lxml.etree : 4.2.1
html5lib : 1.0.1
pymysql : None
psycopg2 : None
jinja2 : 2.10
IPython : 6.4.0
pandas_datareader: None
bs4 : 4.6.0
bottleneck : 1.2.1
fastparquet : None
gcsfs : None
lxml.etree : 4.2.1
matplotlib : 3.1.1
numexpr : 2.6.5
odfpy : None
openpyxl : 2.5.3
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : 1.1.0
sqlalchemy : 1.2.7
tables : 3.4.3
xarray : None
xlrd : 1.1.0
xlwt : 1.3.0
xlsxwriter : 1.0.4