Description
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
(optional) I have confirmed this bug exists on the master branch of pandas.
Code Sample, a copy-pastable example
In[43]: df = pd.DataFrame({'a':[[1, 2, 3]]})
In[44]: df
Out[44]:
a
0 [1, 2, 3]
In[45]: df.to_hdf('sample.hdf', 'test')
/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/pandas/core/generic.py:2449: PerformanceWarning:
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->block0_values] [items->Index(['a'], dtype='object')]
encoding=encoding,
In[46]: df.to_hdf('sample.hdf', 'test')
/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/pandas/core/generic.py:2449: PerformanceWarning:
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->block0_values] [items->Index(['a'], dtype='object')]
encoding=encoding,
In[47]: df.to_hdf('sample.hdf', 'test', format='table')
Traceback (most recent call last):
File "/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3418, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-47-13d7ad5be031>", line 1, in <module>
df.to_hdf('sample.hdf', 'test', format='table')
File "/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/pandas/core/generic.py", line 2449, in to_hdf
encoding=encoding,
File "/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/pandas/io/pytables.py", line 270, in to_hdf
f(store)
File "/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/pandas/io/pytables.py", line 262, in <lambda>
encoding=encoding,
File "/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/pandas/io/pytables.py", line 1129, in put
track_times=track_times,
File "/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/pandas/io/pytables.py", line 1801, in _write_to_group
track_times=track_times,
File "/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/pandas/io/pytables.py", line 4238, in write
data_columns=data_columns,
File "/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/pandas/io/pytables.py", line 3907, in _create_axes
errors=self.errors,
File "/home/pgali/anaconda3/envs/cudf_dev/lib/python3.7/site-packages/pandas/io/pytables.py", line 4896, in _maybe_convert_for_string_atom
for i in range(len(block.shape[0])):
TypeError: object of type 'int' has no len()Problem description
My dataframe has list-like values and when we pass format=table, I'm not able to write the dataframe to hdf.
Expected Output
Output of pd.show_versions()
INSTALLED VERSIONS
commit : b5958ee
python : 3.7.9.final.0
python-bits : 64
OS : Linux
OS-release : 5.8.0-36-generic
Version : #40~20.04.1-Ubuntu SMP Wed Jan 6 10:15:55 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
pandas : 1.1.5
numpy : 1.19.5
pytz : 2020.5
dateutil : 2.8.1
pip : 20.3.3
setuptools : 49.6.0.post20210108
Cython : 0.29.21
pytest : 6.2.1
hypothesis : 6.0.0
sphinx : 3.4.3
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.19.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : 0.8.5
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : 2.7.2
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 1.0.1
pytables : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : 3.6.1
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : 0.52.0