Skip to content

BUG: HDFStore unable to create colindex w/o error thrown #34983

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.1.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1027,6 +1027,7 @@ I/O
- :meth:`HDFStore.keys` has now an optional `include` parameter that allows the retrieval of all native HDF5 table names (:issue:`29916`)
- Bug in :meth:`read_excel` for ODS files removes 0.0 values (:issue:`27222`)
- Bug in :meth:`ujson.encode` was raising an `OverflowError` with numbers larger than sys.maxsize (:issue: `34395`)
- Bug in :meth:`~HDFStore.create_table` now raises an error when `column` argument was not specified in `data_columns` on input (:issue:`28156`)

Plotting
^^^^^^^^
Expand Down
8 changes: 7 additions & 1 deletion pandas/io/pytables.py
Original file line number Diff line number Diff line change
Expand Up @@ -3562,7 +3562,6 @@ def create_index(self, columns=None, optlevel=None, kind: Optional[str] = None):
for c in columns:
v = getattr(table.cols, c, None)
if v is not None:

# remove the index if the kind/optlevel have changed
if v.is_indexed:
index = v.index
Expand Down Expand Up @@ -3590,6 +3589,13 @@ def create_index(self, columns=None, optlevel=None, kind: Optional[str] = None):
"data_columns when initializing the table."
)
v.create_index(**kw)
elif c in self.non_index_axes[0][1]:
# GH 28156
raise AttributeError(
f"column {c} is not a data_column.\n"
f"In order to read column {c} you must reload the dataframe \n"
f"into HDFStore and include {c} with the data_columns argument."
)

def _read_axes(
self, where, start: Optional[int] = None, stop: Optional[int] = None
Expand Down
10 changes: 10 additions & 0 deletions pandas/tests/io/pytables/test_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -1708,11 +1708,16 @@ def col(t, column):
df = tm.makeTimeDataFrame()
df["string"] = "foo"
df["string2"] = "bar"
df["string3"] = "foobar"
store.append("f", df, data_columns=["string", "string2"])
assert col("f", "index").is_indexed is True
assert col("f", "string").is_indexed is True
assert col("f", "string2").is_indexed is True

msg = "'Cols' object has no attribute 'string3'"
with pytest.raises(AttributeError, match=msg):
col("f", "string3").is_indexed

# specify index=columns
store.append(
"f2", df, index=["string"], data_columns=["string", "string2"]
Expand All @@ -1727,6 +1732,11 @@ def col(t, column):
with pytest.raises(TypeError):
store.create_table_index("f2")

# try to index a col which isn't a data_column
# GH 28156
with pytest.raises(AttributeError):
store.create_table_index("f", columns=["string3"])

def test_append_hierarchical(self, setup_path):
index = MultiIndex(
levels=[["foo", "bar", "baz", "qux"], ["one", "two", "three"]],
Expand Down