Skip to content

BUG: DataFrame.apply axis=1 for str ops with no axis argument #39212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 19, 2021
Merged

BUG: DataFrame.apply axis=1 for str ops with no axis argument #39212

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8a6f790
BUG: DataFrame.apply axis=1 for str ops with no axis argument
rhshadrach Jan 16, 2021
5247e14
whatsnew
rhshadrach Jan 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.3.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -346,6 +346,7 @@ Reshaping
- Bug in :func:`join` over :class:`MultiIndex` returned wrong result, when one of both indexes had only one level (:issue:`36909`)
- :meth:`merge_asof` raises ``ValueError`` instead of cryptic ``TypeError`` in case of non-numerical merge columns (:issue:`29130`)
- Bug in :meth:`DataFrame.join` not assigning values correctly when having :class:`MultiIndex` where at least one dimension is from dtype ``Categorical`` with non-alphabetically sorted categories (:issue:`38502`)
- Bug in :meth:`DataFrame.apply` would give incorrect results when used with a string argument and ``axis=1`` when the axis argument was not supported and now raises a ``ValueError`` instead (:issue:`39211`)
-

Sparse
Expand Down
2 changes: 2 additions & 0 deletions pandas/core/apply.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,8 @@ def maybe_apply_str(self) -> Optional[FrameOrSeriesUnion]:
sig = inspect.getfullargspec(func)
if "axis" in sig.args:
self.kwds["axis"] = self.axis
elif self.axis != 0:
raise ValueError(f"Operation {f} does not support axis=1")
return self.obj._try_aggregate_string_function(f, *self.args, **self.kwds)

def maybe_apply_multiple(self) -> Optional[FrameOrSeriesUnion]:
Expand Down
10 changes: 10 additions & 0 deletions pandas/tests/frame/apply/test_frame_apply.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,16 @@ def test_apply_with_string_funcs(self, request, float_frame, func, args, kwds, h
expected = getattr(float_frame, func)(*args, **kwds)
tm.assert_series_equal(result, expected)

@pytest.mark.parametrize(
"how, args", [("pct_change", ()), ("nsmallest", (1, ["a", "b"])), ("tail", 1)]
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this already raise when a function is passed? ideally can you co-locate these tests.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When function is passed, apply with axis=1 computes the correct result. E.g.

df = DataFrame([[1, 2, 3], [4, 5, 6]])
print(df.apply(lambda x: x.tail(1), axis=1))

# gives:
   2
0  3
1  6

The argument axis=1 is intercepted and then apply_standard is computed on the rows of the frame without an axis argument.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok cool, can you co-locate this test near the others

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is meant by others? Currently located below the other test for str arguments. Tests for axis=1 and tests that raise are fairly spread out in this file, not sure how else to locate this test.

As an aside, I've been wanting to consolidate (and then start to cleanup) the apply/agg/transform tests into pandas.tests.apply. Does that move make sense?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok cool
yeah would love to break up some of these test files more logically

def test_apply_str_axis_1_raises(self, how, args):
# GH 39211 - some ops don't support axis=1
df = DataFrame({"a": [1, 2], "b": [3, 4]})
msg = f"Operation {how} does not support axis=1"
with pytest.raises(ValueError, match=msg):
df.apply(how, axis=1, args=args)

def test_apply_broadcast(self, float_frame, int_frame_const_col):

# scalars
Expand Down