Skip to content

Commit fb48a14

Browse files
YikunHyukjinKwon
authored andcommitted
[SPARK-39611][PYTHON][PS] Fix wrong aliases in __array_ufunc__
### What changes were proposed in this pull request? This PR fix the wrong aliases in `__array_ufunc__` ### Why are the changes needed? When running test with numpy 1.23.0 (current latest), hit a bug: `NotImplementedError: pandas-on-Spark objects currently do not support <ufunc 'divide'>.` In `__array_ufunc__` we first call `maybe_dispatch_ufunc_to_dunder_op` to try dunder methods first, and then we try pyspark API. `maybe_dispatch_ufunc_to_dunder_op` is from pandas code. pandas fix a bug pandas-dev/pandas#44822 (comment) pandas-dev/pandas@206b249 when upgrade to numpy 1.23.0, we need to also sync this. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Current CI passed - The exsiting UT `test_series_datetime` already cover this, I also test it in my local env with 1.23.0 ```shell pip install "numpy==1.23.0" python/run-tests --testnames 'pyspark.pandas.tests.test_series_datetime SeriesDateTimeTest.test_arithmetic_op_exceptions' ``` Closes #37078 from Yikun/SPARK-39611. Authored-by: Yikun Jiang <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent 947e271 commit fb48a14

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

python/pyspark/pandas/numpy_compat.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,7 @@ def maybe_dispatch_ufunc_to_dunder_op(
166166
"true_divide": "truediv",
167167
"power": "pow",
168168
"remainder": "mod",
169-
"divide": "div",
169+
"divide": "truediv",
170170
"equal": "eq",
171171
"not_equal": "ne",
172172
"less": "lt",

0 commit comments

Comments
 (0)