Skip to content

Commit f7b6378

Browse files
lithomas1lysnikolaouQuLogic
authored
Assorted backports for 2.2.x (#59785)
* Backport PR #59065: ENH: Fix Python 3.13 test failures & enable CI * Remove deprecated plot_date calls (#58484) * Remove deprecated plot_date calls These were deprecated in Matplotlib 3.9. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit c9bc480) * Pick out fastparquet xfails for green CI * pin pytz to fix test_arrays.py * more workflow tweaks for pytz and Python 3.13 * fix typing and tune tests for copy on write * remove WASM stuff * more arm skips * go for green --------- Co-authored-by: Lysandros Nikolaou <[email protected]> Co-authored-by: Elliott Sales de Andrade <[email protected]>
1 parent 2127b42 commit f7b6378

28 files changed

+83
-69
lines changed

‎.circleci/config.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,10 @@ jobs:
1414
steps:
1515
- checkout
1616
- run: .circleci/setup_env.sh
17-
- run: >
18-
PATH=$HOME/miniconda3/envs/pandas-dev/bin:$HOME/miniconda3/condabin:$PATH
19-
LD_PRELOAD=$HOME/miniconda3/envs/pandas-dev/lib/libgomp.so.1:$LD_PRELOAD
17+
- run: |
2018
sudo apt-get update && sudo apt-get install -y libegl1 libopengl0
19+
PATH=$HOME/miniconda3/envs/pandas-dev/bin:$HOME/miniconda3/condabin:$PATH \
20+
LD_PRELOAD=$HOME/miniconda3/envs/pandas-dev/lib/libgomp.so.1:$LD_PRELOAD \
2121
ci/run_tests.sh
2222
linux-musl:
2323
docker:
@@ -35,7 +35,7 @@ jobs:
3535
/opt/python/cp311-cp311/bin/python -m venv ~/virtualenvs/pandas-dev
3636
. ~/virtualenvs/pandas-dev/bin/activate
3737
python -m pip install --no-cache-dir -U pip wheel setuptools meson-python==0.13.1 meson[ninja]==1.2.1
38-
python -m pip install --no-cache-dir versioneer[toml] cython numpy python-dateutil pytz pytest>=7.3.2 pytest-xdist>=2.2.0 hypothesis>=6.46.1
38+
python -m pip install --no-cache-dir versioneer[toml] cython numpy python-dateutil "pytz<2024.2" pytest>=7.3.2 pytest-xdist>=2.2.0 hypothesis>=6.46.1
3939
python -m pip install --no-cache-dir --no-build-isolation -e . --config-settings=setup-args="--werror"
4040
python -m pip list --no-cache-dir
4141
- run: |

‎.github/workflows/unit-tests.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -257,7 +257,7 @@ jobs:
257257
. ~/virtualenvs/pandas-dev/bin/activate
258258
python -m pip install --no-cache-dir -U pip wheel setuptools meson[ninja]==1.2.1 meson-python==0.13.1
259259
python -m pip install numpy --config-settings=setup-args="-Dallow-noblas=true"
260-
python -m pip install --no-cache-dir versioneer[toml] cython python-dateutil pytz pytest>=7.3.2 pytest-xdist>=2.2.0 hypothesis>=6.46.1
260+
python -m pip install --no-cache-dir versioneer[toml] cython python-dateutil "pytz<2024.2" pytest>=7.3.2 pytest-xdist>=2.2.0 hypothesis>=6.46.1
261261
python -m pip install --no-cache-dir --no-build-isolation -e . --config-settings=setup-args="--werror"
262262
python -m pip list --no-cache-dir
263263
export PANDAS_CI=1
@@ -295,7 +295,7 @@ jobs:
295295
/opt/python/cp311-cp311/bin/python -m venv ~/virtualenvs/pandas-dev
296296
. ~/virtualenvs/pandas-dev/bin/activate
297297
python -m pip install --no-cache-dir -U pip wheel setuptools meson-python==0.13.1 meson[ninja]==1.2.1
298-
python -m pip install --no-cache-dir versioneer[toml] cython numpy python-dateutil pytz pytest>=7.3.2 pytest-xdist>=2.2.0 hypothesis>=6.46.1
298+
python -m pip install --no-cache-dir versioneer[toml] cython numpy python-dateutil "pytz<2024.2" pytest>=7.3.2 pytest-xdist>=2.2.0 hypothesis>=6.46.1
299299
python -m pip install --no-cache-dir --no-build-isolation -e . --config-settings=setup-args="--werror"
300300
python -m pip list --no-cache-dir
301301
@@ -329,7 +329,7 @@ jobs:
329329
# To freeze this file, uncomment out the ``if: false`` condition, and migrate the jobs
330330
# to the corresponding posix/windows-macos/sdist etc. workflows.
331331
# Feel free to modify this comment as necessary.
332-
if: false # Uncomment this to freeze the workflow, comment it to unfreeze
332+
# if: false # Uncomment this to freeze the workflow, comment it to unfreeze
333333
defaults:
334334
run:
335335
shell: bash -eou pipefail {0}
@@ -361,15 +361,15 @@ jobs:
361361
- name: Set up Python Dev Version
362362
uses: actions/setup-python@v5
363363
with:
364-
python-version: '3.12-dev'
364+
python-version: '3.13-dev'
365365

366366
- name: Build Environment
367367
run: |
368368
python --version
369369
python -m pip install --upgrade pip setuptools wheel meson[ninja]==1.2.1 meson-python==0.13.1
370370
python -m pip install --pre --extra-index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple numpy
371371
python -m pip install versioneer[toml]
372-
python -m pip install python-dateutil pytz tzdata cython hypothesis>=6.46.1 pytest>=7.3.2 pytest-xdist>=2.2.0 pytest-cov
372+
python -m pip install python-dateutil "pytz<2024.2" tzdata cython hypothesis>=6.46.1 pytest>=7.3.2 pytest-xdist>=2.2.0 pytest-cov
373373
python -m pip install -ve . --no-build-isolation --no-index --no-deps --config-settings=setup-args="--werror"
374374
python -m pip list
375375

‎.github/workflows/wheels.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -170,13 +170,13 @@ jobs:
170170
shell: pwsh
171171
run: |
172172
$TST_CMD = @"
173-
python -m pip install hypothesis>=6.46.1 pytest>=7.3.2 pytest-xdist>=2.2.0;
173+
python -m pip install hypothesis>=6.46.1 pytest>=7.3.2 pytest-xdist>=2.2.0 pytz<2024.2;
174174
python -m pip install `$(Get-Item pandas\wheelhouse\*.whl);
175175
python -c `'import pandas as pd; pd.test(extra_args=[`\"--no-strict-data-files`\", `\"-m not clipboard and not single_cpu and not slow and not network and not db`\"])`';
176176
"@
177177
# add rc to the end of the image name if the Python version is unreleased
178-
docker pull python:${{ matrix.python[1] == '3.12' && '3.12-rc' || format('{0}-windowsservercore', matrix.python[1]) }}
179-
docker run --env PANDAS_CI='1' -v ${PWD}:C:\pandas python:${{ matrix.python[1] == '3.12' && '3.12-rc' || format('{0}-windowsservercore', matrix.python[1]) }} powershell -Command $TST_CMD
178+
docker pull python:${{ matrix.python[1] == '3.13' && '3.13-rc' || format('{0}-windowsservercore', matrix.python[1]) }}
179+
docker run --env PANDAS_CI='1' -v ${PWD}:C:\pandas python:${{ matrix.python[1] == '3.13' && '3.13-rc' || format('{0}-windowsservercore', matrix.python[1]) }} powershell -Command $TST_CMD
180180
181181
- uses: actions/upload-artifact@v4
182182
with:

‎ci/deps/actions-310.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,8 @@ dependencies:
2020
# required dependencies
2121
- python-dateutil
2222
- numpy
23-
- pytz
23+
# pytz 2024.2 timezones cause wrong results
24+
- pytz<2024.2
2425

2526
# optional dependencies
2627
- beautifulsoup4>=4.11.2

‎ci/deps/actions-311-downstream_compat.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,8 @@ dependencies:
2222
# required dependencies
2323
- python-dateutil
2424
- numpy
25-
- pytz
25+
# pytz 2024.2 timezones cause wrong results
26+
- pytz<2024.2
2627

2728
# optional dependencies
2829
- beautifulsoup4>=4.11.2

‎ci/deps/actions-311-numpydev.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ dependencies:
2121

2222
# pandas dependencies
2323
- python-dateutil
24-
- pytz
24+
# pytz 2024.2 timezones cause wrong results
25+
- pytz<2024.2
2526
- pip
2627

2728
- pip:

‎ci/deps/actions-311-pyarrownightly.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,8 @@ dependencies:
1919
# required dependencies
2020
- python-dateutil
2121
- numpy<2
22-
- pytz
22+
# pytz 2024.2 timezones cause wrong results
23+
- pytz<2024.2
2324
- pip
2425

2526
- pip:

‎ci/deps/actions-311.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,8 @@ dependencies:
2020
# required dependencies
2121
- python-dateutil
2222
- numpy
23-
- pytz
23+
# pytz 2024.2 timezones cause wrong results
24+
- pytz<2024.2
2425

2526
# optional dependencies
2627
- beautifulsoup4>=4.11.2

‎ci/deps/actions-312.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,8 @@ dependencies:
2020
# required dependencies
2121
- python-dateutil
2222
- numpy
23-
- pytz
23+
# pytz 2024.2 timezones cause wrong results
24+
- pytz<2024.2
2425

2526
# optional dependencies
2627
- beautifulsoup4>=4.11.2

‎ci/deps/actions-39.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,8 @@ dependencies:
2020
# required dependencies
2121
- python-dateutil
2222
- numpy
23-
- pytz
23+
# pytz 2024.2 timezones cause wrong results
24+
- pytz<2024.2
2425

2526
# optional dependencies
2627
- beautifulsoup4>=4.11.2

‎ci/deps/actions-pypy-39.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ dependencies:
2222
# required
2323
- numpy
2424
- python-dateutil
25+
# pytz 2024.2 timezones cause wrong results
2526
- pytz
2627
- pip:
2728
- tzdata>=2022.7

‎ci/deps/circle-310-arm64.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ dependencies:
2121
# required dependencies
2222
- python-dateutil
2323
- numpy
24-
- pytz
24+
# pytz 2024.2 timezones cause wrong results
25+
- pytz < 2024.2
2526

2627
# optional dependencies
2728
- beautifulsoup4>=4.11.2

‎pandas/_libs/src/vendored/ujson/python/objToJSON.c

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -410,8 +410,8 @@ static void NpyArr_iterBegin(JSOBJ _obj, JSONTypeContext *tc) {
410410
npyarr->type_num = PyArray_DESCR(obj)->type_num;
411411

412412
if (GET_TC(tc)->transpose) {
413-
npyarr->dim = PyArray_DIM(obj, npyarr->ndim);
414-
npyarr->stride = PyArray_STRIDE(obj, npyarr->ndim);
413+
npyarr->dim = PyArray_DIM(obj, (int)npyarr->ndim);
414+
npyarr->stride = PyArray_STRIDE(obj, (int)npyarr->ndim);
415415
npyarr->stridedim = npyarr->ndim;
416416
npyarr->index[npyarr->ndim] = 0;
417417
npyarr->inc = -1;
@@ -452,8 +452,8 @@ static void NpyArrPassThru_iterEnd(JSOBJ obj, JSONTypeContext *tc) {
452452
return;
453453
}
454454
const PyArrayObject *arrayobj = (const PyArrayObject *)npyarr->array;
455-
npyarr->dim = PyArray_DIM(arrayobj, npyarr->stridedim);
456-
npyarr->stride = PyArray_STRIDE(arrayobj, npyarr->stridedim);
455+
npyarr->dim = PyArray_DIM(arrayobj, (int)npyarr->stridedim);
456+
npyarr->stride = PyArray_STRIDE(arrayobj, (int)npyarr->stridedim);
457457
npyarr->dataptr += npyarr->stride;
458458

459459
NpyArr_freeItemValue(obj, tc);
@@ -524,8 +524,8 @@ static int NpyArr_iterNext(JSOBJ _obj, JSONTypeContext *tc) {
524524
}
525525
const PyArrayObject *arrayobj = (const PyArrayObject *)npyarr->array;
526526

527-
npyarr->dim = PyArray_DIM(arrayobj, npyarr->stridedim);
528-
npyarr->stride = PyArray_STRIDE(arrayobj, npyarr->stridedim);
527+
npyarr->dim = PyArray_DIM(arrayobj, (int)npyarr->stridedim);
528+
npyarr->stride = PyArray_STRIDE(arrayobj, (int)npyarr->stridedim);
529529
npyarr->index[npyarr->stridedim] = 0;
530530

531531
((PyObjectEncoder *)tc->encoder)->npyCtxtPassthru = npyarr;

‎pandas/_libs/tslibs/offsets.pyx

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4960,7 +4960,12 @@ cpdef to_offset(freq, bint is_period=False):
49604960
if result is None:
49614961
raise ValueError(INVALID_FREQ_ERR_MSG.format(freq))
49624962

4963-
if is_period and not hasattr(result, "_period_dtype_code"):
4963+
try:
4964+
has_period_dtype_code = hasattr(result, "_period_dtype_code")
4965+
except ValueError:
4966+
has_period_dtype_code = False
4967+
4968+
if is_period and not has_period_dtype_code:
49644969
if isinstance(freq, str):
49654970
raise ValueError(f"{result.name} is not supported as period frequency")
49664971
else:

‎pandas/io/gbq.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
from pandas.util._exceptions import find_stack_level
1212

1313
if TYPE_CHECKING:
14-
import google.auth
14+
from google.auth.credentials import Credentials
1515

1616
from pandas import DataFrame
1717

@@ -37,7 +37,7 @@ def read_gbq(
3737
dialect: str | None = None,
3838
location: str | None = None,
3939
configuration: dict[str, Any] | None = None,
40-
credentials: google.auth.credentials.Credentials | None = None,
40+
credentials: Credentials | None = None,
4141
use_bqstorage_api: bool | None = None,
4242
max_results: int | None = None,
4343
progress_bar_type: str | None = None,
@@ -230,7 +230,7 @@ def to_gbq(
230230
table_schema: list[dict[str, str]] | None = None,
231231
location: str | None = None,
232232
progress_bar: bool = True,
233-
credentials: google.auth.credentials.Credentials | None = None,
233+
credentials: Credentials | None = None,
234234
) -> None:
235235
warnings.warn(
236236
"to_gbq is deprecated and will be removed in a future version. "

‎pandas/tests/groupby/test_groupby.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2816,7 +2816,9 @@ def test_rolling_wrong_param_min_period():
28162816
test_df = DataFrame([name_l, val_l]).T
28172817
test_df.columns = ["name", "val"]
28182818

2819-
result_error_msg = r"__init__\(\) got an unexpected keyword argument 'min_period'"
2819+
result_error_msg = (
2820+
r"^[a-zA-Z._]*\(\) got an unexpected keyword argument 'min_period'"
2821+
)
28202822
with pytest.raises(TypeError, match=result_error_msg):
28212823
test_df.groupby("name")["val"].rolling(window=2, min_period=1).sum()
28222824

‎pandas/tests/indexes/interval/test_interval_tree.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,6 @@ def test_construction_overflow(self):
190190
expected = (50 + np.iinfo(np.int64).max) / 2
191191
assert result == expected
192192

193-
@pytest.mark.xfail(not IS64, reason="GH 23440")
194193
@pytest.mark.parametrize(
195194
"left, right, expected",
196195
[

‎pandas/tests/indexes/test_common.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -452,6 +452,7 @@ def test_sort_values_invalid_na_position(index_with_missing, na_position):
452452
index_with_missing.sort_values(na_position=na_position)
453453

454454

455+
@pytest.mark.fails_arm_wheels
455456
@pytest.mark.filterwarnings(r"ignore:PeriodDtype\[B\] is deprecated:FutureWarning")
456457
@pytest.mark.parametrize("na_position", ["first", "last"])
457458
def test_sort_values_with_missing(index_with_missing, na_position, request):

‎pandas/tests/indexing/interval/test_interval.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
import pytest
33

44
from pandas._libs import index as libindex
5-
from pandas.compat import IS64
65

76
import pandas as pd
87
from pandas import (
@@ -210,7 +209,6 @@ def test_mi_intervalindex_slicing_with_scalar(self):
210209
expected = Series([1, 6, 2, 8, 7], index=expected_index, name="value")
211210
tm.assert_series_equal(result, expected)
212211

213-
@pytest.mark.xfail(not IS64, reason="GH 23440")
214212
@pytest.mark.parametrize(
215213
"base",
216214
[101, 1010],

‎pandas/tests/indexing/interval/test_interval_new.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,6 @@
33
import numpy as np
44
import pytest
55

6-
from pandas.compat import IS64
7-
86
from pandas import (
97
Index,
108
Interval,
@@ -211,7 +209,6 @@ def test_loc_getitem_missing_key_error_message(
211209
obj.loc[[4, 5, 6]]
212210

213211

214-
@pytest.mark.xfail(not IS64, reason="GH 23440")
215212
@pytest.mark.parametrize(
216213
"intervals",
217214
[

‎pandas/tests/io/parser/test_dialect.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ def custom_dialect():
2626
"escapechar": "~",
2727
"delimiter": ":",
2828
"skipinitialspace": False,
29-
"quotechar": "~",
29+
"quotechar": "`",
3030
"quoting": 3,
3131
}
3232
return dialect_name, dialect_kwargs

‎pandas/tests/io/test_common.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -485,7 +485,10 @@ def test_warning_missing_utf_bom(self, encoding, compression_):
485485
df.to_csv(path, compression=compression_, encoding=encoding)
486486

487487
# reading should fail (otherwise we wouldn't need the warning)
488-
msg = r"UTF-\d+ stream does not start with BOM"
488+
msg = (
489+
r"UTF-\d+ stream does not start with BOM|"
490+
r"'utf-\d+' codec can't decode byte"
491+
)
489492
with pytest.raises(UnicodeError, match=msg):
490493
pd.read_csv(path, compression=compression_, encoding=encoding)
491494

‎pandas/tests/io/test_parquet.py

Lines changed: 5 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@
1616
pa_version_under11p0,
1717
pa_version_under13p0,
1818
pa_version_under15p0,
19-
pa_version_under17p0,
2019
)
2120

2221
import pandas as pd
@@ -449,12 +448,8 @@ def test_read_filters(self, engine, tmp_path):
449448
repeat=1,
450449
)
451450

452-
def test_write_index(self, engine, using_copy_on_write, request):
451+
def test_write_index(self, engine):
453452
check_names = engine != "fastparquet"
454-
if using_copy_on_write and engine == "fastparquet":
455-
request.applymarker(
456-
pytest.mark.xfail(reason="fastparquet write into index")
457-
)
458453

459454
df = pd.DataFrame({"A": [1, 2, 3]})
460455
check_round_trip(df, engine)
@@ -1064,9 +1059,6 @@ def test_read_dtype_backend_pyarrow_config_index(self, pa):
10641059
expected=expected,
10651060
)
10661061

1067-
@pytest.mark.xfail(
1068-
pa_version_under17p0, reason="pa.pandas_compat passes 'datetime64' to .astype"
1069-
)
10701062
def test_columns_dtypes_not_invalid(self, pa):
10711063
df = pd.DataFrame({"string": list("abc"), "int": list(range(1, 4))})
10721064

@@ -1314,7 +1306,10 @@ def test_empty_dataframe(self, fp):
13141306
expected = df.copy()
13151307
check_round_trip(df, fp, expected=expected)
13161308

1317-
@pytest.mark.skipif(using_copy_on_write(), reason="fastparquet writes into Index")
1309+
@pytest.mark.xfail(
1310+
_HAVE_FASTPARQUET and Version(fastparquet.__version__) > Version("2022.12"),
1311+
reason="fastparquet bug, see https://github.com/dask/fastparquet/issues/929",
1312+
)
13181313
def test_timezone_aware_index(self, fp, timezone_aware_date_list):
13191314
idx = 5 * [timezone_aware_date_list]
13201315

‎pandas/tests/io/xml/test_xml.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1044,7 +1044,7 @@ def test_utf16_encoding(xml_baby_names, parser):
10441044
UnicodeError,
10451045
match=(
10461046
"UTF-16 stream does not start with BOM|"
1047-
"'utf-16-le' codec can't decode byte"
1047+
"'utf-16(-le)?' codec can't decode byte"
10481048
),
10491049
):
10501050
read_xml(xml_baby_names, encoding="UTF-16", parser=parser)

0 commit comments

Comments
 (0)