Skip to content

Commit 2c834b0

Browse files
phoflphofl
authored
Merge branch 'main' into cow_fillna_chained
2 parents c9f948a + 4da9cb6 commit 2c834b0

File tree

196 files changed

+6892
-4510
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

196 files changed

+6892
-4510
lines changed

‎.github/actions/build_pandas/action.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,13 @@ runs:
1414
micromamba list
1515
shell: bash -el {0}
1616

17+
- name: Uninstall existing Pandas installation
18+
run: |
19+
if pip list | grep -q ^pandas; then
20+
pip uninstall -y pandas || true
21+
fi
22+
shell: bash -el {0}
23+
1724
- name: Build Pandas
1825
run: |
1926
if [[ ${{ inputs.editable }} == "true" ]]; then

‎.github/workflows/unit-tests.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ jobs:
8181
ENV_FILE: ci/deps/${{ matrix.env_file }}
8282
PATTERN: ${{ matrix.pattern }}
8383
EXTRA_APT: ${{ matrix.extra_apt || '' }}
84-
LANG: ${{ matrix.lang || '' }}
84+
LANG: ${{ matrix.lang || 'C.UTF-8' }}
8585
LC_ALL: ${{ matrix.lc_all || '' }}
8686
PANDAS_COPY_ON_WRITE: ${{ matrix.pandas_copy_on_write || '0' }}
8787
PANDAS_CI: ${{ matrix.pandas_ci || '1' }}
@@ -122,7 +122,7 @@ jobs:
122122
- 5432:5432
123123

124124
moto:
125-
image: motoserver/moto:4.1.4
125+
image: motoserver/moto:4.1.12
126126
env:
127127
AWS_ACCESS_KEY_ID: foobar_key
128128
AWS_SECRET_ACCESS_KEY: foobar_secret
@@ -227,7 +227,7 @@ jobs:
227227
fi
228228
- name: Build environment and Run Tests
229229
run: |
230-
/opt/python/cp39-cp39/bin/python -m venv ~/virtualenvs/pandas-dev
230+
/opt/python/cp311-cp311/bin/python -m venv ~/virtualenvs/pandas-dev
231231
. ~/virtualenvs/pandas-dev/bin/activate
232232
python -m pip install -U pip wheel setuptools meson[ninja]==1.0.1 meson-python==0.13.1
233233
python -m pip install --no-cache-dir versioneer[toml] cython numpy python-dateutil pytz pytest>=7.3.2 pytest-xdist>=2.2.0 pytest-asyncio>=0.17 hypothesis>=6.46.1
@@ -265,7 +265,7 @@ jobs:
265265
apk add musl-locales
266266
- name: Build environment
267267
run: |
268-
/opt/python/cp39-cp39/bin/python -m venv ~/virtualenvs/pandas-dev
268+
/opt/python/cp311-cp311/bin/python -m venv ~/virtualenvs/pandas-dev
269269
. ~/virtualenvs/pandas-dev/bin/activate
270270
python -m pip install -U pip wheel setuptools meson-python==0.13.1 meson[ninja]==1.0.1
271271
python -m pip install --no-cache-dir versioneer[toml] cython numpy python-dateutil pytz pytest>=7.3.2 pytest-xdist>=2.2.0 pytest-asyncio>=0.17 hypothesis>=6.46.1

‎.pre-commit-config.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -339,6 +339,11 @@ repos:
339339
language: python
340340
entry: python scripts/validate_unwanted_patterns.py --validation-type="strings_with_wrong_placed_whitespace"
341341
types_or: [python, cython]
342+
- id: unwanted-patterns-nodefault-used-not-only-for-typing
343+
name: Check that `pandas._libs.lib.NoDefault` is used only for typing
344+
language: python
345+
entry: python scripts/validate_unwanted_patterns.py --validation-type="nodefault_used_not_only_for_typing"
346+
types: [python]
342347
- id: use-pd_array-in-core
343348
name: Import pandas.array as pd_array in core
344349
language: python

‎ci/code_checks.sh

Lines changed: 1 addition & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -21,23 +21,6 @@ BASE_DIR="$(dirname $0)/.."
2121
RET=0
2222
CHECK=$1
2323

24-
function invgrep {
25-
# grep with inverse exist status and formatting for azure-pipelines
26-
#
27-
# This function works exactly as grep, but with opposite exit status:
28-
# - 0 (success) when no patterns are found
29-
# - 1 (fail) when the patterns are found
30-
#
31-
# This is useful for the CI, as we want to fail if one of the patterns
32-
# that we want to avoid is found by grep.
33-
grep -n "$@" | sed "s/^/$INVGREP_PREPEND/" | sed "s/$/$INVGREP_APPEND/" ; EXIT_STATUS=${PIPESTATUS[0]}
34-
return $((! $EXIT_STATUS))
35-
}
36-
37-
if [[ "$GITHUB_ACTIONS" == "true" ]]; then
38-
INVGREP_PREPEND="##[error]"
39-
fi
40-
4124
### CODE ###
4225
if [[ -z "$CHECK" || "$CHECK" == "code" ]]; then
4326

@@ -105,41 +88,7 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
10588
pandas.errors.UnsupportedFunctionCall \
10689
pandas.test \
10790
pandas.NaT \
108-
pandas.arrays.TimedeltaArray \
109-
pandas.Period.asfreq \
110-
pandas.Period.now \
111-
pandas.arrays.PeriodArray \
112-
pandas.CategoricalDtype.categories \
113-
pandas.CategoricalDtype.ordered \
114-
pandas.Categorical.dtype \
115-
pandas.Categorical.categories \
116-
pandas.Categorical.ordered \
117-
pandas.Categorical.codes \
118-
pandas.Categorical.__array__ \
119-
pandas.SparseDtype \
120-
pandas.DatetimeTZDtype.unit \
121-
pandas.DatetimeTZDtype.tz \
122-
pandas.PeriodDtype.freq \
123-
pandas.IntervalDtype.subtype \
124-
pandas_dtype \
125-
pandas.api.types.is_bool \
126-
pandas.api.types.is_complex \
127-
pandas.api.types.is_float \
128-
pandas.api.types.is_integer \
129-
pandas.api.types.pandas_dtype \
130-
pandas.read_clipboard \
131-
pandas.ExcelFile \
132-
pandas.ExcelFile.parse \
133-
pandas.DataFrame.to_html \
13491
pandas.io.formats.style.Styler.to_html \
135-
pandas.HDFStore.put \
136-
pandas.HDFStore.append \
137-
pandas.HDFStore.get \
138-
pandas.HDFStore.select \
139-
pandas.HDFStore.info \
140-
pandas.HDFStore.keys \
141-
pandas.HDFStore.groups \
142-
pandas.HDFStore.walk \
14392
pandas.read_feather \
14493
pandas.DataFrame.to_feather \
14594
pandas.read_parquet \
@@ -152,30 +101,6 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
152101
pandas.io.stata.StataReader.value_labels \
153102
pandas.io.stata.StataReader.variable_labels \
154103
pandas.io.stata.StataWriter.write_file \
155-
pandas.core.resample.Resampler.__iter__ \
156-
pandas.core.resample.Resampler.groups \
157-
pandas.core.resample.Resampler.indices \
158-
pandas.core.resample.Resampler.get_group \
159-
pandas.core.resample.Resampler.ffill \
160-
pandas.core.resample.Resampler.asfreq \
161-
pandas.core.resample.Resampler.count \
162-
pandas.core.resample.Resampler.nunique \
163-
pandas.core.resample.Resampler.max \
164-
pandas.core.resample.Resampler.mean \
165-
pandas.core.resample.Resampler.median \
166-
pandas.core.resample.Resampler.min \
167-
pandas.core.resample.Resampler.ohlc \
168-
pandas.core.resample.Resampler.prod \
169-
pandas.core.resample.Resampler.size \
170-
pandas.core.resample.Resampler.sem \
171-
pandas.core.resample.Resampler.std \
172-
pandas.core.resample.Resampler.sum \
173-
pandas.core.resample.Resampler.var \
174-
pandas.core.resample.Resampler.quantile \
175-
pandas.describe_option \
176-
pandas.reset_option \
177-
pandas.get_option \
178-
pandas.set_option \
179104
pandas.plotting.deregister_matplotlib_converters \
180105
pandas.plotting.plot_params \
181106
pandas.plotting.register_matplotlib_converters \
@@ -184,32 +109,7 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
184109
pandas.util.hash_pandas_object \
185110
pandas_object \
186111
pandas.api.interchange.from_dataframe \
187-
pandas.Index.asof_locs \
188-
pandas.Index.get_slice_bound \
189-
pandas.RangeIndex \
190-
pandas.RangeIndex.start \
191-
pandas.RangeIndex.stop \
192-
pandas.RangeIndex.step \
193-
pandas.RangeIndex.from_range \
194-
pandas.CategoricalIndex.codes \
195-
pandas.CategoricalIndex.categories \
196-
pandas.CategoricalIndex.ordered \
197-
pandas.CategoricalIndex.reorder_categories \
198-
pandas.CategoricalIndex.set_categories \
199-
pandas.CategoricalIndex.as_ordered \
200-
pandas.CategoricalIndex.as_unordered \
201-
pandas.CategoricalIndex.equals \
202-
pandas.IntervalIndex.values \
203-
pandas.IntervalIndex.to_tuples \
204-
pandas.MultiIndex.dtypes \
205-
pandas.MultiIndex.drop \
206112
pandas.DatetimeIndex.snap \
207-
pandas.DatetimeIndex.as_unit \
208-
pandas.DatetimeIndex.to_pydatetime \
209-
pandas.DatetimeIndex.to_series \
210-
pandas.DatetimeIndex.mean \
211-
pandas.DatetimeIndex.std \
212-
pandas.TimedeltaIndex \
213113
pandas.core.window.rolling.Rolling.max \
214114
pandas.core.window.rolling.Rolling.cov \
215115
pandas.core.window.rolling.Rolling.skew \
@@ -270,6 +170,7 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
270170
pandas.api.extensions.ExtensionArray.factorize \
271171
pandas.api.extensions.ExtensionArray.fillna \
272172
pandas.api.extensions.ExtensionArray.insert \
173+
pandas.api.extensions.ExtensionArray.interpolate \
273174
pandas.api.extensions.ExtensionArray.isin \
274175
pandas.api.extensions.ExtensionArray.isna \
275176
pandas.api.extensions.ExtensionArray.ravel \

‎ci/deps/actions-310.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ dependencies:
1515
- pytest-cov
1616
- pytest-xdist>=2.2.0
1717
- pytest-asyncio>=0.17.0
18+
- pytest-localserver>=0.7.1
1819
- boto3
1920

2021
# required dependencies

‎ci/deps/actions-311-downstream_compat.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ dependencies:
1616
- pytest-cov
1717
- pytest-xdist>=2.2.0
1818
- pytest-asyncio>=0.17.0
19+
- pytest-localserver>=0.7.1
1920
- boto3
2021

2122
# required dependencies

‎ci/deps/actions-311.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ dependencies:
1515
- pytest-cov
1616
- pytest-xdist>=2.2.0
1717
- pytest-asyncio>=0.17.0
18+
- pytest-localserver>=0.7.1
1819
- boto3
1920

2021
# required dependencies

‎ci/deps/actions-39-minimum_versions.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ dependencies:
1717
- pytest-cov
1818
- pytest-xdist>=2.2.0
1919
- pytest-asyncio>=0.17.0
20+
- pytest-localserver>=0.7.1
2021
- boto3
2122

2223
# required dependencies

‎ci/deps/actions-39.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ dependencies:
1515
- pytest-cov
1616
- pytest-xdist>=2.2.0
1717
- pytest-asyncio>=0.17.0
18+
- pytest-localserver>=0.7.1
1819
- boto3
1920

2021
# required dependencies

‎ci/deps/circle-310-arm64.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ dependencies:
1515
- pytest-cov
1616
- pytest-xdist>=2.2.0
1717
- pytest-asyncio>=0.17.0
18+
- pytest-localserver>=0.7.1
1819
- boto3
1920

2021
# required dependencies

‎ci/run_tests.sh

Lines changed: 1 addition & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -8,19 +8,9 @@ export PYTHONHASHSEED=$(python -c 'import random; print(random.randint(1, 429496
88
# May help reproduce flaky CI builds if set in subsequent runs
99
echo PYTHONHASHSEED=$PYTHONHASHSEED
1010

11-
if [[ "not network" == *"$PATTERN"* ]]; then
12-
export http_proxy=http://1.2.3.4 https_proxy=http://1.2.3.4;
13-
fi
14-
1511
COVERAGE="-s --cov=pandas --cov-report=xml --cov-append --cov-config=pyproject.toml"
1612

17-
# If no X server is found, we use xvfb to emulate it
18-
if [[ $(uname) == "Linux" && -z $DISPLAY ]]; then
19-
export DISPLAY=":0"
20-
XVFB="xvfb-run "
21-
fi
22-
23-
PYTEST_CMD="MESONPY_EDITABLE_VERBOSE=1 ${XVFB}pytest -r fEs -n $PYTEST_WORKERS --dist=loadfile $TEST_ARGS $COVERAGE $PYTEST_TARGET"
13+
PYTEST_CMD="MESONPY_EDITABLE_VERBOSE=1 pytest -r fEs -n $PYTEST_WORKERS --dist=loadfile $TEST_ARGS $COVERAGE $PYTEST_TARGET"
2414

2515
if [[ "$PATTERN" ]]; then
2616
PYTEST_CMD="$PYTEST_CMD -m \"$PATTERN\""

‎doc/source/conf.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -240,7 +240,7 @@
240240
"footer_start": ["pandas_footer", "sphinx-version"],
241241
"github_url": "https://github.com/pandas-dev/pandas",
242242
"twitter_url": "https://twitter.com/pandas_dev",
243-
"analytics": {"google_analytics_id": "UA-27880019-2"},
243+
"analytics": {"google_analytics_id": "G-5RE31C1RNW"},
244244
"logo": {"image_dark": "https://pandas.pydata.org/static/img/pandas_white.svg"},
245245
"navbar_end": ["version-switcher", "theme-switcher", "navbar-icon-links"],
246246
"switcher": {

‎doc/source/development/contributing_codebase.rst

Lines changed: 10 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -612,23 +612,17 @@ deleted when the context block is exited.
612612
Testing involving network connectivity
613613
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
614614

615-
It is highly discouraged to add a test that connects to the internet due to flakiness of network connections and
616-
lack of ownership of the server that is being connected to. If network connectivity is absolutely required, use the
617-
``tm.network`` decorator.
615+
A unit test should not access a public data set over the internet due to flakiness of network connections and
616+
lack of ownership of the server that is being connected to. To mock this interaction, use the ``httpserver`` fixture from the
617+
`pytest-localserver plugin. <https://github.com/pytest-dev/pytest-localserver>`_ with synthetic data.
618618

619619
.. code-block:: python
620620
621-
@tm.network # noqa
622-
def test_network():
623-
result = package.call_to_internet()
624-
625-
If the test requires data from a specific website, specify ``check_before_test=True`` and the site in the decorator.
626-
627-
.. code-block:: python
628-
629-
@tm.network("https://www.somespecificsite.com", check_before_test=True)
630-
def test_network():
631-
result = pd.read_html("https://www.somespecificsite.com")
621+
@pytest.mark.network
622+
@pytest.mark.single_cpu
623+
def test_network(httpserver):
624+
httpserver.serve_content(content="content")
625+
result = pd.read_html(httpserver.url)
632626
633627
Example
634628
^^^^^^^
@@ -770,7 +764,7 @@ install pandas) by typing::
770764
your installation is probably fine and you can start contributing!
771765

772766
Often it is worth running only a subset of tests first around your changes before running the
773-
entire suite (tip: you can use the [pandas-coverage app](https://pandas-coverage.herokuapp.com/))
767+
entire suite (tip: you can use the [pandas-coverage app](https://pandas-coverage-12d2130077bc.herokuapp.com/))
774768
to find out which tests hit the lines of code you've modified, and then run only those).
775769

776770
The easiest way to do this is with::
@@ -867,7 +861,7 @@ performance regressions. pandas is in the process of migrating to
867861
`asv benchmarks <https://github.com/airspeed-velocity/asv>`__
868862
to enable easy monitoring of the performance of critical pandas operations.
869863
These benchmarks are all found in the ``pandas/asv_bench`` directory, and the
870-
test results can be found `here <https://pandas.pydata.org/speed/pandas/>`__.
864+
test results can be found `here <https://asv-runner.github.io/asv-collection/pandas>`__.
871865

872866
To use all features of asv, you will need either ``conda`` or
873867
``virtualenv``. For more details please check the `asv installation

‎doc/source/reference/extensions.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ objects.
5151
api.extensions.ExtensionArray.factorize
5252
api.extensions.ExtensionArray.fillna
5353
api.extensions.ExtensionArray.insert
54+
api.extensions.ExtensionArray.interpolate
5455
api.extensions.ExtensionArray.isin
5556
api.extensions.ExtensionArray.isna
5657
api.extensions.ExtensionArray.ravel

‎doc/source/user_guide/copy_on_write.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,7 @@ following methods:
211211
- :meth:`DataFrame.astype` / :meth:`Series.astype`
212212
- :meth:`DataFrame.convert_dtypes` / :meth:`Series.convert_dtypes`
213213
- :meth:`DataFrame.join`
214+
- :meth:`DataFrame.eval`
214215
- :func:`concat`
215216
- :func:`merge`
216217

‎doc/source/user_guide/io.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2664,7 +2664,7 @@ Links can be extracted from cells along with the text using ``extract_links="all
26642664
"""
26652665
26662666
df = pd.read_html(
2667-
html_table,
2667+
StringIO(html_table),
26682668
extract_links="all"
26692669
)[0]
26702670
df

‎doc/source/whatsnew/v0.10.0.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -181,7 +181,7 @@ labeled the aggregated group with the end of the interval: the next day).
181181
``X0``, ``X1``, ...) can be reproduced by specifying ``prefix='X'``:
182182

183183
.. ipython:: python
184-
:okwarning:
184+
:okexcept:
185185
186186
import io
187187

‎doc/source/whatsnew/v0.24.0.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -286,7 +286,7 @@ value. (:issue:`17054`)
286286

287287
.. ipython:: python
288288
289-
result = pd.read_html("""
289+
result = pd.read_html(StringIO("""
290290
<table>
291291
<thead>
292292
<tr>
@@ -298,7 +298,7 @@ value. (:issue:`17054`)
298298
<td colspan="2">1</td><td>2</td>
299299
</tr>
300300
</tbody>
301-
</table>""")
301+
</table>"""))
302302
303303
*Previous behavior*:
304304

0 commit comments

Comments
 (0)