Skip to content

CI: Unpin MyPy #36012

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Sep 1, 2020
Merged

CI: Unpin MyPy #36012

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
be7b516
CI: Unpin MyPy
simonjayhawkins Aug 31, 2020
700205d
Merge remote-tracking branch 'upstream/master' into unpin-mypy
simonjayhawkins Sep 1, 2020
5f537a1
pin mypy=0.782
simonjayhawkins Sep 1, 2020
9d4f36b
update setup.cfg for io.excel._xlsxwriter fixes
simonjayhawkins Sep 1, 2020
3c11ad5
update .pre-commit-config.yaml
simonjayhawkins Sep 1, 2020
709c2fc
Merge remote-tracking branch 'upstream/master' into unpin-mypy
simonjayhawkins Sep 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion environment.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ dependencies:
- flake8-comprehensions>=3.1.0 # used by flake8, linting of unnecessary comprehensions
- flake8-rst>=0.6.0,<=0.7.0 # linting of code blocks in rst files
- isort>=5.2.1 # check that imports are in the right order
- mypy=0.730
- mypy=0.782
- pycodestyle # used by flake8

# documentation
Expand Down
4 changes: 1 addition & 3 deletions pandas/_config/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -460,9 +460,7 @@ def register_option(
path = key.split(".")

for k in path:
# NOTE: tokenize.Name is not a public constant
# error: Module has no attribute "Name" [attr-defined]
if not re.match("^" + tokenize.Name + "$", k): # type: ignore[attr-defined]
if not re.match("^" + tokenize.Name + "$", k):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

latest typeshed (bundled with mypy) exposes undocumented attributes

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this mean theres a lower bound for what mypy we want?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that in general it is recommended to pin mypy see https://mypy.readthedocs.io/en/stable/existing_code.html?highlight=pin#continuous-integration

does this mean theres a lower bound for what mypy we want?

with warn_unused_ignores, yes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I could change to mypy=0.782 (best practice) or mypy>=0.782 (see #27568 (comment))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commented above, prefer either of these to no version

raise ValueError(f"{k} is not a valid identifier")
if keyword.iskeyword(k):
raise ValueError(f"{k} is a python keyword")
Expand Down
7 changes: 6 additions & 1 deletion pandas/core/common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from datetime import datetime, timedelta
from functools import partial
import inspect
from typing import Any, Collection, Iterable, Iterator, List, Union
from typing import Any, Collection, Iterable, Iterator, List, Union, cast
import warnings

import numpy as np
Expand Down Expand Up @@ -277,6 +277,11 @@ def maybe_iterable_to_list(obj: Union[Iterable[T], T]) -> Union[Collection[T], T
"""
if isinstance(obj, abc.Iterable) and not isinstance(obj, abc.Sized):
return list(obj)
# error: Incompatible return value type (got
# "Union[pandas.core.common.<subclass of "Iterable" and "Sized">,
# pandas.core.common.<subclass of "Iterable" and "Sized">1, T]", expected
# "Union[Collection[T], T]") [return-value]
obj = cast(Collection, obj)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

our signature here is not entirely correct. If we pass an Iterable with a __len__ method which is not necessarily a collection, the object is returned unchanged.

I think best to leave signature unchanged for now and could use Protocol in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since i'm lying to the type checker a #type: ignore is probably more preferable to a cast.

return obj


Expand Down
2 changes: 1 addition & 1 deletion pandas/core/computation/expr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -364,7 +364,7 @@ class BaseExprVisitor(ast.NodeVisitor):

unary_ops = _unary_ops_syms
unary_op_nodes = "UAdd", "USub", "Invert", "Not"
unary_op_nodes_map = dict(zip(unary_ops, unary_op_nodes))
unary_op_nodes_map = {k: v for k, v in zip(unary_ops, unary_op_nodes)}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mypy giving

error: Need type annotation for 'unary_op_nodes_map' (hint: "unary_op_nodes_map: Dict[, ] = ...") [var-annotated]

mypy is ok with a dictcomp, or could add Dict[str, str] as variable type annotation instead


rewrite_map = {
ast.Eq: ast.In,
Expand Down
6 changes: 4 additions & 2 deletions pandas/core/indexes/frozen.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,5 +103,7 @@ def __str__(self) -> str:
def __repr__(self) -> str:
return f"{type(self).__name__}({str(self)})"

__setitem__ = __setslice__ = __delitem__ = __delslice__ = _disabled
pop = append = extend = remove = sort = insert = _disabled
__setitem__ = __setslice__ = _disabled # type: ignore[assignment]
__delitem__ = __delslice__ = _disabled # type: ignore[assignment]
pop = append = extend = _disabled # type: ignore[assignment]
remove = sort = insert = _disabled # type: ignore[assignment]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we probably could make FrozenList Generic to allow type annotations such as FrozenList[Label]

This is an exercise for another day.

3 changes: 1 addition & 2 deletions pandas/core/resample.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -966,8 +966,7 @@ def __init__(self, obj, *args, **kwargs):
for attr in self._attributes:
setattr(self, attr, kwargs.get(attr, getattr(parent, attr)))

# error: Too many arguments for "__init__" of "object"
super().__init__(None) # type: ignore[call-arg]
super().__init__(None)
self._groupby = groupby
self._groupby.mutated = True
self._groupby.grouper.mutated = True
Expand Down
2 changes: 1 addition & 1 deletion requirements-dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ flake8<3.8.0
flake8-comprehensions>=3.1.0
flake8-rst>=0.6.0,<=0.7.0
isort>=5.2.1
mypy==0.730
mypy==0.782
pycodestyle
gitpython
gitdb
Expand Down
71 changes: 66 additions & 5 deletions setup.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,10 +127,7 @@ show_error_codes = True
[mypy-pandas.tests.*]
check_untyped_defs=False

[mypy-pandas.conftest]
ignore_errors=True

[mypy-pandas.tests.tools.test_to_datetime]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

errors were fixed with latest typeshed

[mypy-pandas.conftest,pandas.tests.window.conftest]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar error in pandas.tests.window.conftest as in pandas.conftest,pandas, so bundled together.

this is a mypy inference issue we have encountered before

error: List item 0 has incompatible type "ParameterSet"; expected "Sequence[Collection[object]]" [list-item]

ignore_errors=True

[mypy-pandas._testing]
Expand All @@ -139,7 +136,22 @@ check_untyped_defs=False
[mypy-pandas._version]
check_untyped_defs=False

[mypy-pandas.core.arrays.interval]
[mypy-pandas.compat.pickle_compat]
check_untyped_defs=False

[mypy-pandas.core.apply]
check_untyped_defs=False

[mypy-pandas.core.arrays.base]
check_untyped_defs=False

[mypy-pandas.core.arrays.datetimelike]
check_untyped_defs=False

[mypy-pandas.core.arrays.sparse.array]
check_untyped_defs=False

[mypy-pandas.core.arrays.string_]
check_untyped_defs=False

[mypy-pandas.core.base]
Expand All @@ -151,6 +163,9 @@ check_untyped_defs=False
[mypy-pandas.core.computation.expressions]
check_untyped_defs=False

[mypy-pandas.core.computation.ops]
check_untyped_defs=False

[mypy-pandas.core.computation.pytables]
check_untyped_defs=False

Expand All @@ -163,6 +178,9 @@ check_untyped_defs=False
[mypy-pandas.core.generic]
check_untyped_defs=False

[mypy-pandas.core.groupby.base]
check_untyped_defs=False

[mypy-pandas.core.groupby.generic]
check_untyped_defs=False

Expand All @@ -172,15 +190,33 @@ check_untyped_defs=False
[mypy-pandas.core.groupby.ops]
check_untyped_defs=False

[mypy-pandas.core.indexes.base]
check_untyped_defs=False

[mypy-pandas.core.indexes.category]
check_untyped_defs=False

[mypy-pandas.core.indexes.datetimelike]
check_untyped_defs=False

[mypy-pandas.core.indexes.datetimes]
check_untyped_defs=False

[mypy-pandas.core.indexes.extension]
check_untyped_defs=False

[mypy-pandas.core.indexes.interval]
check_untyped_defs=False

[mypy-pandas.core.indexes.multi]
check_untyped_defs=False

[mypy-pandas.core.indexes.period]
check_untyped_defs=False

[mypy-pandas.core.indexes.range]
check_untyped_defs=False

[mypy-pandas.core.internals.blocks]
check_untyped_defs=False

Expand All @@ -190,15 +226,27 @@ check_untyped_defs=False
[mypy-pandas.core.internals.managers]
check_untyped_defs=False

[mypy-pandas.core.internals.ops]
check_untyped_defs=False

[mypy-pandas.core.missing]
check_untyped_defs=False

[mypy-pandas.core.ops.docstrings]
check_untyped_defs=False

[mypy-pandas.core.resample]
check_untyped_defs=False

[mypy-pandas.core.reshape.concat]
check_untyped_defs=False

[mypy-pandas.core.reshape.merge]
check_untyped_defs=False

[mypy-pandas.core.series]
check_untyped_defs=False

[mypy-pandas.core.strings]
check_untyped_defs=False

Expand All @@ -214,6 +262,9 @@ check_untyped_defs=False
[mypy-pandas.io.clipboard]
check_untyped_defs=False

[mypy-pandas.io.common]
check_untyped_defs=False

[mypy-pandas.io.excel._base]
check_untyped_defs=False

Expand All @@ -232,6 +283,9 @@ check_untyped_defs=False
[mypy-pandas.io.formats.css]
check_untyped_defs=False

[mypy-pandas.io.formats.csvs]
check_untyped_defs=False

[mypy-pandas.io.formats.excel]
check_untyped_defs=False

Expand Down Expand Up @@ -270,3 +324,10 @@ check_untyped_defs=False

[mypy-pandas.plotting._matplotlib.misc]
check_untyped_defs=False

[mypy-pandas.plotting._misc]
check_untyped_defs=False

[mypy-pandas.util._decorators]
check_untyped_defs=False