Check that literal strings/int/float belong to /is excluded from a set/range of values

[sametmax]

Opened in python/mypy#4040, but moved here after @JukkaL 's advice.

Some debate took place in there, but I'll copy the original post here for context:

---

It's a common practice to pass literal strings as arguments. In Python, it's even more important, as strings are often used instead of byte flags, constants or enums.

You often end up checking if those literals are passed correctly so you can give some debug information:

• sorry, the parameter mode except a string among "started, paused, cancelled";

• you can only used an integer between 0 and 16.

etc.

The problem with that is it's done at runtime despite the fact those argument are completely static ones. So the IDE can't use this information to help you write code. Typically, I would have code completion and linting helping me with "started/pause/cancelled" if it was an enum, but not if it's a string.

With this concept I could do:

MODES = ('started', 'paused', 'cancelled')
LEVELS = (
    range(0, 5),
    range(40, float('+inf')), 
)

def foo(mode: typing.Among(MODES), levels: typing.Among(LEVELS)):
    pass

So that mypy could alert my users if they do:

def foo("Start", 6):

Of course, it's only for strings/ints/floats literals, maybe bytes and tuple, as we need the information to be immutable and simple to be usable.

[ilevkivskyi]

a lot of people don't use enum. It's very overkill when you start a project, especially with Python and its simple syntax

I don't see how it is a "very overkill" to write:

from your_lib import foo, mode

foo(mode.started)

instead of

from your_lib import foo

foo('started')

Anyway, if we are going to consider something like this, I would propose an already discussed idea: literal types, so that the first example will be

from typing import Union, Literal

Mode = Union[Literal['strated'], Literal['paused'], Literal['cancelled']]

def foo(mode: Mode): ...

foo('started') # OK
foo('wrong') # Error!

x = 'started'
foo(x) # Also an error, we can't track a 'str' variable

The same would work e.g. to describe open (currently special-cased in mypy) etc.:

@overload
def open(name: str, mode: Literal['r']) -> IO[str]: ...
@overload
def open(name: str, mode: Literal['rb']) -> IO[bytes]: ...
@overload
def open(name: str, mode: str) -> IO[Any]: ...

This will not cover the integer ranges, but I think it is too advanced for a typechecker (I think we could support just simple/common dependent types).

[shoyer]

Another use-case is booleans, e.g., the return value from pandas.to_datetime() is either a pandas.DatetimeIndex or numpy.ndarray, based upon whether it is called with the arguments box=True or box=False.

Both boolean flags and strings as enums are endemic within Python's numerical computing ecosystem.

[JukkaL]

@shoyer It would help us if you can give additional concrete examples of cases where string or boolean values affect types (in numerical libraries or elsewhere).

[shoyer]

Let me give examples culled from the API docs for pandas, which I think reflect common usage in numerical libraries:

Format: argument_name=literal_value -> return_type

pandas.read_csv
squeeze=False -> DataFrame
squeeze=True -> Union[DataFrame, Series]

pandas.cut
retbins=False -> Series
retbins=True -> Tuple[Series, ndarray]

pandas.concat
axis=0 -> Series
axis=1 -> DataFrame

pandas.get_dummies
sparse=False -> DataFrame
sparse=True -> SparseDataFrame

pandas.to_datetime
box=True -> Series
box=False -> ndarray

pandas.Series.drop
inplace=False -> Series
inplace=True -> None

pandas.Series.reset_index
drop=True -> Series
drop=False -> DataFrame

pandas.DataFrame.to_dict
orient='dict' -> dict like {column -> {index -> value}}
orient='list' -> dict like {column -> [values]}
orient='series' -> dict like {column -> Series(values)}
orient='split' -> dict like {index -> [index], columns -> [columns], data -> [values]}
orient='records' -> list like [{column -> value}, ... , {column -> value}]
orient='index' -> dict like {index -> {column -> value}}

These are not cherry-picked examples -- these are common functions/methods used in a large fraction of code using pandas. So in practice, this will be a very real obstacle when/if we try to add type annotations to pandas (pandas-dev/pandas#14468). Certainly pandas does not exhibit best practices here, but nonetheless it's a very popular library.

This is intentionally only including cases where type signature itself varies gives based on the literal value, which precludes usefully typing a function almost at all unless we have a way of recognizing literal notes. There are many more examples, including in libraries like numpy or scipy, where a limited set of strings describes all valid values. From a principled perspective, most of these should probably be using enums instead of strings or booleans, but the value dependent semantics remains.

[JukkaL]

@shoyer Thanks for the additional examples! They are very helpful. The current type system clearly seems to work poorly for numerical Python libraries.

Here are a few additional things that would be useful for moving this forward:

• Do a similar analysis as above for numpy and scipy (and perhaps other popular numerical libraries).
• Collect a corpus of open-source numerical Python code (from GitHub, perhaps) that we can use to look for additional real-world use cases.

[TeamSpen210]

Another example is tkinter, although that's more of a wrapper around another language and isn't exactly Pythonic in a lot of cases. There most function arguments are either numbers, or string enums. It would be desirable to ensure they match the method that's called. It doesn't have many functions where the return type depends on a literal though, although many do change between a return value or None depending on the number of arguments (if they match the default).

[shoyer]

Looking through NumPy, I notice some themes. There are a few common arguments that only have particular valid values (enum equivalents):

• order : {‘K’, ‘A’, ‘C’, ‘F’}
• casting : {‘no’, ‘equiv’, ‘safe’, ‘same_kind’, ‘unsafe’}
• subok : bool: if True, result can be an ndarray subclass, otherwise only a baseclass ndarray

These are found on a very large portion of NumPy's API (e.g., most array creation routines, shape changing routines, copying routines and all ufuncs).

NumPy doesn't use enums, so the "strings as enums" pattern is endemic throughout the library:
numpy.seterr and numpy.errstate: arguments can be of {‘ignore’, ‘warn’, ‘raise’, ‘call’, ‘print’, ‘log’}
numpy.fft routines: norm can be either None or 'ortho'
numpy.take, numpy.choose and numpy.put: mode can be any of {‘raise’, ‘wrap’, ‘clip’}
numpy.nditer: quite a few arguments, all of which can only be particular strings
numpy.linalg.norm: {non-zero int, inf, -inf, ‘fro’, ‘nuc’}
numpy.pad: mode can take on any of 10 string values
numpy.sort and related routines: kind can be any of {‘quicksort’, ‘mergesort’, ‘heapsort’}
numpy.correlate and numpy.convolve: mode can be any of {‘valid’, ‘same’, ‘full’},
numpy.histogram: bins can be an integer, sequence or any of 7 specific string values

There are also a few functions whose return value depends on boolean flags:
numpy.linalg.svd: if compute_uv=True, returns a tuple of three arrays, otherwise just one
numpy.unique: returns either a single array, or a tuple of 2-4 arrays, depending on three boolean flag values

I have not looked into the more esoteric corners of the library (e.g., masked arrays, numpy.matrix, polynomials or financial functions).

So on the whole, NumPy is certainly in a much better position than pandas: there are only a handful of functions where the return type depends on a literal values (although they are widely used).

I'm not going to bother going through SciPy as the API is larger and more varied, and I'm less familiar with it, but I hope you'll trust me that its situation is very similar to that for NumPy. The is large overlap in the community maintaining both libraries, and for a short while they were even integrated in a single project. Certainly strings are used as enums throughout. I don't know if there are any commonly used functions that can return multiple types.

One last thing I'll note is that there are quite a few further examples of type unstable behavior (even in NumPy) if we ever tried to make array shapes part of the type system.

[rowillia]

@JukkaL I recall talking to @ambv about this at PyCon as well in the context of something simple like correctly typing open, which has the same problem (e.g. open(filename, 'w') returns IO[str] whereas open(filename, 'wb') returns IO[bytes]).

[gvanrossum]

Note that we solved the specific problem with open() using a plugin. That is, if the argument is a literal string, we parse it and infer the appropriate type. If the argument is a correctly-typed expression that's not a literal, the old behavior still applies.

In theory you can now write numpy-specific plugins that do the same thing for the numpy APIs you list above. As long as the call sites are typically passing literals.

Adding a new mechanism to the type system that would let you define a subtype of str that is restricted to a given set of values would be more challenging (though the plugin mechanism might be up to this as well -- I'm not sure, but in principle it has arbitrary powers, which include shooting yourself in the foot :-).

[emmatyping]

While plugins work, they are less than ideal. I think it would be much better to have literal types in the type system. It is a not too uncommon pattern to have literal keyword arguments cause different behavior. Additionally, it is much more maintainable in my mind to write the types via an overload as compared to a plugin (less indirection, less magic, etc). That being said, for smaller cases, plugins are an acceptable stop-gap measure.

[gvanrossum]

No argument there! We've just been worried about the cost of adding literal types to the type system vs. the cost of developing a plugin system. (Also, the syntax for the mode argument to open() is a little too complicated, but this is an exception to the exceptions. :-)

[ilevkivskyi]

Another example of use in numpy is dtype='int8' and similar. I was thinking about this for some time, and scrolling through my old numeric code. It looks we will need three ingredients:
1). Literal types, I would propose a simple syntax:

var: Literal['value']

2). Constant qualifier, it is needed for situations like this:

@overload
def func(mode: Literal['b']) -> IO[bytes]: ...
@overload
def func(mode: Literal['s']) -> IO[str]: ...

MODE: Const[str] = 'b'

func(MODE) # this should be IO[bytes]

This example is oversimplified, but I have seen such patterns for data types, array/matrix dimensions/sizes etc. (see the next point).
3). Integer generic or shape type, this is needed for fixed size arrays/matrices. This one is hardest and will definitely require a plugin.

I think that we could move step by step here. The first two are actually not so hard to implement, will already cover large amount of numeric code, and could be also useful for other (non-numeric) code.
I would propose to start from these two (we can put them in typing_extensions first).

The idea is that Literal will be mostly used in library code, wile Const will be mostly used in user code. I am not sure how much of static constant "calculations" we need to support, it seems to me that basic constant propagation (like in the example above) will be sufficient for vast majority of code.

[shoyer]

Another example of use in numpy is dtype='int8' and similar. I was thinking about this for some time, and scrolling through my old numeric code.

This is true, but strings like 'int8' have always been a short-cut for np.int8. I would be OK requiring user code to types something like dtype=np.int8 if they want an array to type-check as having int8 dtype.

@gvanrossum @ilevkivskyi I assume by "plugin" you are referring specifically to mypy?

[emmatyping]

@ilevkivskyi I'm not sure I understand why Literal types are needed, especially if there is a Const type. I understand they are used in different ways, but isn't a Literal an in-place Const?