`Literal`s overlap when defaults given

[llchan]

It appears that Literals with default values lead to overlapping signatures. In the following example, the defaults are necessary since this argument follows another defaulted argument.

• Are you reporting a bug, or opening a feature request? bug
• Please insert below the code you are checking with mypy.

  @overload
  def foo(x: int = 0, y: Literal[False] = False) -> int: ...
  @overload
  def foo(x: int = 0, y: Literal[True] = True) -> str: ...
  @overload
  def foo(x: int = 0, y: bool = False) -> Union[int, str]: ...

• What is the actual behavior/output?

  Overloaded function signatures 1 and 2 overlap with incompatible return types
  

• What is the behavior/output you expect?
  No error
• What are the versions of mypy and Python you are using?
  mypy 0.680+dev.4e0a1583aeb00b248e187054980771f1897a1d31 (current master)
  CPython 3.7.1

[gvanrossum]

But mypy is right. When you call foo() it won’t know whether the return is int or str.

[llchan]

In the foo() case, I agree it could be either return type because there is no literal False nor literal True so it would match signature 3. However, as soon as there's a literal it should be unambiguous, I think. Am I'm overlooking something?

[gvanrossum]

The point is that without arguments the defaults take over.

[llchan]

I guess maybe let me rephrase the question: how do I specify a literal keyword argument in the middle of other keyword arguments? It seems that we may need a sentinel value so we can signal to the overload to only match that signature if the keyword is explicitly present as a literal.

@overload
def foo(x: int = 0, y: Literal[False] = REQUIRED) -> int: ...
@overload
def foo(x: int = 0, y: Literal[True] = REQUIRED) -> str: ...
@overload
def foo(x: int = 0, y: bool = False) -> Union[int, str]: ...

The desired behavior:

reveal_type(foo())          # Union[int, str]
reveal_type(foo(0, False))  # int
reveal_type(foo(0, True))   # str
reveal_type(foo(y=False))   # int
reveal_type(foo(y=True))    # str
reveal_type(foo(y=z))       # Union[int, str]

[llchan]

I should mention that I'm adding types to code that must continue to support PY2 for now (I know, I know, but migrating takes time, and there are many downstream users), so making these keyword-only with the star is not an option.

[gvanrossum]

You don't have to provide a default to make something a keyword argument.

def foo(x: int, y: Literal[True]) -> ...

can still be called as

foo(x=42, y=True)

With overloading I would recommend overloading all variants with mandatory arguments, e.g.

@overload
def foo(x: int, y: Literal[True]) -> str: ...
@overload
def foo(x: int, y: Literal[False]) -> int: ...
@overload
def foo(x: int) -> int: ...

[llchan]

My bad, my terminology was a bit imprecise. I meant a literal after one or more defaulted arguments. The x argument has a default that forces us to stick a default on the y argument. I'd like for us to be able to have a way to say "match this only when y has a literal binding", but we need some placeholder to put as the default.

[gvanrossum]

So make two sets of overloads -- one set with x: int and one set without it. Then in each overload x is mandatory if it is present at all, and you don't need a default for y.

[llchan]

This toy example only has one defaulted argument before the literal, but the actual function has many other defaulted arguments before, which would require 2N 2^(N +1) overloads manually spelled out. I'd argue that the sentinel value solves this more elegantly and conveys the intent in a more readable manner.

If this is a wont-fix due to limited dev resources, that's understandable, I just want to make sure I'm getting the proposal across.

EDIT: was only thinking positional calls. it would need at least 2^(N+1) overloads, and also this strategy doesnt work if any other defaulted arguments have a matching type (e.g. if x were a bool in the example above).

[JukkaL]

It looks like this is a real problem, and mypy could probably support this use case without requiring a huge number of overloads. I suspect that the same issue will at least occasionally come up when using literal types in library stubs. Reopening the issue, though I can't estimate when the mypy team might have resources to fix this.