Description
Bug report
Several repositories (like Prefect) make use of deferred execution of functions. They utilise inspect.Signature to create a bound method, and turn a list of parameters into args and kwargs to be passed in *args, **kwargs.
When it works fine, it looks like this:
import inspect
def add(a, b, c=100):
return a + b + c
bound = inspect.signature(add).bind(1, 2, c=100)
print(f"args: {bound.args}, kwargs: {bound.kwargs}, result: {add(*bound.args, **bound.kwargs)}")And this prints out:
args: (1, 2, 100), kwargs: {}, result: 103
Notice that the 100 has moved from a kwarg into an arg, but that's fine, it will still run.
Things get complicated when decorators are introduced, and it seems base python has no method of getting args and kwargs that will work with both the wrapped and unwrapped signature.
The below code has two methods, each decorated, and both of them fail the bound execution.
from functools import wraps
import inspect
def decorator(fn):
@wraps(fn)
def wrapper(a, b, **kwargs):
print(f"kwargs are {kwargs}")
return fn(a, b, **kwargs)
return wrapper
@decorator
def add(a, b, c=100):
return a + b + c
def decorator2(fn):
@wraps(fn)
def wrapper(a, b, c=100):
return fn(a, b, c=c)
return wrapper
@decorator2
def add2(a, b, **kwargs):
return a + b + kwargs.get("c", 100)
try:
bound_signature = inspect.signature(add).bind(1, 2, c=100)
print(bound_signature.args, bound_signature.kwargs)
add(*bound_signature.args, **bound_signature.kwargs)
except Exception as e:
print("Failure in first method: ", e)
try:
bound_signature = inspect.signature(add2, follow_wrapped=False).bind(1, 2, c=100)
print(bound_signature.args, bound_signature.kwargs)
add(*bound_signature.args, **bound_signature.kwargs)
except Exception as e:
print("Failure in second method: ", e)And fails with:
(1, 2, 100) {}
Failure in first method: wrapper() takes 2 positional arguments but 3 were given
(1, 2, 100) {}
Failure in second method: wrapper() takes 2 positional arguments but 3 were given
They fail because of that default behaviour of treating kwargs as args.
In terms of expected behaviour, I would have assumed that a signature on a wrapped method, when resolving to args and kwargs, would be able to assign parameters between args and kwargs reliably.
Potential Solution
I notice that in inspect.py -> BoundArguments -> args/kwargs properties that some logic might be modified here.
Instead of making all _POSITIONAL_OR_KEYWORD parameters args, I feel the language would be more correct to assign them based upon whether or not the parameter.default is Parameter.empty. This may fix the issue and stop greedy conversion to arguments.
Your environment
- CPython versions tested on: 3.9.13
- Operating system and architecture: MacOS