Feature request: preserve function parameter names for improved compatibility with Python libraries

[ikappaki]

(See https://clojurians.slack.com/archives/C071RFV2Z1D/p1734990790884169 for context)

Hi,

There appear to exist Python libraries that provide function decorators which expect the function they decorate to have specific parameter names.

One such library is FastAPI. Below is an example taken from the FastAPI documentation:

It creates two endpoints:

• The / endpoint will return the dictionary {"message": "World"} dict (.e.g. at http://localhost/).
• The /items/{itemd_id} endpoint will return a dictionary with the item_id, e.g. at http://localhost/items/abcd it will return {"item_id": "abcd"}

@app.get("/")
def read_root():
    return {"message": "World"}


@app.get("/items/{item_id}")
def read_item(item_id: int, q: Union[str, None] = None):
    return {"item_id": item_id, "q": q}

In this example, the {item_id} in the decorator corresponds to the item_id parameter in the function. This allows FastAPI to bind the path parameter to the function parameter automatically.

However, Basilisp appears to modify function parameter names by appending a suffix through the basilisp.lang.util.genname function. As a result, a hypothetical parameter like abc would become abc_nnn when compiled down to Python. This makes it impossible to align the parameter names with those expected by decorators from libraries like FastAPI.

For instance, if you try to use the following namespace with a FastAPI decorator that expects a name parameter:

(ns basilex-fastapi.ex
  (:import [fastapi :as f]
           [uvicorn :as uv]))

(defonce app (f/FastAPI))

(defn root  {:async true
             :decorators [(.get app "/")]}
  []
  {"message" "Hi there"})

(defn hello  {:async true
              :decorators [(.get app "/hello/{name}")]}
  [name]
  {"name" name})

(comment
  (def server (future (uv/run app ** :host "127.0.0.1" :port 8000))))

The resulting Python code will fail to bind the path parameter name to the function parameter because Basilisp renames the name parameter to name_nnn when creating the function. Therefore, FastAPI will not find a name parameter to bind the value passed in the URL (e.g., http://localhost:8000/hello/xyz).

This issue arises due to Basilisp’s use of genname, which changes the parameter names. The

basilisp/src/basilisp/lang/compiler/generator.py

Line 1638 in 5525a69

arg_name = genname(munge(binding.name))

snippet shows how genname is used to alter parameter names:

def __fn_args_to_py_ast(
    ctx: GeneratorContext, params: Iterable[Binding], body: Do
) -> tuple[list[ast.arg], Optional[ast.arg], list[ast.stmt], Iterable[PyASTNode]]:
    """Generate a list of Python AST nodes from function method parameters."""
    fn_args, varg = [], None
    fn_body_ast: list[ast.stmt] = []
    fn_def_deps: list[PyASTNode] = []
    for binding in params:
        assert binding.init is None, ":fn nodes cannot have binding :inits"
        assert varg is None, "Must have at most one variadic arg"
        arg_name = genname(munge(binding.name))
#...

It's unclear why genname is essential in this context. If I remove genname (but keep the mungeing), the isolated example works as expected. However, removing genname causes Basilisp to fail to bootstrap when recompiling the codebase from scratch.

Is there a way to bypass the function parameter name uniquefication process in some cases, particularly when it doesn't appear to be required?

One possible approach would be to introduce metadata to indicate that a parameter name should be preserved. For example, we could use a metadata marker like :preserve-parameter-name to advise the Basilisp generator not to modify the parameter name during compilation.

(defn hello  {:async true
                    :decorators [(.get app "/hello/{name}")]}
  [^:preserve-parameter-name name]
  {"name" name})

Thanks

[qubit55]

I'm hitting the exact same issue now with your dynamically updating routes function.

[ikappaki]

HI @qubit55,

I had a quick look at how to circumvent named parameter and here is a hack that uses a single parameter, :taged as fastapi/Request, to extract info about the request parameters:

(defn hello  {:async true
              :decorators [(.get app "/hello/{name}")]}
  [^f/Request r]
  (let [nm (aget (.-path-params r) "name")]
    {"hello" nm}))

As discussed in the update-route! hack, it might be a good idea to create a Basilisp library with lightweight macros to address the current, hopefully temporary, limitations with parameter name mungling in Basilisp.

I hope this helps.

[qubit55]

Hi @ikappaki,

Ah, right, we can work directly with the request object, nice fix!

Yeah, I agree, there should definitely be a macro to automate all that boilerplate.

I wonder how things are with other Python frameworks, they all seem to handle routing via decorators?

Here's a minimal working example of everything put together, including the modification for the interactive workflow (redefining the routes on the fly), for reference:

(import [fastapi :as f]
        [uvicorn :as uv])

(defonce app (f/FastAPI))

(defn update-route!
  [app path]
  (fn [func]
    (let [routes (seq (.. app -router -routes))]
      (doseq [route routes]
        (when (and (= path (.-path route)) (not= (.-endpoint route) func))
          (.remove (.. app -router -routes) route))))
    func))

(defn root {:async true
            :decorators [(update-route! app "/")
                         (.get app "/")]}
  []
  {"message" "Hi there"})

(defn item {:async true
            :decorators [(update-route! app "/item/{id}")
                         (.get app "/item/{id}")]}
  [^f/Request r]
  (let [id (aget (.-path-params r) "id")]
    {"Item's id " id}))

(comment
  (def server (future (uv/run app ** :host "127.0.0.1" :port 8000)))
)

[amano-kenji]

I just looked into clojure's defn. clojure defn doesn't seem to change argument names.

[amano-kenji]

According to 9.2. Python Scopes and Namespaces, there are class local scope, function local scope, and global scope consisting of module names and built-in names. Functions defined in functions have their own local scope and have access to outer scopes including enclosing functions, enclosing classes, and enclosing modules.

So, from my vantage point, I don't see a reason to change argument names in defn because python functions have their own local scope. Hy language translates let into a python function because python doesn't have a block scope that corresponds to let.

[chrisrink10]

Fixed in #1222

[ikappaki]

According to 9.2. Python Scopes and Namespaces, there are class local scope, function local scope, and global scope consisting of module names and built-in names. Functions defined in functions have their own local scope and have access to outer scopes including enclosing functions, enclosing classes, and enclosing modules.

So, from my vantage point, I don't see a reason to change argument names in defn because python functions have their own local scope. Hy language translates let into a python function because python doesn't have a block scope that corresponds to let.

Hi @amano-kenji, as noted in the #1222 comment, function param names are made globally unique to support inlining. Ideally, this wouldn’t be necessary, but there are technical challenges to consider if it has to be changed.

[chrisrink10]

Hi @amano-kenji, as noted in the #1222 comment, function param names are made globally unique to support inlining. Ideally, this wouldn’t be necessary, but there are technical challenges to consider if it has to be changed.

Basilisp has generated unique parameter names almost since the beginning, not because of inlining specifically. Many parts of designing a compiler are made much easier when you don't have to worry about name clashes.