Macro expansion: Provide information about lexical environment

[AriMKatz]

https://github.com/jolin-io/WhereTraits.jl cites the following limitation with attendant explanation:

Top Level Only
Currently only top-level functions are supported, as the syntax stores and needs information about previous function definitions, which it stores globally. If macros would get informed about whether they are defined within another function, WhereTraits could also support innerfunctions.

I know flisp is involved with macro expansion. Is the above limitation something that can be addressed with this package?

[c42f]

The docs say

If macros would get informed about whether they are defined within another function

The difficulty I see with this is that macros are expanded prior to the builtin code lowering passes. But knowing "whether they are defined within another function" is, by definition part of the closure detection and lowering pass.

So requiring knowledge of this from within a macro seems close to a circular dependency.

---

It might be valid to intersperse user-defined macro expansion with other code desugaring steps in a hierarchical way. Particularly, this seems valid if we assume macros are pure functions, and only intersperse macro expansion with other strictly hierarchical code desugaring steps.

In that case I think it would be consistent to provide information from the lexical environment to macros. For example, which top level function they belong to. In fact we already do this by providing the __module__ parameter to macro expansion.

However, it's relatively easy to arrange for the __module__ to exist prior to macro expansion. Making sure a function exists in eval()'d form is more problematic — it more comprehensively breaks the separation between lowering (a symbolic transformation) and evaluation (with all the side effects that eval has on the Julia runtime).

A saner alternative could be to provide only symbolic information about lexical environment to macro expansion. So for example, macros could potentially know which name of top level function they are being expanded within, but not have access to the eval'd function object itself.

IIUC, Jeff previously (a couple of years ago in a discussion) expressed some dissatisfaction with how macro expansion redoes some work which is later done in lowering. So there may be appetite to revisit how some of this works.

---

As a summary, I think

• This is an interesting feature request which probably could be implemented
• It requires a fairly major change to internals of how lowering and macro expansion interacts with lowering
• However, such an internal change might be needed anyway to fix some other related implementation issues and should be non-breaking