Skip to content

Should tokens passed through macro_rules be able to join? #127123

Open
@petrochenkov

Description

@petrochenkov

Should this example

macro_rules! mac {
    ($a:tt sep $b:tt) => { 1 $a$b 2 } 
}

fn main() {
    mac!(= sep =);
}

emit 1 == 2 and compile successfully?
Or, in other words, should the first emitted = have the Spacing::Joint set, if the output is processed by a proc macro?

We need some well-defined rule for setting spacing for tokens produced from macro variables.

Possible alternatives:

    • Always use Joint.
      • This is very unreasonable.
    • Always use Alone.
      • This is the first reasonable alternative.
    • Inherit from the passed token - that means Alone in the example above because there is a space after the first = in mac!(= sep =).
      • Probably unreasonable, the token sequence in the input have little relation to the output sequence for which we emit the spacing, in the example above it is demonstrated by sep being present in the input but not in the output.
      • Note: the issue is about single passed token trees (or last token trees in sequences), tokens passed in the middle of something like $($tt)* should of course use this option.
    • Inherit from the macro parameter declaration - i.e. $a:tt in the example (or just the $a part of it).
      • This doesn't seem reasonable to me
    • Inherit from the macro parameter use, that means Joint in the example above because there is no space after $a in 1 $a$b 2.
      • This is the second reasonable alternative.

Right now the spacing is emitted inconsistently, sometimes it follows the rule 2., sometimes 3..
Adopting any consistent rule may cause breakage (@nnethercote may remember which exactly) and needs to go through crater.

  • I would personally try the rule 5. first, because it gives macro author freedom to control and change spacing as they want, unlike rule 2. that prevents joining entirely.
  • However we may be forced to adopt rule 2. instead due to backward compatibility issues. It would be less flexible, but macro variables creating a "barrier" for joining is also a reasonable enough model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-macrosArea: All kinds of macros (custom derive, macro_rules!, proc macros, ..)A-proc-macro-back-compatArea: Backwards compatibility hacks for proc macrosC-discussionCategory: Discussion or questions that doesn't represent real issues.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.T-langRelevant to the language team, which will review and decide on the PR/issue.WG-macrosWorking group: Macros

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions