Tracking Issue for autodiff

[traviscross]

This is a tracking issue for the automatic differentiation experiment ("autodiff").

The feature gate for the issue will be #![feature(autodiff)].

About tracking issues

Tracking issues are used to record the overall progress of implementation. They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions. A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature. Instead, open a dedicated issue for the specific matter and add the relevant feature gate label. Discussion comments will get marked as off-topic or deleted. Repeated discussions on the tracking issue may lead to the tracking issue getting locked.

Steps

• Get compiler MCP approved.
  • Integrate Enzyme into nightly rustc compiler-team#611
• Get lang experiment approved.
  • We approved this in the lang triage meeting on 2024-05-01.
• Land the experimental implementation in nightly.
  • Combined change for reference: Autodiff Upstreaming - single commit #129175
  • Autodiff Upstreaming - enzyme backend #129176
  • Autodiff Upstreaming - enzyme frontend #129458
  • Autodiff Upstreaming - rustc_codegen_llvm changes #130060
  • add has_enzyme/needs-enzyme to the test infra #131044
  • add test infra to explicitely test rustc with autodiff/enzyme disabled #131470
  • Autodiff Upstreaming - rustc_codegen_ssa, rustc_middle #133429
  • Separate Builder methods from tcx #135581
  • fix autodiff compile time regression #136413
  • adding autodiff tests #136419
  • improvements towards building enzyme in CI #136428
  • Autodiff batching #137880
• Accept an RFC.
• Add documentation to the dev guide.
  • See the instructions.
• Add documentation to the reference.
  • See the instructions.
• Add formatting for new syntax to the style guide.
  • See the nightly style procedure.

Related

• Integrate Enzyme into nightly rustc compiler-team#611
• tool builds with Enzyme/autodiff fails #130637
• move enzyme flags from general cargo to rustc-specific cargo #130648

Unresolved Questions

TODO.

cc @ZuseZ4 @oli-obk

[traviscross]

@rustbot labels -I-lang-nominated

We discussed this on the lang triage call today and were happy to see this move forward as an experiment. I'll be continuing as the liaison for this, so please let me know whenever anything is needed to keep this moving along.