Create a tool to generate semantically correct tags files

[chris-morgan]

Currently we have the TAGS.vi and TAGS.emacs rules which use Exuberant Ctags and regular expressions to produce a very useful but not particularly brilliant result. Now that we have librustdoc, we can do much better, rather quickly! I want a tool (quite probably built into rustdoc, actually) which can generate semantically correct and optimally useful tags files for the Vim and Emacs standards. Doing this absolutely correctly will be a great help to developers, who will already benefit if they go to the trouble of ensuring they have the tags files loaded in their editor.

This will allow things like adding enum variants to the tags file which probably can't be done correctly at present (and no attempt is made) as I think you could have further nested parentheses inside the variant, e.g. I think Foo((), (bar, ())) would be valid, but it'd make a mess of a regex-based rule unless recursion could actually be managed in a nasty hacky way.

[dcrewi]

I'm working on this. As someone who doesn't use vim for code editing, would the extra information in the extended format actually be used intelligently for rust programs? I'm talking about marking what kind of tag it is and marking what class a method belongs to and such.

[chris-morgan]

@dcrewi Yes, Vim can make use of extended information; for example, I tried modifying my TAGS.vi to add trait and type information (the contents after the word f on the first two lines is what I added):

push_str    /home/chris/vc/rust/src/libstd/str.rs   /^    fn push_str(&mut self, rhs: &str) {$/;"   f   trait:OwnedStr  type:~str
push_str    /home/chris/vc/rust/src/libstd/str.rs   /^    fn push_str(&mut self, rhs: &str);$/;"    f   trait:OwnedStr
push_str    /home/chris/vc/rust/src/libstd/str.rs   /^pub fn push_str(lhs: &mut ~str, rhs: &str) {$/;"  f

:ts push_str then shows "trait:OwnedStr type:~str" as an extra line for the first and "trait:OwnedStr" for the second, in the right place.

(Of course, we are here speaking of tags which are inherently fairly dumb; intelligent completion is a completely distinct project of significantly greater difficulty which is covered elsewhere.)

The ability to specify whether private things should have tags made for them or not would be good, too. (If I'm working on a crate, I want all the tags, but if it's just a dependency of mine I probably only want public tags.)

[nerdrew]

Is there any movement on this? Is there any progress that can be continued?

[dcrewi]

There was some progress, but it has bitrotted. The general idea is to walk a filesystem searching for "lib.rs" or "main.rs" (though now that cargo exists, it should probably look for cargo files as well), use libsyntax to parse crates into AST, and collect tags by walking the AST using a syntax::visit::Visitor instance.

[nikomatsakis]

Some thoughts:

There is no reason to do this as part of rustdoc that I can see, it seems independent. I'd probably like it to be part of rustc. I imagine it working roughly like this: you write --etags=path-to-TAGS-file or --ctags=path-to-tags-file (or something), and it will start to generate a TAGS file alongside the binary. If there is already an existing file, it is loaded and parsed: all existing entries that correspond to files that are within the current source directory (the subdirectory containing lib.rs or whatever) are removed. New entries are added for the new source files. This way you can concatenate multiple crates together into one file. Bonus points for integrating this into cargo.

Coupling it to rustc seems a bit strange but also damn convenient, because you basically want these to always get regenerated whenever you build. Also there is no stable interface to rustc so it seems ok. I guess I'd say that long term it should live wherever the dxr source lives, which is currently rustc.

The TAGS file should be generated after parsing but before type check, so that you get TAGS files even if compilation errors occur.

[nikomatsakis]

(Probably we should make the command line interface something explicitly unstable.)

[nikomatsakis]

Anyway I really want this, wherever it lives ;)

[steveklabnik]

I'm pulling a massive triage effort to get us ready for 1.0. As part of this, I'm moving stuff that's wishlist-like to the RFCs repo, as that's where major new things should get discussed/prioritized.

This issue has been moved to the RFCs repo: rust-lang/rfcs#656