Review-comment directly on erroneous code if possible

[varkor]

At the moment, the bot can sometimes clog up issues with analysis messages if tests repeatedly fail. Though having the comments around after the issues have been fixed is probably still useful (deleting old messages would work, but I can imagine some people not liking treating the comment thread so mutably), what would be really nice is if the bot could make review comments, instead of top-level comments, on the lines of code that contained issues. This way, when the lines are changed, the bot's comments will automatically be hidden by GitHub's "outdated comment" interface.

[TimNN]

I would love to do this, however I don't think it is feasible to automatically identify the lines that cause problems.

This means I cannot use the GitHub API to post review comments. Commit comments also don't help, since, while they show up in the PR discussion, they stay there, even if new commits are pushed.

[TimNN]

I tried some experiments with review comments. Sadly, review comments are not collapsed when a review is approved / dismissed.

[varkor]

@TimNN: in some cases, where there are explicit Rust error messages (for example in this comment), a regex for --> [dir]/[filename].rs:[line]:[column] would be a good heuristic? This would obviously only be available for some errors (explicit Rust errors, plus tidy messages, for instance), but it would be useful where possible.

[pietroalbini]

FYI, GitHub introduced collapsable comments right now (lol).

[TimNN]

I contacted GitHub support about an API for the hiding comments feature. They don't have an API at this time, however they have registered our interest / use case.

[varkor]

Maybe this: https://blog.github.com/2018-05-07-introducing-checks-api/ could provide a better interface now?

[pietroalbini]

We can't really use that without a lot of changes, it requires to use a GitHub app.