[clang] Introduce diagnostics suppression mappings (#112517)

This implements

https://discourse.llvm.org/t/rfc-add-support-for-controlling-diagnostics-severities-at-file-level-granularity-through-command-line/81292.

Users now can suppress warnings for certain headers by providing a
mapping with globs, a sample file looks like:
```
[unused]
src:*
src:*clang/*=emit
```

This will suppress warnings from `-Wunused` group in all files that
aren't under `clang/` directory. This mapping file can be passed to
clang via `--warning-suppression-mappings=foo.txt`.

At a high level, mapping file is stored in DiagnosticOptions and then
processed with rest of the warning flags when creating a
DiagnosticsEngine. This is a functor that uses SpecialCaseLists
underneath to match against globs coming from the mappings file.

This implies processing warning options now performs IO, relevant
interfaces are updated to take in a VFS, falling back to RealFileSystem
when one is not available.

Author: kadircet

clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp

@@ -81,7 +81,8 @@ ExpandModularHeadersPPCallbacks::ExpandModularHeadersPPCallbacks(
   Diags.setSourceManager(&Sources);
   // FIXME: Investigate whatever is there better way to initialize DiagEngine
   // or whatever DiagEngine can be shared by multiple preprocessors
-  ProcessWarningOptions(Diags, Compiler.getDiagnosticOpts());
+  ProcessWarningOptions(Diags, Compiler.getDiagnosticOpts(),
+                        Compiler.getVirtualFileSystem());
 
   LangOpts.Modules = false;
 

clang/docs/ReleaseNotes.rst

@@ -869,6 +869,9 @@ New features
   attribute, the compiler can generate warnings about the use of any language features, or calls to
   other functions, which may block.
 
+- Introduced ``-warning-suppression-mappings`` flag to control diagnostic
+  suppressions per file. See `documentation <https://clang.llvm.org/docs/WarningSuppressionMappings.html>_` for details.
+
 Crash and bug fixes
 ^^^^^^^^^^^^^^^^^^^
 

clang/docs/UsersManual.rst

@@ -151,6 +151,10 @@ Options to Control Error and Warning Messages
   instantiation backtrace for a single warning or error. The default is 10, and
   the limit can be disabled with `-ftemplate-backtrace-limit=0`.
 
+.. option:: --warning-suppression-mappings=foo.txt
+
+   :ref:`Suppress certain diagnostics for certain files. <warning_suppression_mappings>`
+
 .. _cl_diag_formatting:
 
 Formatting of Diagnostics
@@ -1315,6 +1319,34 @@ with its corresponding `Wno-` option.
 Note that when combined with :option:`-w` (which disables all warnings),
 disabling all warnings wins.
 
+.. _warning_suppression_mappings:
+
+Controlling Diagnostics via Suppression Mappings
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Warning suppression mappings enable users to suppress Clang's diagnostics in a
+per-file granular manner. Enabling enforcement of diagnostics in specific parts
+of the project, even if there are violations in some headers.
+
+.. code-block:: console
+
+  $ cat mappings.txt
+  [unused]
+  src:foo/*
+
+  $ clang --warning-suppression-mappings=mapping.txt -Wunused foo/bar.cc
+  # This compilation won't emit any unused findings for sources under foo/
+  # directory. But it'll still complain for all the other sources, e.g:
+  $ cat foo/bar.cc
+  #include "dir/include.h" // Clang flags unused declarations here.
+  #include "foo/include.h" // but unused warnings under this source is omitted.
+  #include "next_to_bar_cc.h" // as are unused warnings from this header file.
+  // Further, unused warnings in the remainder of bar.cc are also omitted.
+
+
+See :doc:`WarningSuppressionMappings` for details about the file format and
+functionality.
+
 Controlling Static Analyzer Diagnostics
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 

clang/docs/WarningSuppressionMappings.rst

@@ -0,0 +1,97 @@
+============================
+Warning suppression mappings
+============================
+
+.. contents::
+   :local:
+
+Introduction
+============
+
+Warning suppression mappings enable users to suppress Clang's diagnostics in a
+per-file granular manner. Enabling enforcement of diagnostics in specific parts
+of the project, even if there are violations in some headers.
+
+Goal and usage
+==============
+
+Clang allows diagnostics to be configured at a translation-unit granularity.
+If a ``foo.cpp`` is compiled with ``-Wfoo``, all transitively included headers
+also need to be clean. Hence turning on new warnings in large codebases requires
+cleaning up all the existing warnings. This might not be possible when some
+dependencies aren't in the project owner's control or because new violations are
+creeping up quicker than the clean up.
+
+Warning suppression mappings aim to alleviate some of these concerns by making
+diagnostic configuration granularity finer, at a source file level.
+
+To achieve this, user can create a file that lists which :doc:`diagnostic
+groups <DiagnosticsReference>` to suppress in which files or paths, and pass it
+as a command line argument to Clang with the ``--warning-suppression-mappings``
+flag.
+
+Note that this mechanism won't enable any diagnostics on its own. Users should
+still turn on warnings in their compilations with explicit ``-Wfoo`` flags.
+`Controlling diagnostics pragmas
+<https://clang.llvm.org/docs/UsersManual.html#controlling-diagnostics-via-pragmas>`_
+take precedence over suppression mappings. Ensuring code author's explicit
+intent is always preserved.
+
+Example
+=======
+
+.. code-block:: bash
+
+  $ cat my/user/code.cpp
+  #include <foo/bar.h>
+  namespace { void unused_func1(); }
+
+  $ cat foo/bar.h
+  namespace { void unused_func2(); }
+
+  $ cat suppression_mappings.txt
+  # Suppress -Wunused warnings in all files, apart from the ones under `foo/`.
+  [unused]
+  src:*
+  src:*foo/*=emit
+  $ clang -Wunused --warning-suppression-mappings=suppression_mappings.txt my/user/code.cpp
+  # prints warning: unused function 'unused_func2', but no warnings for `unused_func1`.
+
+Format
+======
+
+Warning suppression mappings uses the same format as
+:doc:`SanitizerSpecialCaseList`.
+
+Sections describe which diagnostic group's behaviour to change, e.g.
+``[unused]``. When a diagnostic is matched by multiple sections, the latest
+section takes precedence.
+
+Afterwards in each section, users can have multiple entities that match source
+files based on the globs. These entities look like ``src:*/my/dir/*``.
+Users can also use the ``emit`` category to exclude a subdirectory from
+suppression.
+Source files are matched against these globs either:
+
+- as paths relative to the current working directory
+- as absolute paths.
+
+When a source file matches multiple globs in a section, the longest one takes
+precedence.
+
+.. code-block:: bash
+
+    # Lines starting with # are ignored.
+    # Configure suppression globs for `-Wunused` warnings
+    [unused]
+    # Suppress on all files by default.
+    src:*
+    # But enforce for all the sources under foo/.
+    src:*foo/*=emit
+
+    # unused-function warnings are a subgroup of `-Wunused`. So this section
+    # takes precedence over the previous one for unused-function warnings, but
+    # not for unused-variable warnings.
+    [unused-function]
+    # Only suppress for sources under bar/.
+    src:*bar/*

clang/docs/index.rst

@@ -22,6 +22,7 @@ Using Clang as a Compiler
    ClangCommandLineReference
    AttributeReference
    DiagnosticsReference
+   WarningSuppressionMappings
    CrossCompilation
    ClangStaticAnalyzer
    ThreadSafetyAnalysis

clang/include/clang/Basic/Diagnostic.h

@@ -20,6 +20,7 @@
 #include "clang/Basic/Specifiers.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/FunctionExtras.h"
 #include "llvm/ADT/IntrusiveRefCntPtr.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
@@ -40,6 +41,10 @@
 namespace llvm {
 class Error;
 class raw_ostream;
+class MemoryBuffer;
+namespace vfs {
+class FileSystem;
+} // namespace vfs
 } // namespace llvm
 
 namespace clang {
@@ -555,6 +560,10 @@ class DiagnosticsEngine : public RefCountedBase<DiagnosticsEngine> {
   void *ArgToStringCookie = nullptr;
   ArgToStringFnTy ArgToStringFn;
 
+  /// Whether the diagnostic should be suppressed in FilePath.
+  llvm::unique_function<bool(diag::kind, llvm::StringRef /*FilePath*/) const>
+      DiagSuppressionMapping;
+
 public:
   explicit DiagnosticsEngine(IntrusiveRefCntPtr<DiagnosticIDs> Diags,
                              IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts,
@@ -946,6 +955,27 @@ class DiagnosticsEngine : public RefCountedBase<DiagnosticsEngine> {
     return (Level)Diags->getDiagnosticLevel(DiagID, Loc, *this);
   }
 
+  /// Diagnostic suppression mappings can be used to suppress specific
+  /// diagnostics in specific files.
+  /// Mapping file is expected to be a special case list with sections denoting
+  /// diagnostic groups and `src` entries for globs to suppress. `emit` category
+  /// can be used to disable suppression. Longest glob that matches a filepath
+  /// takes precedence. For example:
+  ///   [unused]
+  ///   src:clang/*
+  ///   src:clang/foo/*=emit
+  ///   src:clang/foo/bar/*
+  ///
+  /// Such a mappings file suppress all diagnostics produced by -Wunused in all
+  /// sources under `clang/` directory apart from `clang/foo/`. Diagnostics
+  /// under `clang/foo/bar/` will also be suppressed. Note that the FilePath is
+  /// matched against the globs as-is.
+  /// These take presumed locations into account, and can still be overriden by
+  /// clang-diagnostics pragmas.
+  void setDiagSuppressionMapping(llvm::MemoryBuffer &Input);
+  bool isSuppressedViaMapping(diag::kind DiagId,
+                              llvm::StringRef FilePath) const;
+
   /// Issue the message to the client.
   ///
   /// This actually returns an instance of DiagnosticBuilder which emits the
@@ -1759,7 +1789,7 @@ const char ToggleHighlight = 127;
 /// warning options specified on the command line.
 void ProcessWarningOptions(DiagnosticsEngine &Diags,
                            const DiagnosticOptions &Opts,
-                           bool ReportDiags = true);
+                           llvm::vfs::FileSystem &VFS, bool ReportDiags = true);
 void EscapeStringForDiagnostic(StringRef Str, SmallVectorImpl<char> &OutStr);
 } // namespace clang
 

clang/include/clang/Basic/DiagnosticDriverKinds.td

@@ -834,4 +834,7 @@ def err_drv_triple_version_invalid : Error<
 
 def warn_missing_include_dirs : Warning<
   "no such include directory: '%0'">, InGroup<MissingIncludeDirs>, DefaultIgnore;
+
+def err_drv_malformed_warning_suppression_mapping : Error<
+  "failed to process suppression mapping file '%0': %1">;
 }

clang/include/clang/Basic/DiagnosticOptions.h

@@ -108,6 +108,9 @@ class DiagnosticOptions : public RefCountedBase<DiagnosticOptions>{
   /// The file to serialize diagnostics to (non-appending).
   std::string DiagnosticSerializationFile;
 
+  /// Path for the file that defines diagnostic suppression mappings.
+  std::string DiagnosticSuppressionMappingsFile;
+
   /// The list of -W... options used to alter the diagnostic mappings, with the
   /// prefixes removed.
   std::vector<std::string> Warnings;

clang/include/clang/Driver/Options.td

@@ -9019,3 +9019,8 @@ def wasm_opt : Flag<["--"], "wasm-opt">,
   Group<m_Group>,
   HelpText<"Enable the wasm-opt optimizer (default)">,
   MarshallingInfoNegativeFlag<LangOpts<"NoWasmOpt">>;
+
+def warning_suppression_mappings_EQ : Joined<["--"],
+  "warning-suppression-mappings=">, Group<Diag_Group>,
+  HelpText<"File containing diagnostic suppresion mappings. See user manual "
+  "for file format.">, Visibility<[ClangOption, CC1Option]>;

clang/include/clang/Frontend/CompilerInstance.h

@@ -24,6 +24,7 @@
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/BuryPointer.h"
 #include "llvm/Support/FileSystem.h"
+#include "llvm/Support/VirtualFileSystem.h"
 #include <cassert>
 #include <list>
 #include <memory>
@@ -701,11 +702,10 @@ class CompilerInstance : public ModuleLoader {
   /// used by some diagnostics printers (for logging purposes only).
   ///
   /// \return The new object on success, or null on failure.
-  static IntrusiveRefCntPtr<DiagnosticsEngine>
-  createDiagnostics(DiagnosticOptions *Opts,
-                    DiagnosticConsumer *Client = nullptr,
-                    bool ShouldOwnClient = true,
-                    const CodeGenOptions *CodeGenOpts = nullptr);
+  static IntrusiveRefCntPtr<DiagnosticsEngine> createDiagnostics(
+      DiagnosticOptions *Opts, DiagnosticConsumer *Client = nullptr,
+      bool ShouldOwnClient = true, const CodeGenOptions *CodeGenOpts = nullptr,
+      IntrusiveRefCntPtr<llvm::vfs::FileSystem> VFS = nullptr);
 
   /// Create the file manager and replace any existing one with it.
   ///