[SpecialCaseList] Add option to use Globs instead of Regex to match patterns

Add an option in `SpecialCaseList` to use Globs instead of Regex to match patterns. `GlobPattern` was extended in https://reviews.llvm.org/D153587 to support brace expansions which allows us to use patterns like `*/src/foo.{c,cpp}`. It turns out that most patterns only take advantage of `*` so using Regex was overkill and required lots of escaping in practice. This often led to bugs due to forgetting to escape special characters.

Since this would be a breaking change, we temporarily support Regex by default and use Globs when `#!special-case-list-v2` is the first line in the file. Users should switch to the glob format described in https://llvm.org/doxygen/classllvm_1_1GlobPattern.html. For example, `(abc|def)` should become `{abc,def}`.

See discussion in https://reviews.llvm.org/D152762 and https://discourse.llvm.org/t/use-glob-instead-of-regex-for-specialcaselists/71666.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D154014

Author: ellishg

clang/docs/SanitizerSpecialCaseList.rst

@@ -15,7 +15,7 @@ file at compile-time.
 Goal and usage
 ==============
 
-User of sanitizer tools, such as :doc:`AddressSanitizer`, :doc:`ThreadSanitizer`
+Users of sanitizer tools, such as :doc:`AddressSanitizer`, :doc:`ThreadSanitizer`
 or :doc:`MemorySanitizer` may want to disable or alter some checks for
 certain source-level entities to:
 
@@ -54,48 +54,58 @@ Format
 Ignorelists consist of entries, optionally grouped into sections. Empty lines
 and lines starting with "#" are ignored.
 
-Section names are regular expressions written in square brackets that denote
+.. note::
+
+  In `D154014 <https://reviews.llvm.org/D154014>`_ we transitioned to using globs instead
+  of regexes to match patterns in special case lists. Since this was a
+  breaking change, we will temporarily support the original behavior using
+  regexes. If ``#!special-case-list-v2`` is the first line of the file, then
+  we will use the new behavior using globs. For more details, see
+  `this discourse post <https://discourse.llvm.org/t/use-glob-instead-of-regex-for-specialcaselists/71666>`_.
+
+
+Section names are globs written in square brackets that denote
 which sanitizer the following entries apply to. For example, ``[address]``
-specifies AddressSanitizer while ``[cfi-vcall|cfi-icall]`` specifies Control
+specifies AddressSanitizer while ``[{cfi-vcall,cfi-icall}]`` specifies Control
 Flow Integrity virtual and indirect call checking. Entries without a section
 will be placed under the ``[*]`` section applying to all enabled sanitizers.
 
-Entries contain an entity type, followed by a colon and a regular expression,
+Entries contain an entity type, followed by a colon and a glob,
 specifying the names of the entities, optionally followed by an equals sign and
-a tool-specific category, e.g. ``fun:*ExampleFunc=example_category``.  The
-meaning of ``*`` in regular expression for entity names is different - it is
-treated as in shell wildcarding. Two generic entity types are ``src`` and
+a tool-specific category, e.g. ``fun:*ExampleFunc=example_category``.
+Two generic entity types are ``src`` and
 ``fun``, which allow users to specify source files and functions, respectively.
 Some sanitizer tools may introduce custom entity types and categories - refer to
 tool-specific docs.
 
 .. code-block:: bash
 
+    #!special-case-list-v2
+    # The line above is explained in the note above
     # Lines starting with # are ignored.
-    # Turn off checks for the source file (use absolute path or path relative
-    # to the current working directory):
-    src:/path/to/source/file.c
+    # Turn off checks for the source file
+    # Entries without sections are placed into [*] and apply to all sanitizers
+    src:path/to/source/file.c
+    src:*/source/file.c
     # Turn off checks for this main file, including files included by it.
     # Useful when the main file instead of an included file should be ignored.
     mainfile:file.c
     # Turn off checks for a particular functions (use mangled names):
-    fun:MyFooBar
     fun:_Z8MyFooBarv
-    # Extended regular expressions are supported:
-    fun:bad_(foo|bar)
+    # Glob brace expansions and character ranges are supported
+    fun:bad_{foo,bar}
     src:bad_source[1-9].c
-    # Shell like usage of * is supported (* is treated as .*):
+    # "*" matches zero or more characters
     src:bad/sources/*
     fun:*BadFunction*
     # Specific sanitizer tools may introduce categories.
     src:/special/path/*=special_sources
     # Sections can be used to limit ignorelist entries to specific sanitizers
     [address]
     fun:*BadASanFunc*
-    # Section names are regular expressions
-    [cfi-vcall|cfi-icall]
+    # Section names are globs
+    [{cfi-vcall,cfi-icall}]
     fun:*BadCfiCall
-    # Entries without sections are placed into [*] and apply to all sanitizers
 
 ``mainfile`` is similar to applying ``-fno-sanitize=`` to a set of files but
 does not need plumbing into the build system. This works well for internal

clang/lib/Basic/ProfileList.cpp

@@ -36,8 +36,8 @@ class ProfileSpecialCaseList : public llvm::SpecialCaseList {
   bool isEmpty() const { return Sections.empty(); }
 
   bool hasPrefix(StringRef Prefix) const {
-    for (auto &SectionIter : Sections)
-      if (SectionIter.Entries.count(Prefix) > 0)
+    for (const auto &It : Sections)
+      if (It.second.Entries.count(Prefix) > 0)
         return true;
     return false;
   }

clang/lib/Basic/SanitizerSpecialCaseList.cpp

@@ -37,7 +37,8 @@ SanitizerSpecialCaseList::createOrDie(const std::vector<std::string> &Paths,
 }
 
 void SanitizerSpecialCaseList::createSanitizerSections() {
-  for (auto &S : Sections) {
+  for (auto &It : Sections) {
+    auto &S = It.second;
     SanitizerMask Mask;
 
 #define SANITIZER(NAME, ID)                                                    \

llvm/include/llvm/Support/SpecialCaseList.h

@@ -5,54 +5,15 @@
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //===----------------------------------------------------------------------===//
 //
-// This is a utility class used to parse user-provided text files with
-// "special case lists" for code sanitizers. Such files are used to
-// define an "ABI list" for DataFlowSanitizer and allow/exclusion lists for
-// sanitizers like AddressSanitizer or UndefinedBehaviorSanitizer.
-//
-// Empty lines and lines starting with "#" are ignored. Sections are defined
-// using a '[section_name]' header and can be used to specify sanitizers the
-// entries below it apply to. Section names are regular expressions, and
-// entries without a section header match all sections (e.g. an '[*]' header
-// is assumed.)
-// The remaining lines should have the form:
-//   prefix:wildcard_expression[=category]
-// If category is not specified, it is assumed to be empty string.
-// Definitions of "prefix" and "category" are sanitizer-specific. For example,
-// sanitizer exclusion support prefixes "src", "mainfile", "fun" and "global".
-// Wildcard expressions define, respectively, source files, main files,
-// functions or globals which shouldn't be instrumented.
-// Examples of categories:
-//   "functional": used in DFSan to list functions with pure functional
-//                 semantics.
-//   "init": used in ASan exclusion list to disable initialization-order bugs
-//           detection for certain globals or source files.
-// Full special case list file example:
-// ---
-// [address]
-// # Excluded items:
-// fun:*_ZN4base6subtle*
-// global:*global_with_bad_access_or_initialization*
-// global:*global_with_initialization_issues*=init
-// type:*Namespace::ClassName*=init
-// src:file_with_tricky_code.cc
-// src:ignore-global-initializers-issues.cc=init
-// mainfile:main_file.cc
-//
-// [dataflow]
-// # Functions with pure functional semantics:
-// fun:cos=functional
-// fun:sin=functional
-// ---
-// Note that the wild card is in fact an llvm::Regex, but * is automatically
-// replaced with .*
+// This file implements a Special Case List for code sanitizers.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_SUPPORT_SPECIALCASELIST_H
 #define LLVM_SUPPORT_SPECIALCASELIST_H
 
 #include "llvm/ADT/StringMap.h"
+#include "llvm/Support/GlobPattern.h"
 #include "llvm/Support/Regex.h"
 #include <memory>
 #include <string>
@@ -66,6 +27,45 @@ namespace vfs {
 class FileSystem;
 }
 
+/// This is a utility class used to parse user-provided text files with
+/// "special case lists" for code sanitizers. Such files are used to
+/// define an "ABI list" for DataFlowSanitizer and allow/exclusion lists for
+/// sanitizers like AddressSanitizer or UndefinedBehaviorSanitizer.
+///
+/// Empty lines and lines starting with "#" are ignored. Sections are defined
+/// using a '[section_name]' header and can be used to specify sanitizers the
+/// entries below it apply to. Section names are globs, and
+/// entries without a section header match all sections (e.g. an '[*]' header
+/// is assumed.)
+/// The remaining lines should have the form:
+///   prefix:glob_pattern[=category]
+/// If category is not specified, it is assumed to be empty string.
+/// Definitions of "prefix" and "category" are sanitizer-specific. For example,
+/// sanitizer exclusion support prefixes "src", "mainfile", "fun" and "global".
+/// "glob_pattern" defines source files, main files, functions or globals which
+/// shouldn't be instrumented.
+/// Examples of categories:
+///   "functional": used in DFSan to list functions with pure functional
+///                 semantics.
+///   "init": used in ASan exclusion list to disable initialization-order bugs
+///           detection for certain globals or source files.
+/// Full special case list file example:
+/// ---
+/// [address]
+/// # Excluded items:
+/// fun:*_ZN4base6subtle*
+/// global:*global_with_bad_access_or_initialization*
+/// global:*global_with_initialization_issues*=init
+/// type:*Namespace::ClassName*=init
+/// src:file_with_tricky_code.cc
+/// src:ignore-global-initializers-issues.cc=init
+/// mainfile:main_file.cc
+///
+/// [dataflow]
+/// # Functions with pure functional semantics:
+/// fun:cos=functional
+/// fun:sin=functional
+/// ---
 class SpecialCaseList {
 public:
   /// Parses the special case list entries from files. On failure, returns
@@ -88,7 +88,7 @@ class SpecialCaseList {
   /// \code
   ///   @Prefix:<E>=@Category
   /// \endcode
-  /// where @Query satisfies wildcard expression <E> in a given @Section.
+  /// where @Query satisfies the glob <E> in a given @Section.
   bool inSection(StringRef Section, StringRef Prefix, StringRef Query,
                  StringRef Category = StringRef()) const;
 
@@ -97,7 +97,7 @@ class SpecialCaseList {
   /// \code
   ///   @Prefix:<E>=@Category
   /// \endcode
-  /// where @Query satisfies wildcard expression <E> in a given @Section.
+  /// where @Query satisfies the glob <E> in a given @Section.
   /// Returns zero if there is no exclusion entry corresponding to this
   /// expression.
   unsigned inSectionBlame(StringRef Section, StringRef Prefix, StringRef Query,
@@ -114,36 +114,36 @@ class SpecialCaseList {
   SpecialCaseList(SpecialCaseList const &) = delete;
   SpecialCaseList &operator=(SpecialCaseList const &) = delete;
 
-  /// Represents a set of regular expressions.  Regular expressions which are
-  /// "literal" (i.e. no regex metacharacters) are stored in Strings.  The
-  /// reason for doing so is efficiency; StringMap is much faster at matching
-  /// literal strings than Regex.
+  /// Represents a set of globs and their line numbers
   class Matcher {
   public:
-    bool insert(std::string Regexp, unsigned LineNumber, std::string &REError);
+    Error insert(StringRef Pattern, unsigned LineNumber, bool UseRegex);
     // Returns the line number in the source file that this query matches to.
     // Returns zero if no match is found.
     unsigned match(StringRef Query) const;
 
   private:
-    StringMap<unsigned> Strings;
+    StringMap<std::pair<GlobPattern, unsigned>> Globs;
     std::vector<std::pair<std::unique_ptr<Regex>, unsigned>> RegExes;
   };
 
   using SectionEntries = StringMap<StringMap<Matcher>>;
 
   struct Section {
     Section(std::unique_ptr<Matcher> M) : SectionMatcher(std::move(M)){};
+    Section() : Section(std::make_unique<Matcher>()) {}
 
     std::unique_ptr<Matcher> SectionMatcher;
     SectionEntries Entries;
   };
 
-  std::vector<Section> Sections;
+  StringMap<Section> Sections;
+
+  Expected<Section *> addSection(StringRef SectionStr, unsigned LineNo,
+                                 bool UseGlobs = true);
 
   /// Parses just-constructed SpecialCaseList entries from a memory buffer.
-  bool parse(const MemoryBuffer *MB, StringMap<size_t> &SectionsMap,
-             std::string &Error);
+  bool parse(const MemoryBuffer *MB, std::string &Error);
 
   // Helper method for derived classes to search by Prefix, Query, and Category
   // once they have already resolved a section entry.