Crash in clang-format (17.x) with AlignArrayOfStructures set to Left

[tru]

I am seeing a crash in clang-format on the following reduced code:

static const Test::Bar a[1] = {
    #define buf(a, b) \
      { #a, #b },

    { Test, bar },
};

With the following .clang-format

BasedOnStyle : Microsoft 
AlignArrayOfStructures: Left

The crash happens here:

crash backlog

PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: c:\\code\\llvm\\llvm-project\\out\\debug\\bin\\clang-format.exe reduced.cpp
Exception Code: 0xC0000005
 #0 0x00007ff67f23bd01 clang::Token::is(enum clang::tok::TokenKind) const C:\code\llvm\llvm-project\clang\include\clang\Lex\Token.h:98:0
 #1 0x00007ff67f23bbed clang::format::FormatToken::is(enum clang::tok::TokenKind) const C:\code\llvm\llvm-project\clang\lib\Format\FormatToken.h:560:0
 #2 0x00007ff67f3943cd clang::format::WhitespaceManager::getCells(unsigned int, unsigned int) C:\code\llvm\llvm-project\clang\lib\Format\WhitespaceManager.cpp:1402:0
 #3 0x00007ff67f3936d7 clang::format::WhitespaceManager::alignArrayInitializers(unsigned int, unsigned int) C:\code\llvm\llvm-project\clang\lib\Format\WhitespaceManager.cpp:1222:0
 #4 0x00007ff67f390427 clang::format::WhitespaceManager::alignArrayInitializers(void) C:\code\llvm\llvm-project\clang\lib\Format\WhitespaceManager.cpp:1206:0
 #5 0x00007ff67f38e592 clang::format::WhitespaceManager::generateReplacements(void) C:\code\llvm\llvm-project\clang\lib\Format\WhitespaceManager.cpp:112:0
 #6 0x00007ff67f226dca clang::format::`anonymous namespace'::Formatter::analyze C:\code\llvm\llvm-project\clang\lib\Format\Format.cpp:2391:0
 #7 0x00007ff67f34f6c7 clang::format::TokenAnalyzer::process(bool) C:\code\llvm\llvm-project\clang\lib\Format\TokenAnalyzer.cpp:132:0
 #8 0x00007ff67f22f4c5 clang::format::internal::reformat::<lambda_12>::operator() C:\code\llvm\llvm-project\clang\lib\Format\Format.cpp:3673:0
 #9 0x00007ff67f22f450 std::invoke<`lambda at C:/code/llvm/llvm-project/clang/lib/Format/Format.cpp:3672:23' &,const clang::format::Environment &> C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.37.32822\include\type_traits:1762:0
#10 0x00007ff67f22f35b std::_Func_impl_no_alloc<`lambda at C:/code/llvm/llvm-project/clang/lib/Format/Format.cpp:3672:23',std::pair<clang::tooling::Replacements,unsigned int>,const clang::format::Environment &>::_Do_call C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.37.32822\include\functional:841:0
#11 0x00007ff67f2322e5 std::_Func_class<struct std::pair<class clang::tooling::Replacements, unsigned int>, class clang::format::Environment const &>::operator()(class clang::format::Environment const &) const C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.37.32822\include\functional:883:0
#12 0x00007ff67f21d722 clang::format::internal::reformat(struct clang::format::FormatStyle const &, class llvm::StringRef, class llvm::ArrayRef<class clang::tooling::Range>, unsigned int, unsigned int, unsigned int, class llvm::StringRef, struct clang::format::FormattingAttemptStatus *) C:\code\llvm\llvm-project\clang\lib\Format\Format.cpp:3688:0
#13 0x00007ff67f21e5e0 clang::format::reformat(struct clang::format::FormatStyle const &, class llvm::StringRef, class llvm::ArrayRef<class clang::tooling::Range>, class llvm::StringRef, struct clang::format::FormattingAttemptStatus *) C:\code\llvm\llvm-project\clang\lib\Format\Format.cpp:3731:0
#14 0x00007ff67f113d1d clang::format::format C:\code\llvm\llvm-project\clang\tools\clang-format\ClangFormat.cpp:499:0
#15 0x00007ff67f1128c7 main C:\code\llvm\llvm-project\clang\tools\clang-format\ClangFormat.cpp:631:0
#16 0x00007ff67f6c001c invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78:0
#17 0x00007ff67f6c001c __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288:0
#18 0x00007ff8395f257d (C:\WINDOWS\System32\KERNEL32.DLL+0x1257d)
#19 0x00007ff83a34aa78 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x5aa78)

The actual crash is because getNextNonComment() returns nullptr inside WhitespaceManager::getCells()

[llvmbot]

@llvm/issue-subscribers-clang-format

Author: Tobias Hieta (tru)

I am seeing a crash in clang-format on the following reduced code:

static const Test::Bar a[1] = {
    #define buf(a, b) \
      { #a, #b },

    { Test, bar },
};

With the following .clang-format

BasedOnStyle : Microsoft 
AlignArrayOfStructures: Left

The crash happens here:

<details>
<summary>
crash backlog
</summary>

PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: c:\\code\\llvm\\llvm-project\\out\\debug\\bin\\clang-format.exe reduced.cpp
Exception Code: 0xC0000005
 #<!-- -->0 0x00007ff67f23bd01 clang::Token::is(enum clang::tok::TokenKind) const C:\code\llvm\llvm-project\clang\include\clang\Lex\Token.h:98:0
 #<!-- -->1 0x00007ff67f23bbed clang::format::FormatToken::is(enum clang::tok::TokenKind) const C:\code\llvm\llvm-project\clang\lib\Format\FormatToken.h:560:0
 #<!-- -->2 0x00007ff67f3943cd clang::format::WhitespaceManager::getCells(unsigned int, unsigned int) C:\code\llvm\llvm-project\clang\lib\Format\WhitespaceManager.cpp:1402:0
 #<!-- -->3 0x00007ff67f3936d7 clang::format::WhitespaceManager::alignArrayInitializers(unsigned int, unsigned int) C:\code\llvm\llvm-project\clang\lib\Format\WhitespaceManager.cpp:1222:0
 #<!-- -->4 0x00007ff67f390427 clang::format::WhitespaceManager::alignArrayInitializers(void) C:\code\llvm\llvm-project\clang\lib\Format\WhitespaceManager.cpp:1206:0
 #<!-- -->5 0x00007ff67f38e592 clang::format::WhitespaceManager::generateReplacements(void) C:\code\llvm\llvm-project\clang\lib\Format\WhitespaceManager.cpp:112:0
 #<!-- -->6 0x00007ff67f226dca clang::format::`anonymous namespace'::Formatter::analyze C:\code\llvm\llvm-project\clang\lib\Format\Format.cpp:2391:0
 #<!-- -->7 0x00007ff67f34f6c7 clang::format::TokenAnalyzer::process(bool) C:\code\llvm\llvm-project\clang\lib\Format\TokenAnalyzer.cpp:132:0
 #<!-- -->8 0x00007ff67f22f4c5 clang::format::internal::reformat::&lt;lambda_12&gt;::operator() C:\code\llvm\llvm-project\clang\lib\Format\Format.cpp:3673:0
 #<!-- -->9 0x00007ff67f22f450 std::invoke&lt;`lambda at C:/code/llvm/llvm-project/clang/lib/Format/Format.cpp:3672:23' &amp;,const clang::format::Environment &amp;&gt; C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.37.32822\include\type_traits:1762:0
#<!-- -->10 0x00007ff67f22f35b std::_Func_impl_no_alloc&lt;`lambda at C:/code/llvm/llvm-project/clang/lib/Format/Format.cpp:3672:23',std::pair&lt;clang::tooling::Replacements,unsigned int&gt;,const clang::format::Environment &amp;&gt;::_Do_call C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.37.32822\include\functional:841:0
#<!-- -->11 0x00007ff67f2322e5 std::_Func_class&lt;struct std::pair&lt;class clang::tooling::Replacements, unsigned int&gt;, class clang::format::Environment const &amp;&gt;::operator()(class clang::format::Environment const &amp;) const C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.37.32822\include\functional:883:0
#<!-- -->12 0x00007ff67f21d722 clang::format::internal::reformat(struct clang::format::FormatStyle const &amp;, class llvm::StringRef, class llvm::ArrayRef&lt;class clang::tooling::Range&gt;, unsigned int, unsigned int, unsigned int, class llvm::StringRef, struct clang::format::FormattingAttemptStatus *) C:\code\llvm\llvm-project\clang\lib\Format\Format.cpp:3688:0
#<!-- -->13 0x00007ff67f21e5e0 clang::format::reformat(struct clang::format::FormatStyle const &amp;, class llvm::StringRef, class llvm::ArrayRef&lt;class clang::tooling::Range&gt;, class llvm::StringRef, struct clang::format::FormattingAttemptStatus *) C:\code\llvm\llvm-project\clang\lib\Format\Format.cpp:3731:0
#<!-- -->14 0x00007ff67f113d1d clang::format::format C:\code\llvm\llvm-project\clang\tools\clang-format\ClangFormat.cpp:499:0
#<!-- -->15 0x00007ff67f1128c7 main C:\code\llvm\llvm-project\clang\tools\clang-format\ClangFormat.cpp:631:0
#<!-- -->16 0x00007ff67f6c001c invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78:0
#<!-- -->17 0x00007ff67f6c001c __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288:0
#<!-- -->18 0x00007ff8395f257d (C:\WINDOWS\System32\KERNEL32.DLL+0x1257d)
#<!-- -->19 0x00007ff83a34aa78 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x5aa78)

</details>

The actual crash is because getNextNonComment() returns nullptr inside WhitespaceManager::getCells()

[owenca]

I am seeing a crash in clang-format on the following reduced code:

static const Test::Bar a[1] = {
    #define buf(a, b) \
      { #a, #b },

    { Test, bar },
};

With the following .clang-format

BasedOnStyle : Microsoft 
AlignArrayOfStructures: Left

It doesn't crash on the current main branch and is formatted as:

static const Test::Bar a[1] = {
#define buf(a, b)                                                                                                      \
    {                                                                                                                  \
        #a, #b                                                                                                         \
    }                                                                                                                  \
    ,

    {Test, bar},
};

[owenca]

"Fixed" by 063e3fe.

[tru]

Confirmed that cherry-picking 063e3fe to 17.x branch also fixed my issue. Do we want to backport that one?

[owenca]

@rymiel (and @mydeveloperday and @HazardyKnusperkeks) are you comfortable with backporting it?

[rymiel]

I don't oppose it!

[HazardyKnusperkeks]

@rymiel (and @mydeveloperday and @HazardyKnusperkeks) are you comfortable with backporting it?

Why not? Crashes should be fixed.

[owenca]

@tru please go ahead then.

[tru]

/cherry-pick 063e3fe