[lldb-dap] TestDAP_exception_cpp test flakey

[Michael137]

I've seen this fail occasionally on the public macOS buildbots. Example: https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/as-lldb-cmake/15188/execution/node/97/log/?consoleFull

FAIL: LLDB (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/lldb-build/bin/clang-arm64) :: test_stopped_description (TestDAP_exception_cpp.TestDAP_exception_cpp)
Restore dir to: /Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/lldb-build/tools/lldb/test
======================================================================
ERROR: test_stopped_description (TestDAP_exception_cpp.TestDAP_exception_cpp)
   Test that exception description is shown correctly in stopped
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/lldb/test/API/tools/lldb-dap/exception/cpp/TestDAP_exception_cpp.py", line 26, in test_stopped_description
    self.assertIsNotNone(exceptionInfo["details"])
KeyError: 'details'
Config=arm64-/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/lldb-build/bin/clang
----------------------------------------------------------------------
Ran 1 test in 1.832s

FAILED (errors=1)

--

This assertion was added in #105905

[llvmbot]

@llvm/issue-subscribers-lldb

Author: Michael Buch (Michael137)

I've seen this fail occasionally on the public macOS buildbots. Example: https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/as-lldb-cmake/15188/execution/node/97/log/?consoleFull

FAIL: LLDB (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/lldb-build/bin/clang-arm64) :: test_stopped_description (TestDAP_exception_cpp.TestDAP_exception_cpp)
Restore dir to: /Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/lldb-build/tools/lldb/test
======================================================================
ERROR: test_stopped_description (TestDAP_exception_cpp.TestDAP_exception_cpp)
   Test that exception description is shown correctly in stopped
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/lldb/test/API/tools/lldb-dap/exception/cpp/TestDAP_exception_cpp.py", line 26, in test_stopped_description
    self.assertIsNotNone(exceptionInfo["details"])
KeyError: 'details'
Config=arm64-/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/lldb-build/bin/clang
----------------------------------------------------------------------
Ran 1 test in 1.832s

FAILED (errors=1)

--

This assertion was added in #105905

[Michael137]

@ashgti mind taking a look?

[Michael137]

Full build/test log attached:

consoleText.txt

[ashgti]

Since 'details' is missing, it looks like we're not getting the SBThread.GetCurrentException result here:

https://github.com/llvm/llvm-project/blob/main/lldb/tools/lldb-dap/lldb-dap.cpp#L1233-L1234

I'll see if I can reproduce this running locally.

[ashgti]

I've run this multiple times locally on a few commits today (my current main is 95b680e) but I haven't reproduced this locally, I'm not 100% sure why we're missing the c++ exception in some cases.

I'll keep an eye on https://ci.swift.org/job/llvm.org/view/LLDB/job/as-lldb-cmake/ and see if I can figure out anything from the CI logs. We might need to add some additional logging or something since I'm not currently reproducing this issue.

[ashgti]

For reference, there is another test that uses this API:

llvm-project/lldb/test/API/lang/objc/exceptions/TestObjCExceptions.py

Lines 239 to 258 in ec066d3

	def test_cxx_exceptions_at_abort(self):
	self.build()
	target = self.dbg.CreateTarget(self.getBuildArtifact("a.out"))
	self.assertTrue(target, VALID_TARGET)
	self.runCmd("run 2")
	# We should be stopped at pthread_kill because of an unhandled exception
	self.expect("thread list", substrs=["stopped", "stop reason = signal SIGABRT"])
	self.expect("thread exception", substrs=["exception ="])
	process = self.dbg.GetSelectedTarget().process
	thread = process.GetSelectedThread()
	self.assertTrue(thread.GetCurrentException().IsValid())
	# C++ exception backtraces are not exposed in the API (yet).
	self.assertFalse(thread.GetCurrentExceptionBacktrace().IsValid())

[Michael137]

Thanks for taking a look. I haven't seen it very frequently, so will probably be tricky to reproduce. But I'll ping if it happens again

[Michael137]

FYI, just saw this on the macOS Swift fork bots (running Intel): https://ci.swift.org/view/LLDB/job/oss-lldb-incremental-macos-cmake/8439/execution/node/423/log/

[ashgti]

Do you know what version of clang is used on the swift build bots? It looks like the other test has a filter for @skipIf(compiler="clang", compiler_version=["<", "13.0"]) but I think thats a somewhat older version of clang, if I'm not mistaken.

I could add that filter to the lldb-dap test to be safe though.

[Michael137]

Do you know what version of clang is used on the swift build bots? It looks like the other test has a filter for @skipIf(compiler="clang", compiler_version=["<", "13.0"]) but I think thats a somewhat older version of clang, if I'm not mistaken.

I could add that filter to the lldb-dap test to be safe though.

Pretty sure it uses a locally built top-of-tree clang to build the tests.