compiletest: Fix flaky Android gdb test runs #38883
Merged
+39
−42
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Local testing showed that I was able to reproduce an error where debuginfo tests on Android would fail with "connection reset by peer". Further investigation turned out that the gdb tests are android with bit of process management: * First an `adb forward` command is run to ensure that the host's port 5039 is the same as the emulator's. * Next an `adb shell` command is run to execute the `gdbserver` executable inside the emulator. This gdb server will attach to port 5039 and listen for remote gdb debugging sessions. * Finally, we run `gdb` on the host (not in the emulator) and then connect to this gdb server to send it commands. The problem was happening when the host's gdb was failing to connect to the remote gdbserver running inside the emulator. The previous test for this was that after `adb shell` executed we'd sleep for a second and then attempt to make a TCP connection to port 5039. If successful we'd run gdb and on failure we'd sleep again. It turns out, however, that as soon as we've executed `adb forward` all TCP connections to 5039 will succeed. This means that we would only ever sleep for at most one second, and if this wasn't enough time we'd just fail later because we would assume that gdbserver had started but it may not have done so yet. This commit fixes these issues by removing the TCP connection to test if gdbserver is ready to go. Instead we read the stdout of the process and wait for it to print that it's listening at which point we start running gdb. I've found that locally at least I was unable to reproduce the failure after these changes. Closes rust-lang#38710
|
r? @brson (rust_highfive has picked a reviewer for you, use r? to override) |
|
@bors r+ p=1 |
|
📌 Commit 9ced901 has been approved by |
|
⌛ Testing commit 9ced901 with merge 4ecca05... |
|
💔 Test failed - status-travis |
|
@bors: retry
* network error
…On Sat, Jan 7, 2017 at 12:02 AM bors ***@***.***> wrote:
💔 Test failed - status-travis
<https://travis-ci.org/rust-lang/rust/builds/189744685>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#38883 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAD95MC9JQfru4t0q3NHNWZY4AhOSs94ks5rP0Z8gaJpZM4Lc-pf>
.
|
|
⌛ Testing commit 9ced901 with merge af674f2... |
|
💔 Test failed - status-appveyor |
|
@bors: retry
* weird backtrace error on windows?
…On Sat, Jan 7, 2017 at 11:21 AM, bors ***@***.***> wrote:
💔 Test failed - status-appveyor
<https://ci.appveyor.com/project/rust-lang/rust/build/1.0.1445>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#38883 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAD95M4i-d1N1zE_VWwyQ7u1ScX2X0q7ks5rP-WxgaJpZM4Lc-pf>
.
|
bors
added a commit
that referenced
this pull request
Jan 8, 2017
compiletest: Fix flaky Android gdb test runs Local testing showed that I was able to reproduce an error where debuginfo tests on Android would fail with "connection reset by peer". Further investigation turned out that the gdb tests are android with bit of process management: * First an `adb forward` command is run to ensure that the host's port 5039 is the same as the emulator's. * Next an `adb shell` command is run to execute the `gdbserver` executable inside the emulator. This gdb server will attach to port 5039 and listen for remote gdb debugging sessions. * Finally, we run `gdb` on the host (not in the emulator) and then connect to this gdb server to send it commands. The problem was happening when the host's gdb was failing to connect to the remote gdbserver running inside the emulator. The previous test for this was that after `adb shell` executed we'd sleep for a second and then attempt to make a TCP connection to port 5039. If successful we'd run gdb and on failure we'd sleep again. It turns out, however, that as soon as we've executed `adb forward` all TCP connections to 5039 will succeed. This means that we would only ever sleep for at most one second, and if this wasn't enough time we'd just fail later because we would assume that gdbserver had started but it may not have done so yet. This commit fixes these issues by removing the TCP connection to test if gdbserver is ready to go. Instead we read the stdout of the process and wait for it to print that it's listening at which point we start running gdb. I've found that locally at least I was unable to reproduce the failure after these changes. Closes #38710
|
☀️ Test successful - status-appveyor, status-travis |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Local testing showed that I was able to reproduce an error where debuginfo tests
on Android would fail with "connection reset by peer". Further investigation
turned out that the gdb tests are android with bit of process management:
adb forwardcommand is run to ensure that the host's port 5039 isthe same as the emulator's.
adb shellcommand is run to execute thegdbserverexecutableinside the emulator. This gdb server will attach to port 5039 and listen for
remote gdb debugging sessions.
gdbon the host (not in the emulator) and then connect tothis gdb server to send it commands.
The problem was happening when the host's gdb was failing to connect to the
remote gdbserver running inside the emulator. The previous test for this was
that after
adb shellexecuted we'd sleep for a second and then attempt to makea TCP connection to port 5039. If successful we'd run gdb and on failure we'd
sleep again.
It turns out, however, that as soon as we've executed
adb forwardall TCPconnections to 5039 will succeed. This means that we would only ever sleep for
at most one second, and if this wasn't enough time we'd just fail later because
we would assume that gdbserver had started but it may not have done so yet.
This commit fixes these issues by removing the TCP connection to test if
gdbserver is ready to go. Instead we read the stdout of the process and wait for
it to print that it's listening at which point we start running gdb. I've found
that locally at least I was unable to reproduce the failure after these changes.
Closes #38710