fix(ci): Temporarily allow ember-release tests to fail in CI
#4227
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AbhiPrasad
approved these changes
Dec 3, 2021
size-limit report
|
onurtemizkan
pushed a commit
that referenced
this pull request
Dec 19, 2021
Right now, tests against the current release of ember are failing, because ember 4.0 brought changes our SDK isn't equipped to handle without breaking changes of its own. (See #4179.) Until we make those changes, we know that test scenario is going to fail. Meanwhile, GH is inconsistent in terms of what it considers success or failure in CI. GHA has a setting, `continue-on-error`, both for jobs[1] and for steps[2], which is essentially an xfail for a job within a workflow run (in other words, that job failing doesn't make the whole workflow run fail) and for a step within a job, respectively. This (in theory) allows processes which depend on the workflow or job succeeding to keep humming right along without the failure being silently swallowed. In our case, we have this set on the job running ember tests, so that if they fail, the entire workflow shouldn't. Indeed, if you look in the Actions tab, it shows up as green, in spite of the ember failures. But in terms of the little X or check next to commits/branches, and in terms of what it reports to craft, it shows up as red. This means that if anything else breaks, we won't know, since we'll already be expecting CI to be broken. It also prevents releases. This xfails the `ember-release` suite so that we can proceed with the rest of the repo as normal. We have the ticket linked above, and it's in our plan for the next major release (if it's not fixed before then), so we won't lose track of getting them fixed, even though we won't have the red X staring us in the face. [1] https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#jobsjob_idcontinue-on-error [2] https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#jobsjob_idstepscontinue-on-error
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Right now, tests against the current release of ember are failing, because ember 4.0 brought changes our SDK isn't equipped to handle without breaking changes of its own. (See #4179.) Until we make those changes, we know that test scenario is going to fail.
Meanwhile, GH is inconsistent in terms of what it considers success or failure in CI. GHA has a setting,
continue-on-error, both for jobs and for steps, which is essentially an xfail for a job within a workflow run (in other words, that job failing doesn't make the whole workflow run fail) and for a step within a job, respectively. This (in theory) allows processes which depend on the workflow or job succeeding to keep humming right along without the failure being silently swallowed.In our case, we have this set on the job running ember tests, so that if they fail, the entire workflow shouldn't. Indeed, if you look in the Actions tab, it shows up as green, in spite of the ember failures.
But in terms of the little X or check next to commits/branches, and in terms of what it reports to craft, it shows up as red.
This means that if anything else breaks, we won't know, since we'll already be expecting CI to be broken. It also prevents releases.
This xfails the
ember-releasesuite so that we can proceed with the rest of the repo as normal. We have the ticket linked above, and it's in our plan for the next major release (if it's not fixed before then), so we won't lose track of getting them fixed, even though we won't have the red X staring us in the face.