Closed
Description
Preconditions
- Any supported Magento version
- Any MFTF version
Steps to reproduce
Run the vendor/bin/mftf run:test
, vendor/bin/mftf run:group
or vendor/bin/mftf run:failed
command with failing tests
Expected result
The command should return a non-zero exit code as errors occured
Actual result
The command returns an exit code of zero
This is especially relevant in a CI environment where many job setups depend on the exit code of the run commands. In this case, we had a job showing as green even though the MFTF tests in the job failed.
This also blocks command chaining in the CLI, e.g. vendor/bin/mftf run:test FailingTest >> test.log && echo "Test success"
will always print out "Test success".
Metadata
Metadata
Assignees
Labels
No labels