Skip to content

Set ordering of acceptance tests #1420

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 2, 2021

Conversation

ghoneycutt
Copy link
Contributor

This allows pre_run to be removed from the spec_helper_acceptance as the
tests will run in a specific order. The pre_run just included the
mysql::server class which now happens first thing.

This allows pre_run to be removed from the spec_helper_acceptance as the
tests will run in a specific order. The pre_run just included the
mysql::server class which now happens first thing.
@ghoneycutt ghoneycutt requested a review from a team as a code owner July 8, 2021 21:23
@ghoneycutt
Copy link
Contributor Author

Based off of @pmcmaw 's previous work in trying to debug acceptance testing with Debian.

@daianamezdrea
Copy link
Contributor

Hi @ghoneycutt, every acceptance test should be independent, if we merge this PR, we'll create dependencies between the tests, which we don't want. Thank you ! (https://softwareengineering.stackexchange.com/questions/46913/writing-acceptance-test-cases)

@ghoneycutt
Copy link
Contributor Author

The acceptance tests already run in a specific order and when someone adds a new file that order will change. This only makes what is already happening explicit.

Also the tests are inherently dependent on each other because they are making changes to a system and checking the state of the system at that time. Unless you run each test block from the code on its own system in the same starting state, this will be the case and our testing frameworks (litmus and beaker) and processes in the community do not support that.

@ghoneycutt
Copy link
Contributor Author

@daianamezdrea @pmcmaw Could you please have a look at this.

@adrianiurca
Copy link
Contributor

Hey @ghoneycutt, thank you for your contribution!

kind regards,
@adrianiurca

@adrianiurca adrianiurca merged commit cb97c87 into puppetlabs:main Aug 2, 2021
@ghoneycutt ghoneycutt deleted the simplify_acceptance_tests branch August 3, 2021 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants