Skip to content

Commit 4fc3430

Browse files
okolesnykokolesnyk
authored
Merge pull request #316 from magento-devdocs/db_mftfdocs
Moving MFTF docs into this repo.
2 parents db370cf + cb5a2ba commit 4fc3430

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+13415
-10
lines changed

‎.github/CODEOWNERS

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# CODEOWNERS file for /docs/ folder.
2+
# Forces a review from other writers for anything within /docs/.
3+
/docs/ @magento/devdocs-admins

‎.github/CONTRIBUTING.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -160,7 +160,7 @@ Label| Description
160160
[create issue]: https://help.github.com/articles/creating-an-issue/
161161
[create pr]: https://help.github.com/articles/creating-a-pull-request/
162162
[Definition of Done]: https://devdocs.magento.com/guides/v2.2/contributor-guide/contributing_dod.html
163-
[devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md
163+
[devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.html
164164
[existing issues]: https://github.com/magento/magento2-functional-testing-framework/issues?q=is%3Aopen+is%3Aissue
165165
[existing PRs]: https://github.com/magento/magento2-functional-testing-framework/pulls?q=is%3Aopen+is%3Apr
166166
[GitHub documentation]: https://help.github.com/articles/syncing-a-fork

‎README.md

+19-8
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,15 @@
66

77
## Installation
88

9-
For the installation guidelines and system requirements, refer to [Getting Started](https://devdocs.magento.com/mftf/2.3/getting-started.html).
9+
For the installation guidelines and system requirements, refer to [Getting Started][].
1010

1111
## Contributing
1212

1313
We would appreciate your contributions to new components or new features, changes to the existing features, tests, documentation, specifications, bug fixes, optimizations, or just good suggestions.
1414
Report about an issue or request features opening a GitHub issue.
15-
Learn more about contributing in our [Contribution Guidelines](.github/CONTRIBUTING.md).
15+
Learn more about contributing in our [Contribution Guidelines][].
1616

17-
If you want to participate in the documentation work, see [DevDocs Contributing](https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md).
17+
If you want to participate in the documentation work, see [DevDocs Contributing][].
1818

1919
### Labels applied by the MFTF team
2020

@@ -55,15 +55,26 @@ These labels are applied by the MFTF development team to community contributed i
5555

5656
## Reporting security issues
5757

58-
To report security vulnerabilities and other security issues in the Magento software or web sites, send an email with the report at [[email protected]](mailto:[email protected]).
58+
To report security vulnerabilities and other security issues in the Magento software or web sites, send an email with the report at [[email protected]][].
5959
Do not report security issues using GitHub.
60-
Be sure to encrypt your e-mail with our [encryption key](https://info2.magento.com/rs/magentoenterprise/images/security_at_magento.asc) if it includes sensitive information.
61-
Learn more about reporting security issues [here](https://magento.com/security/reporting-magento-security-issue).
60+
Be sure to encrypt your e-mail with our [encryption key][] if it includes sensitive information.
61+
Learn more about reporting security issues [here][].
6262

63-
Stay up-to-date on the latest security news and patches for Magento by signing up for [Security Alert Notifications](https://magento.com/security/sign-up).
63+
Stay up-to-date on the latest security news and patches for Magento by signing up for [Security Alert Notifications][].
6464

6565
## License
6666

6767
Each Magento source file included in this distribution is licensed under AGPL 3.0.
6868

69-
See the license [here](LICENSE_AGPL3.txt) or contact [[email protected]](mailto:[email protected]) for a copy.
69+
See the license [here][] or contact [[email protected]][] for a copy.
70+
71+
<!-- Link Definitions -->
72+
[Getting Started]: https://devdocs.magento.com/mftf/getting-started.html
73+
[Contribution Guidelines]: .github/CONTRIBUTING.html
74+
[DevDocs Contributing]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md
75+
[[email protected]]: mailto:[email protected]
76+
[encryption key]: https://info2.magento.com/rs/magentoenterprise/images/security_at_magento.asc
77+
[here]: https://magento.com/security/reporting-magento-security-issue
78+
[Security Alert Notifications]: https://magento.com/security/sign-up
79+
[here]: LICENSE_AGPL3.txt
80+
[[email protected]]: mailto:[email protected]

‎composer.lock

+1-1
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/best-practices.md

+166
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# Best practices
2+
3+
Check out our best practices below to ensure you are getting the absolute most out of the Magento Functional Testing Framework.
4+
5+
## Action group
6+
7+
1. [Action group] names should be sufficiently descriptive to inform a test writer of what the action group does and when it should be used.
8+
Add additional explanation in comments if needed.
9+
2. Provide default values for the arguments that apply to your most common case scenarios.
10+
11+
## `actionGroups` vs `extends`
12+
13+
Use an action group to wraps a set of actions to reuse them multiple times.
14+
15+
Use an [extension] when a test or action group needs to be repeated with the exception of a few steps.
16+
17+
### When to use `extends`
18+
19+
Use `extends` in your new test or action group when at least one of the following conditions is applicable to your case:
20+
21+
1. You want to keep the original test without any modifications.
22+
2. You want to create a new test that follows the same path as the original test.
23+
3. You want a new action group that behaves similarly to the existing action group, but you do not want to change the functionality of the original action group.
24+
25+
### When to avoid `extends`
26+
27+
Do not use `extends` in the following conditions:
28+
29+
1. You want to change the functionality of the test or action group and do not need to run the original version.
30+
2. You plan to merge the base test or action group.
31+
32+
The following pattern is used when merging with `extends`:
33+
34+
1. The original test is merged.
35+
2. The extended test is created from the merged original test.
36+
3. The extended test is merged.
37+
38+
## Annotation
39+
40+
1. Use [annotations] in a test.
41+
2. Update your annotations correspondingly when updating tests.
42+
43+
## Data entity
44+
45+
1. Keep your testing instance clean.
46+
Remove data after the test if the test required creating any data.
47+
Use a corresponding [`<deleteData>`] test step in your [`<after>`] block when using a [`<createData>`] action in a [`<before>`] block.
48+
2. Make specific data entries under test to be unique.
49+
Enable data uniqueness where data values are required to be unique in a database by test design.
50+
Use `unique=”suffix”` or `unique=”prefix”` to append or prepend a unique value to the [entity] attribute.
51+
This ensures that tests using the entity can be repeated.
52+
3. Do not modify existing data entity fields or merge additional data fields without complete understanding and verifying the usage of existing data in tests.
53+
Create a new data entity for your test if you are not sure.
54+
55+
## Naming conventions
56+
57+
### File names
58+
59+
Name files according to the following patterns to make searching in future more easy:
60+
61+
#### Test file name
62+
63+
Format: {_Admin_ or _Storefront_}{Functionality}_Test.xml_, where Functionality briefly describes the testing functionality.
64+
65+
Example: _StorefrontCreateCustomerTest.xml_.
66+
67+
#### Section file name
68+
69+
Format: {_Admin_ or _Storefront_}{UI Description}_Section.xml_, where UI Description briefly describes the testing UI.
70+
71+
Example: _AdminNavbarSection.xml_.
72+
73+
#### Data file name
74+
75+
Format: {Type}_Data.xml_, where Type represents the entity type.
76+
77+
Example: _ProductData.xml_.
78+
79+
### Object names
80+
81+
Use the _Foo.camelCase_ naming convention, which is similar to _Classes_ and _classProperties_ in PHP.
82+
83+
#### Upper case
84+
85+
Use an upper case first letter for:
86+
87+
- File names. Example: _StorefrontCreateCustomerTest.xml_
88+
- Test name attributes. Example: `<test name="TestAllTheThingsTest">`.
89+
- Data entity names. Example: `<entity name="OutOfStockProduct">`.
90+
- Page name. Example: `<page name="AdminLoginPage">`.
91+
- Section name. Example: `<section name="AdminCategorySidebarActionSection">`.
92+
- Action group name. Example: `<actionGroup name="LoginToAdminActionGroup">`.
93+
94+
#### Lower case
95+
96+
Use a lower case first letter for:
97+
98+
- Data keys. Example: `<data key="firstName">`.
99+
- Element names. Examples: `<element name="confirmDeleteButton"/>`.
100+
101+
## Page object
102+
103+
Use [parameterized selectors] for constructing a selector when test specific or runtime generated information is needed.
104+
Do not use them for static elements.
105+
106+
<span class="color:red">
107+
BAD:
108+
</span>
109+
110+
<!-- {% raw %} -->
111+
112+
``` xml
113+
<element name="relatedProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='{{productType}}']" parameterized="true"/>
114+
```
115+
116+
<!-- {% endraw %} -->
117+
118+
<span class="color:green">
119+
GOOD:
120+
</span>
121+
122+
Define these three elements and reference them by name in the tests.
123+
124+
``` xml
125+
<element name="relatedProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='related']"/>
126+
<element name="upSellProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='upsell']"/>
127+
<element name="crossSellProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='crosssell']"/>
128+
```
129+
130+
## Test
131+
132+
1. Use actions such as [`<waitForElementVisible>`], [`<waitForLoadingMaskToDisappear>`], and [`<waitForElement>`] to wait the exact time required for the test step.
133+
Try to avoid using the [`<wait>`] action, because it forces the test to wait for the time you specify. You may not need to wait so long to proceed.
134+
1. Keep your tests short and granular for target testing, easier reviews, and easier merge conflict resolution.
135+
It also helps you to identify the cause of test failure.
136+
1. Use comments to keep tests readable and maintainable:
137+
- Keep the inline `<!-- XML comments -->` and [`<comment>`] tags up to date.
138+
It helps to inform the reader of what you are testing and to yield a more descriptive Allure report.
139+
- Explain in comments unclear or tricky test steps.
140+
1. Refer to [sections] instead of writing selectors.
141+
142+
## Test step merging order
143+
144+
When setting a [merging] order for a test step, do not depend on steps from Magento modules that could be disabled by an application.
145+
146+
For example, when you write a test step to create a gift card product, set your test step **after** simple product creation and let the MFTF handle the merge order.
147+
Since the configurable product module could be disabled, this approach is more reliable than setting the test step **before** creating a configurable product.
148+
149+
<!-- Link definitions -->
150+
151+
[`<after>`]: test/actions.html#before-and-after
152+
[`<before>`]: test/actions.html#before-and-after
153+
[`<comment>`]: test/actions.html#comment
154+
[`<createData>`]: test/actions.html#createdata
155+
[`<deleteData>`]: test/actions.html#deletedata
156+
[`<wait>`]: test/actions.html#wait
157+
[`<waitForElement>`]: test/actions.html#waitforelement
158+
[`<waitForElementVisible>`]: test/actions.html#waitforelementvisible
159+
[`<waitForLoadingMaskToDisappear>`]: test/actions.html#waitforloadingmasktodisappear
160+
[Action group]: test/action-groups.html
161+
[annotations]: test/annotations.html
162+
[entity]: data.html
163+
[extension]: extending.html
164+
[merging]: merging.html
165+
[parameterized selectors]: section/parameterized-selectors.html
166+
[sections]: section.html

‎docs/commands/codeception.md

+85
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# CLI commands: vendor/bin/codecept
2+
3+
<div class="bs-callout bs-callout-warning" markdown="1">
4+
We do not recommend using Codeception commands directly as they can break the MFTF basic workflow.
5+
All the Codeception commands you need are wrapped using the [mftf tool][].
6+
7+
To run the Codeception testing framework commands directly, change your directory to the `<Magento root>`.
8+
</div>
9+
10+
## Usage examples
11+
12+
Run all the generated tests:
13+
14+
```bash
15+
../../../vendor/bin/codecept run functional -c dev/tests/acceptance/codeception.yml
16+
```
17+
18+
Run all tests without the `<group value="skip"/>` [annotation][]:
19+
20+
```bash
21+
../../../vendor/bin/codecept run functional --skip-group skip -c dev/tests/acceptance/codeception.yml
22+
```
23+
24+
Run all tests with the `<group value="example"/>` [annotation][] but with no `<group value="skpip"/>`:
25+
26+
```bash
27+
../../../vendor/bin/codecept run functional --group example --skip-group skip -c dev/tests/acceptance/codeception.yml
28+
```
29+
30+
## `codecept run`
31+
32+
`codecept run` runs the test suites:
33+
34+
```bash
35+
../../../vendor/bin/codecept run
36+
```
37+
38+
<div class="bs-callout bs-callout-info">
39+
The following documentation corresponds to Codeception 2.3.8.
40+
</div>
41+
42+
```bash
43+
Full reference:
44+
45+
Arguments:
46+
suite suite to be tested
47+
test test to be run
48+
49+
Options:
50+
-o, --override=OVERRIDE Override config values (multiple values allowed)
51+
--config (-c) Use custom path for config
52+
--report Show output in compact style
53+
--html Generate html with results (default: "report.html")
54+
--xml Generate JUnit XML Log (default: "report.xml")
55+
--tap Generate Tap Log (default: "report.tap.log")
56+
--json Generate Json Log (default: "report.json")
57+
--colors Use colors in output
58+
--no-colors Force no colors in output (useful to override config file)
59+
--silent Only outputs suite names and final results
60+
--steps Show steps in output
61+
--debug (-d) Show debug and scenario output
62+
--coverage Run with code coverage (default: "coverage.serialized")
63+
--coverage-html Generate CodeCoverage HTML report in path (default: "coverage")
64+
--coverage-xml Generate CodeCoverage XML report in file (default: "coverage.xml")
65+
--coverage-text Generate CodeCoverage text report in file (default: "coverage.txt")
66+
--coverage-phpunit Generate CodeCoverage PHPUnit report in file (default: "coverage-phpunit")
67+
--no-exit Do not finish with exit code
68+
--group (-g) Groups of tests to be executed (multiple values allowed)
69+
--skip (-s) Skip selected suites (multiple values allowed)
70+
--skip-group (-x) Skip selected groups (multiple values allowed)
71+
--env Run tests in selected environments. (multiple values allowed, environments can be merged with ',')
72+
--fail-fast (-f) Stop after first failure
73+
--help (-h) Display this help message.
74+
--quiet (-q) Do not output any message.
75+
--verbose (-v|vv|vvv) Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
76+
--version (-V) Display this application version.
77+
--ansi Force ANSI output.
78+
--no-ansi Disable ANSI output.
79+
--no-interaction (-n) Do not ask any interactive question.
80+
```
81+
82+
<!-- Link definitions -->
83+
84+
[mftf tool]: mftf.html
85+
[annotation]: ../test/annotations.html

0 commit comments

Comments
 (0)