Documentation gaps

[snez]

Preconditions

1. PHP 7.4.14
2. Magento 2.4-develop
3. selenium-server-standalone: stable 3.141.59
4. MacOS Catalina

Steps to reproduce

1. Follow the MFTF setup instructions all the way up to the section https://devdocs.magento.com/mftf/docs/getting-started.html#run-tests
2. Run the tests with vendor/bin/codecept run functional -c dev/tests/acceptance/codeception.yml

Expected result

1. Successfully run the tests
2. The documentation should explain what this region refers to

Actual result

Codeception PHP Testing Framework v4.1.17
Powered by PHPUnit 9.1.5 by Sebastian Bergmann and contributors.

In ClientResolver.php line 383:

  Invalid configuration value provided for "region". Expected string, but got bool(false)

  region: (string)

    Region to connect to. See
    http://docs.aws.amazon.com/general/latest/gr/rande.html for a list of
    available regions.


run [-o|--override OVERRIDE] [-e|--ext EXT] [--report] [--html [HTML]] [--xml [XML]] [--phpunit-xml [PHPUNIT-XML]] [--tap [TAP]] [--json [JSON]] [--colors] [--no-colors] [--silent] [--steps] [-d|--debug] [--bootstrap [BOOTSTRAP]] [--no-redirect] [--coverage [COVERAGE]] [--coverage-html [COVERAGE-HTML]] [--coverage-xml [COVERAGE-XML]] [--coverage-text [COVERAGE-TEXT]] [--coverage-crap4j [COVERAGE-CRAP4J]] [--coverage-cobertura [COVERAGE-COBERTURA]] [--coverage-phpunit [COVERAGE-PHPUNIT]] [--no-exit] [-g|--group GROUP] [-s|--skip SKIP] [-x|--skip-group SKIP-GROUP] [--env ENV] [-f|--fail-fast] [--no-rebuild] [--seed SEED] [--no-artifacts] [--] [<suite> [<test>]]

Notes

In the file under dev/tests/acceptance/.env, I have the following:

MAGENTO_BASE_URL=http://mftf.loc/
MAGENTO_BACKEND_NAME=admin
MAGENTO_ADMIN_USERNAME=xxxx
MAGENTO_ADMIN_PASSWORD=xxxxxxxxxx
SELENIUM_CLOSE_ALL_SESSIONS=true
BROWSER=chrome
MODULE_ALLOWLIST=Magento_Framework,ConfigurableProductWishlist,ConfigurableProductCatalogSearch
BROWSER_LOG_BLOCKLIST=other
ELASTICSEARCH_VERSION=7

In the file under dev/tests/acceptance/.credentials, I have the same entries as in .credentials.example

[m2-assistant]

Hi @snez. Thank you for your report.
To help us process this issue please make sure that you provided sufficient information.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[jilu1]

@snez
The region is probably referring to CREDENTIAL_AWS_SECRETS_MANAGER_REGION in MFTF .env.example.
You will need to set it only if:

• Tests use sensitive data (referencing _CREDS)?
• You use AWS for secrets storage (you don't have .credentials file in your local)

Is it the case for you?

[snez]

@jilu1 this issue happened at the initial MFTF setup phase before any tests were implemented, so no no AWS or _CREDS were used anywhere. It seems like a step is missing from the documentation setup steps regarding credentials.

[sebastian-ehrling]

I have the same problem @snez ! Could you solve this problem?

[snez]

@sebastian-ehrling its been a while and can't remember but try creating an empty dev/tests/acceptance/.credentials file

[sebastian-ehrling]

create .credentials files an run the test with parameter -r solves the problem

[jaybeckr]

I'm pretty sure I had a .credentials file before attempting to run a group, but using the "-r" (or "--remove") flag helped get around this issue.

[peteracs]

This error is caused by having AWS S3 module enabled without the correct credentials.

It is missing these values from your dev/tests/acceptance/.env

REMOTE_STORAGE_AWSS3_REGION=us-west-2
REMOTE_STORAGE_AWSS3_BUCKET=my-test-bucket
REMOTE_STORAGE_AWSS3_PREFIX=local

If you don't want to use AWS S3, just disable it.
bin/magento module:disable Magento_AwsS3

Then regenerate your tests
vendor/bin/mftf generate:tests -r

Finally tests works
vendor/bin/codecept run functional -c dev/tests/acceptance/codeception.yml

[kolaente]

I have a .credentials file in my project but am still unable to run the tests. I do not want to use s3 nor do I have any tests using it. Trying to run my test suite fails:

$ vendor/bin/mftf run:group Company_Checkout -r -f
ERROR:  1 Suite(s) has(have) tests with annotation errors or some included tests missing. See mftf.log for details.
ERROR: 9 Test(s) failed to generate. See mftf.log for details.

Generate Tests Command Run (with errors)

Codeception PHP Testing Framework v4.1.28
Powered by PHPUnit 9.1.5 by Sebastian Bergmann and contributors.
[Groups] Company_Checkout

In ClientResolver.php line 412:

  [InvalidArgumentException]
  Invalid configuration value provided for "region". Expected string, but got
   bool(false)

  region: (string)

    Region to connect to. See
    http://docs.aws.amazon.com/general/latest/gr/rande.html for a list of
    available regions.

I've been trying to get this to work since a few days now. The dev experience so far is even worse than with normal magento development and I haven't really written any tests yet.

[Nurgiel]

Can't just disable bin/magento module:disable Magento_AwsS3 anymore, it's now a dependant module of Magento_Catalog.

Unable to change status of modules because of the following constraints:
Cannot disable Magento_AwsS3 because modules depend on it:
(...)
Magento_Catalog: Magento_Catalog->Magento_AwsS3

You can remove that module if you add to composer.json:

"replace": {
  "magento/module-aws-s3": "*"
},

and then run: composer remove magento/module-aws-s3