magentoCLI actions not working if docroot points to /pub

[powli]

Preconditions

1. Any compatible Magento version
2. Any compatible MFTF version
3. Have your web server (Apache or Nginx) point to the <MAGE_ROOT_DIR>/pub as recommended in the DevDocs

Steps to reproduce

Run any test or suite with a magentoCLI step in test <before> or <after>

Expected result

CLI command is executed properly

Actual result

The command is not executed via shell exec but via POST call to the Magento instance

The problem is that the file to be executed, /dev/tests/acceptance/utils/command.php, is outside the /pub directory and as such usually not accessible.

To overcome this, there is additional configuration necessary on web server level (example ngnix configuration):

location /dev/tests/acceptance/ {
    root $MAGE_ROOT;

    location ~ ^/dev/tests/acceptance/utils/[^.]+\.php {
        fastcgi_pass   $FASTCGI_BACKEND;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        include        fastcgi_params;

        allow <local_ip>;
        deny all;
    }
}

This should probably either be documented in the setup guides for MFTF or rectified to use the shell command instead.

[powli]

@KevinBKozan I see you implemented the magentoCLI support initially - do you remember why it was deemed necessary to execute the commands via separate CURL call instead of a process calling on the bin/magento command?
Since you need file access anyway to the tested Magento instance for reading the test configurations I see no real benefit over shell execution.

[KevinBKozan]

@powli Good question, the reason is because MFTF needs to support running against a local Magento instance as well as a remote Magento instance (i.e. cloud environments). That essentially outlined the need to create an endpoint for MFTF to be able to interact with bin/magento.

It's obviously tricky though, as I'm sure you've found, given the nearly endless ways Magento can be set up. We had #333 for nginx configuration docs, and I think that would satisfy documented in the setup guides for MFTF.

[powli]

Ok, that's right, but don't you need file access anyway for the ModuleResolver or am I misreading something?
Maybe a configuration setting in the functional.suite.yml for preference would be nice for that. I can update my PR accordingly if necessary.

[KevinBKozan]

@powli It's been a while since I've looked at it, but from what I remember ModuleResolver prefers the local Magento instances registered modules, but is capable of falling back onto pattern matching (this is as I remember it, I need to double check this claim).

As far as your PR, I think it would be interesting to allow both CURL and Shell execution of bin/magento. I'll comment on the PR with suggestions.

[parveen12345]

Hello
How we test a hello world using functional testing please guide me we install the MTFT successful and selenium server is install and we Crete a hello world module but when we run the command then it show errors. Please tell me how we test a hello world module using functional testing

[soumyau]

#343 is merged and is in develop, waiting to be released with the next MFTF version. @powli thank you for your contribution!

[gwharton]

I still value the remote execution option, I just wish it was implemented differently.

I run MFTF against my dev environment which is local, but when I deploy, I then adjust the ".env" file to point to my live site, and I run a number of tests against the live site, but initiated from the staging environment codebase. Its a last minute part of the deploy process for me to validate the deploy worked correctly.

I might just take a copy of the command.php script, edit it to suite the correct paths, and place it in pub, obviously with the appropriate security safeguards.