Magento Functional Testing Framework: Quick Start

Magento 2.0 is considered to be a game-changer. For me, personally, Magento finally did the homework when it comes to Quality Assurance. I really enjoy Unit Tests and all the tools that came along with it.

In November 2017, I’ve joined Jisse Reitsma at his great conference MageTestFest. It was exactly 2 years after the first release of Magento 2 (November 17th, 2015). That was the first time I saw Magento Functional Testing Framework. I was so enthusiastic about it, that I’ve built environment and run tests. That’s how my MFTF story begins.

It does not really matter what is your development setup. Whether you use Docker, Valet Plus or Warden  - you can run MFTF in minutes.

Recording of “Running Magento Functional Testing Framework with Warden” stream

Docker-based environment

Setting up Selenium Container

Selenium is available as a Docker image. Add it with Chrome Driver along with the other roles, in docker-compose.selenium.yml file:

version: "3"
services:
  browser_chrome:
      image: selenium/standalone-chrome-debug:3.8.1
      ports:
        - "5900:5900"

Selenium docker-compose.yml

  1. Selenium versions prior to 3.9.0 are not compatible with Facebook Webdriver, due to removal of Passthrough mode for the server. We use 3.8.1 instead.
  2. We use -debug version of selenium/standalone-chrome as we want to preview the execution of the results.
  3. We open 5900 port, as a -debug a version of Selenium allows you to connect to browser_chrome with VNC using this port.

The connection from Selenium to Nginx with your domain

By default, Docker containers are available to each other with their name. Having that in mind you may consider using container name as a Store domain (app) instead of magento2.test during Magento setup.

or…

Set up an internal network between Nginx and Selenium containers. To achieve that — create a network, by adding at the end of docker-compose.selenium.yml :

networks:
  testing:

You also need to assign Selenium to created network and determine the alias that will be used for connections to app container inside testing network:

version: "3"
services:
  browser_chrome:
      image: selenium/standalone-chrome-debug:3.8.1
      ports:
        - "5900:5900"
      networks:
        - testing
  app:
      networks:
        testing:
          aliases:
            - "magento2.test"

Optionally you can mount dev/tests/acceptance directory, by adding a line to docker-compose.dev.yml:

version: "3"

services:
  app:
    volumes: &appvolumes
      - ./src/dev/tests/acceptance:/var/www/html/dev/tests/acceptance:delegated
      ...

Keep in mind, that you need to run docker-compose for all the files together:

docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.selenium.yml up -d

Running Testing Stack

Running your docker-compose.mftf.yml, you’ll see that Selenium image is being downloaded and built. When it’s done, connect 127.0.0.1:5900 with your favourite VNC Client (for Ubuntu, I use Remmina).

Connecting with Selenium Standalone debug instance with Remmina

You’ll be prompted for the password, which is secret, then your environment is ready to play:

You are ready to go.

Installing Magento Functional Testing Framework

I recommend you to use the latest Magento version, which is 2.3.4 for the time of writing this tutorial. It has MFTF packaged so that everything you need to do is to runvendor/bin/mftf build:project that prepares:

  • /dev/tests/acceptance/.env — The main and most important configuration file for Testing Framework
  • /dev/tests/acceptance/.credentials.example — Contains all the credentials used for tests (API keys, other secrets)
  • /dev/tests/acceptance/utils/command.php — Useful “backdoor” to Magento, that enables you to perform CLI commands from tests.
  • /dev/tests/acceptance/tests/functional.suite.yml — Defines Test Suite details, like API URL or browser resolution.

For the first run, we need to modify .env only:

MAGENTO_BASE_URL=https://magento2.test/
MAGENTO_BACKEND_NAME=admin
MAGENTO_ADMIN_USERNAME=magician
MAGENTO_ADMIN_PASSWORD=Test123!
BROWSER=chrome
MODULE_WHITELIST=Magento_Framework,Magento_ConfigurableProductWishlist,Magento_ConfigurableProductCatalogSearch
SELENIUM_HOST=browser_chrome

SELENIUM_HOST setting was not included in generated .env file, you have to add it on your own. By SELENIUM_HOST we tell Codeception which server it has to connect to reach Selenium.

If you don’t have an Admin account yet, you should create one by yourself running bin/magento admin:user:create. Remember, that if you don’t remember BACKEND_NAME (Admin Panel URL), you should run bin/magento info:adminuri.

Once you saved the file — we’re ready to go! Let’s run one of the simplest tests: vendor/bin/mftf run:test AdminLoginTest

As a result, you should receive CLI output:

Am I ready to run all MFTF tests?

No, unfortunately not. Your setup allows you only to perform tests that do not need CLI commands to run. All that because of the security configuration of Nginx. We need to permit connections to /dev/tests/acceptance/utils/command.php by adding to your Nginx’s .conf file:

location ~* ^/dev/tests/acceptance/utils($|/) {
  root $MAGE_ROOT;
  location ~ ^/dev/tests/acceptance/utils/command.php {
      fastcgi_pass   fastcgi_backend;
      fastcgi_index  index.php;
      fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
      include        fastcgi_params;
  }
}

Now you are ready!

Write a Comment

Your email address will not be published. Required fields are marked *