Workshop: Functional testing made easy with PHPUnit & Selenium (phpCE Poland, November 2017)

Workshop: Functional testing made easy with PHPUnit & Selenium 3 November 2017 Poland Ondřej Machulda ondrej.machulda@gmail.com @OndraM Annotated slides

$ whoami Ondřej Machulda @OndraM Symfony developer & QA automation engineer + Selenium, TDD & QA trainer php-webdriver library co-maintainer

What are you going to learn today? What is possible to do with Selenium How to convert manual routine to automated functional tests Write functional test in a few minutes Why are functional tests irreplaceable Execute tests on your machine in a few seconds Write functional tests in a maintainable manner Easily find root cause of failed test

Cost of fixing bugs Source: Barry Boehm, Equity Keynote Address 2007 Time is money. So we want to find bugs as soon as possible after they are created. Because fixing them earlier in the development stage would be for us much faster and thus cheaper. And automated tests are a way how to accomplish this.

What do we want from tests? FIRST! Fast Isolated Repeatable Self-validating Timely Automated tests should follow the "FIRST" criteria! Fast – tests must be executed fast, we need to get fast feedback if there is a bug. Isolated – tests should not depend on others, on order of execution, on global state etc. This will disallow parallelism. Repeatable – we want to have same stable result each time they are run on the same application. Self-validating – test itself should reliably know whether it passes or not. No human interaction should be required to ensure test result. Timely – tests should be written with the code or maybe before (when applying Test Driven Development)

Source: https://commons.wikimedia.org/wiki/File:Bicycle_diagram-unif.svg, author Fiestoforo, licence CC BY 3.0 Web app is a machine – like a bicycle. Lots of different parts in different layers, which needs to fit together to accomplish the one ultimate purpose of the machine – so you can ride the bike. We usually do test the machine starting from the smallest pieces, because it is easy to test them - you can easily define their behavior and you can usually easily and fast validate it! Like inputs/outputs of method and its behavior in edge cases. However, this is not always enough... If you want to make sure the assembled machine works and everything fits together (eg. you can really drive & steer the bike), you will test the main business critical scenarios from customer point of view. Thats why these kind of tests is irreplaceable – its your output quality control. Next one in the QA chain is usually only the customer, and thats too late :-).

Software Testing Ice-cream Cone Business Source: https://watirmelon.blog/2012/01/31/introducing-the-software-testing-ice-cream-cone/, author Alister Scott However, you should not make manual tests or functional tests the testing layer in which you invest the most - like we see in this ice-cream cone antipattern.

Test Pyramid Timecost FIRST 5 % 15 % 80 % Test pyramid is a visual guide showing how much time you should invest in which layer of tests. Why this ratio? The higher layer, the harder is to write and maintain the tests (they don't fulfil the FIRST criteria) and the slower feedback you have from them. Unit-tests are fast to write and run, stable and helps with code design – so as a developer you want to primary write them. But remember the bicycle – you could miss a lot without functional tests.

It is useless to run the functional tests only from developers laptop. They are necessary part of continuous integration and should not be missing in your automated continuous deployment pipeline. As you know – the faster you find your bugs, the faster and cheaper is to fix them!

Source: https://twitter.com/emilbronikowski/status/808669983639216128

Source: https://twitter.com/thepracticaldev/status/733908215021199360

Selenium ● Just a browser automation library ● Selenium aka Selenium WebDriver ● Platform and language independent ● Supported by all major browser ● Easy setup ● Not depending on any IDE ● Widely used, actively developed ● Many resources on the Internet The tool for automated functional tests is Selenium - an open-source library for browser automation. You tell it what actions in browser should be done and it executes them. The WebDriver protocol will also soon be W3C standard and it is implemented in almost all browsers.

Selenium can... ● Navigate browser to URL (obviously) ● Find element on the page - via CSS / XPath / ... ● Interact with the element ○ Get it contents, location, styles, attributes... ○ Click on in ○ ... ● Take a screenshot ● Fill and submit form ● Change browser windows size ● Navigate through history (back / forward) ● ...

Steward - what it does for you Test runner Parallel execution, logs, handle test dependencies... Convenience layer Syntax sugar, PHPUnit + php-webdriver integration, browser setup, screenshots... https://github.com/lmc-eu/steward We will also use Steward - an open-source tool built on top of PHPUnit and Symfony components. It is a test-runner (controlled from CLI) and also extension for PHPUnit, integrating the php-webdriver library right in your PHPUnit tests.

https://git.io/vFsUR Here is wiki page with links, commands etc. for following examples:

Setting up local development environment ● Repository with examples for the workshop: https://github.com/OndraM/selenium-workshop-phpce cd [your projects directory] git clone git@github.com:OndraM/selenium-workshop-phpce cd selenium-workshop-phpce git checkout step-0 composer install

Start Selenium Server inside Docker docker run -p 4444:4444 -p 5900:5900 selenium/standalone-chrome-debug:3.6.0 localhost:4444 localhost:5900 vncviewer 127.0.0.1:5900 # on Linux open vnc://127.0.0.1:5900 # on Mac

Executing tests using Steward ● Open new terminal window ○ cd selenium-workshop-phpce ● Show Steward help ○ vendor/bin/steward run --help ● run command ○ two required arguments: environment and browser ○ -vvv to set max output verbosity vendor/bin/steward run staging chrome vendor/bin/steward run staging chrome -vvv

Implementing first basic test https://phpce-gz65nia-mxk4rjvb4la6e.eu.platform.sh/ Test scenario: "Product detail loads basic product information" 1. Open product detail page 2. Check product header is as expected 3. Check product price is as expected Solution ⇒ git checkout -f step-1 # force checkout!

Source: https://martinfowler.com/bliki/PageObject.html Page Object is a design pattern from Martin Fowler, which suggest interacting with the webpage UI through an abstraction – ie. an object with methods mapping the UI structure and UI interactions. Because in the tests scenario you want to interact with the UI, not with its HTML implementation. Page objects are also a way how to make your functional tests maintainable in a long-term.

Using Page Object in a test Solution ⇒ git checkout -f step-1-fixed

● Allows to do what user could do & see ● Copies hierarchy of user interface ● Separates test scenario from its HTML implementation ● Its method should return: ○ Primitive data (string, integer, array of strings...) ○ Or another Page Object ● Assertions should be in tests, not inside page object Page Object - main principles

Extending Page Objects Test scenario: "Product could be added to a cart from product detail page" 1. Open product detail page 2. Add product to cart 3. Cart listing is opened 4. Cart contains product added in step 2.

Finding elements Steward simplified syntax (syntax sugar) $element = $this->findByCss('.foo'); $element = $this->findByXpath('//table/tr/td[2]//a'); $elements = $this->findMultipleByCss('.foo'); Other element finding strategies findByClass, findById, findByName, findByLinkText, findByPartialLinkText, findByTag Traditional php-webdriver way $element = $this->wd->findElement(WebDriverBy::cssSelector('.foo')); $elements = $this->wd->findElements(WebDriverBy::cssSelector('.foo'));

Element selectors - ID, CSS, Xpath... ● ID - best option, if available ○ $this->findByCss('#login-button'); ○ $this->findById('login-button'); ● CSS selectors - simple to write ○ $this->findByCss('div.header > button.login'); ● XPath - if there is no other way ○ $this->findByXpath( '//table//a[contains(@href,"sticker-repellendus")] /ancestor::tr/td/span[@class = "sylius-quantity"]/input' ); Beware of selectors stability - too specific vs. too generic

Extending Page Objects Solution ⇒ git checkout -f step-2

Logs, screenshots Test results overview: ● logs/results.xml ● vendor/bin/steward results [-vvv] Screenshots, HTML snapshots: ● directory logs/ ○ TestCaseName-TestName.png ○ TestCaseName-TestName.html

Extending Page Objects II. git checkout -f step-3 Test scenario: "Multiple products could be added to a cart" 1. Open product A detail page 2. Add product A to cart 3. Cart listing is opened 4. Open product B detail page (hint: just add slug) 5. Add product B to cart 6. Cart listing is opened (hint: addToCart() method returns CartPage) 7. Cart contain two products - A and B (hint: finish getNamesOfProductsInCart() method of CartPage)

Run only one testcase class vendor/bin/steward run staging chrome -vvv --pattern ProductDetailTest.php Run only one test method of one testcase class: vendor/bin/steward run staging chrome -vvv --pattern ProductDetailTest.php --filter shouldAddMultipleProductsToCart

Extending Page Objects II. git checkout -f step-3 Test scenario: "Multiple products could be added to a cart" 1. Open product A detail page 2. Add product A to cart 3. Cart listing is opened 4. Open product B detail page (hint: just add slug) 5. Add product B to cart 6. Cart listing is opened (hint: addToCart() method returns CartPage) 7. Cart contain two products - A and B (hint: finish getNamesOfProductsInCart() method of CartPage) Solution ⇒ git checkout -f step-3-fixed

Forms $element = $this->findByName('name'); $element->sendKeys('Some text') $element->clear() $element->submit() <select> elements $element = $this->findByName('country'); $select = new WebDriverSelect($element); $select->selectByVisibleText('Poland'); $select->selectByValue('pl'); $select->selectByIndex(13);

Forms example Test scenario: "User submits order with products in cart" 1. Prerequisite: have a product in cart 2. Open cart listing 3. Go to Checkout 4. Address form is shown 5. Fill required fields ○ e-mail, first name, last name, street, country (select), city, postcode 6. Submit form 7. Shipping details form is shown 8. ... git checkout -f step-4 Solution ⇒ git checkout -f step-4-fixed

Waiting ("explicit wait") ● Web is not a synchronous place to be... ● Selenium doesn't know when your actions are finished (Except loading page via get()) ○ Loading page after user clicks to a link ○ Submitting a form ○ Displaying modal window ○ AJAX ○ ...

Waiting - syntax sugar waitFor...() methods $this->waitForCss('.#modal p'); // or $this->wd->wait()->until( WebDriverExpectedCondition::presenceOfElementLocated( WebDriverBy::cssSelector('#modal p') ) ); $this->waitForTitle('Search results'); // or $this->wd->wait()->until( WebDriverExpectedCondition::titleIs('Search results') );

Advanced WebDriverExpectedConditions $this->wd->wait()->until( WebDriverExpectedCondition::titleIs('text') ); ● titleContains('text'); titleMatches('/.../') ● presenceOfElementLocated($by) ● visibilityOfElementLocated($by) ● elementTextContains($by, 'text'); elementTextIs($by, 'text') ● alertIsPresent() ● urlContains('?login=success'); urlMatches('/.../'); ● numberOfWindowsToBe(2) ● not() ● custom callback https://github.com/facebook/php-webdriver/wiki/HowTo-Wait

Explicit wait example Test scenario: "Registered but unlogged user could login during checkout" 1. Prerequisite: have a product in cart 2. Open cart listing 3. Go to Checkout 4. Address form is shown 5. Fill e-mail and password of existing user ○ E-mail: user@example.com / Password: sylius 6. E-mail input will no longer be shown 7. Name of logged user will be shown in the header 8. ... Solution ⇒ git checkout -f step-5-fixed git checkout -f step-5

Within one test-case class /** * @test * @depends shouldRegisterUser * @param string $username */ public function shouldLoginUser( $username) { ... } Test dependencies Amongst more than one test-case ● annotation @delayMinutes and @delayAfter on testcase class /** * @delayMinutes 2 * @delayAfter MyContactFormTest */ ● Steward allows to pass data between testcases https://github.com/lmc-eu/steward/wiki/Test-dependencies

Test dependencies are evil ☠

File upload $input = $this->findByName('upload'); $input->setFileDetector(new LocalFileDetector()) ->sendKeys('/path/to/image.jpg'); https://simple-u6rzw4q-mxk4rjvb4la6e.eu.platform.sh/upload-simple.html git checkout -f step-6 Solution ⇒ git checkout -f step-6-fixed

JavaScript execution Execute synchronous script (doesn't block test execution) $this->wd->executeScript(' document.body.style.backgroundColor = "red"; '); Execute asynchronous script (wait for callback) $this->driver->executeAsyncScript(' var callback = arguments[arguments.length - 1]; ... callback(); '); https://simple-u6rzw4q-mxk4rjvb4la6e.eu.platform.sh/hidden.html git checkout -f step-7 Solution ⇒ git checkout -f step-7-fixed

Selenium can do much more ● Change window size ($this->wd->manage()) ● Navigate through browsing history (navigate()) ● Switch to different window / iframe / alert (switchTo()) ○ github.com/facebook/php-webdriver/wiki/Alert,-tabs,-frames,-iframes ● Move mouse over element (action()) ● Run your tests in different browsers ● Run Chrome / Firefox in headless mode (R.I.P. PhantomJS)

Other Steward features ● Annotation @noBrowser ○ Eg. seeding test data via API/database ○ Example: SeedDataTest.php ● Set default window size ● Sauce Labs / BrowserStack / TestingBot integration ○ Example: https://saucelabs.com/u/OndraM ● Pass capabilities (= configuration) to the browser ● Generate test execution timeline ● And more: https://github.com/lmc-eu/steward-example

Debugging tests via Xdebug & PHPStorm $ vendor/bin/steward run dev chrome --xdebug https://github.com/lmc-eu/steward/wiki/Debugging-Selenium-tests-With-Steward

Best practices ● Less functional tests is more ● Functional tests are not second-class citizen ● Have a strategy to maintain them ● Do not have long-term broken tests ○ markTestSkipped() ● Use Page Objects ● Unstable (flaky) tests - find root cause, no workarounds ● Tests must be fast ● Implicit wait instead of sleep ● Use self-describing names for tests, methods etc.

Other options how to use Selenium in PHP There are multiple options how to run Selenium tests in PHP - choose the one that fits your needs! Steward - integrates php-webdriver into classic PHPUnit-styled tests and also provides parallelization. Codeception - complex test framework for all layers of tests, which adds BDD-like layer on top of PHPUnit. Laravel Dusk - another semantics for writing selenium tests in a "Laravel" way, it also includes Laravel integration. Behat + Mink - BDD way of writing test scenarios, however uses unmaintained library for Selenium integration Phpunit-selenium - old and outdated extension for PHPUnit ● Steward ● Codeception ● Laravel Dusk ● Behat + Mink ● phpunit-selenium (PHPUnit_Extensions_Selenium2TestCase)

Ondřej Machulda ondrej.machulda@gmail.com www.ondrejmachulda.cz @OndraM A that's all for today! ✌

Workshop: Functional testing made easy with PHPUnit & Selenium (phpCE Poland, November 2017)

More Related Content

What's hot

Similar to Workshop: Functional testing made easy with PHPUnit & Selenium (phpCE Poland, November 2017)

More from Ondřej Machulda

Recently uploaded

Workshop: Functional testing made easy with PHPUnit & Selenium (phpCE Poland, November 2017)