PHP Tests tips How not to shoot yourself in the foot? Damian Sromek damiansromek.pl 2012-06
Agenda ● What's about those tests? ● What are the tests about? ● What to do and not to do?
Test types 1. Acceptance (end-to-end) Test as if end user would use the whole system/feature. 2. Integration Test how different parts of system work together - eg. couple of classes or your class using some web service 3. Unit Test single unit (eg. class). Mock all dependencies, eg. web service, file system.
Tests will save you time & troubles 1. Prove you've done your work 2. Help you check much faster if you're work is done 3. Protect you from breaking things - regresion 4. Help you make better code design - easier to maintain 5. Let you apply changes with less worries - refactoring will improve things without breaking anything
How to be happy about tests? 1. If you're new to testing try with writing tests after writing the code 2. Once you're confident about testing try to write tests before the code - TDD 3. Start with acceptance tests, than integration and unit tests when your functionality is implemented 4. Run tests often - after every change and before any commit 5. Use continuous integration server
You will regret you've got a lot of tests if you: 1. make them complex and hard to read 2. duplicate the test code 3. do not use descriptive fail messages 4. do not run them often - eg. you don't have continuous integration server 5. do not make them run fast
Behavioral tests Draft - Behat Feature: Log in and use the application In order to use the application As a user I need to log in using my smart card @javascript Scenario: Log in with inactive smart card Given I am inactive smart card user Given I am on "http://localhost/somethign.cool.php" When I log in Then I should see log in error message
What's good test like? 1. Finds as much bugs as possible 2. Tells you what the software can do and how to use it 3. Is easy to maintain 4. Independent from other tests and repeatable
How to start testing? 1. Write acceptance test for functionality you're implementing - it should cover main story flow 2. Make the test show meaningful fail message 3. Implement functionality so the test is passing 4. Add tests covering more story flows 5. Improve functionality code so all tests are passing again 6. Refactor (also tests) 7. Enjoy!
How to start even better? 1. "Wait" for a bug 2. Write a test confirming/reproducing that bug 3. Fix the code using the test you've got 4. Enjoy!
TDD flow Repeat this process 1. Write test 2. Make test failure message descriptive and easy to understand 3. Make the test pass - implement feature you've write the test for 4. Refactor
Do not - test something irrelevant <?php // SHOULD TEST PLUGIN BUT IS TESTING DATABASE MODEL AND NOTHING MORE class Plugin_Smooth_Streaming_BoundariesTest extends TestCase_Plugin { ... public function testLowerBoundary() { $settings = Xstream_Loader::getModelHelper("SettingHelper"); $lowerLimit = $settings->setSetting('plugin_boundaries', 'lower_limit',1.0); $this->assertEquals(1.0, $settings->getSetting('plugin_boundaries', 'lower_limit')->value); } ... }
Do not - test too much at once <?php // TRY NOT TO MAKE MORE THAN 2-3 ASSERTIONS IN ONE TEST class ResponseSetTest extends PHPUnit_Framework_TestCase { ... public function testAddItems() { $responseSet = new Xs_Monitor_Response_Set('test'); $this->assertEquals(Xs_Monitor_Interface::STATE_WARNING, $responseSet->getStateAsInteger()); $responseSet->addResponse(new Xs_Monitor_Response('service 1', Xs_Monitor_Interface::STATE_OK, 'detail of service 1')); $this->assertEquals(Xs_Monitor_Interface::STATE_OK, $responseSet->getStateAsInteger()); $this->assertNotEquals('detail of service 1', $responseSet->getDetails()); $responseSet->addResponse(new Xs_Monitor_Response('service 2', 3, 'detail of service 2')); $this->assertEquals(3, $responseSet->getStateAsInteger()); $this->assertEquals('UNKNOWN', $responseSet->getState()); $this->assertEquals('detail of service 2', $responseSet->getDetails()); $responseSet ->addResponse(new Xs_Monitor_Response('service 3', Xs_Monitor_Interface::STATE_WARNING, 'detail of service 3')); $this->assertEquals(Xs_Monitor_Interface::STATE_WARNING, $responseSet->getStateAsInteger()); $this->assertEquals('detail of service 3', $responseSet->getDetails()); $responseSet ->addResponse(new Xs_Monitor_Response('service 4', Xs_Monitor_Interface::STATE_CRITICAL, 'detail of service 4')); $this->assertEquals(Xs_Monitor_Interface::STATE_CRITICAL, $responseSet->getStateAsInteger()); $this->assertEquals('detail of service 4', $responseSet->getDetails()); $responseSet ->addResponse(new Xs_Monitor_Response('service 5', Xs_Monitor_Interface::STATE_CRITICAL, 'detail of service 5')); $this->assertEquals(Xs_Monitor_Interface::STATE_CRITICAL, $responseSet->getStateAsInteger()); $this->assertEquals('detail of service 4', $responseSet->getDetails());
Do - use meaningful test name Test name should be descriptive even if it has very long name. Try to avoid test names like "testAdd" etc. Better would be something like "testAddWillPutNewItemIntoContainer" or "testAddWillThrowExceptionIfContainerIsFull" /** * @test */ public function add() { ...
Do - use data providers <?php namespace XsTestUnit; class StringTest extends TestCase { /** * @param string $heystack * @param string $needle * @param boolean $expected * * @dataProvider endsWithDataProvider */ public function testEndsWith($heystack, $needle, $expected) { $this->assertEquals($expected, XsString::endsWith($heystack, $needle)); } public function endsWithDataProvider() { return array( array('asdf', 'f', true), array('asdf', 'df', true), array('asdf', 'asdf', true), array('asdf', 'd', false), array('asdf', 'xf', false), array('asdf', 'asd', false), ); } }
Do - test one unit in unit test Unit test should NOT: 1. Test more than one unit/class 2. Touch database, filesystem, API/WebService etc. - you should mock all "externals" 3. Execute very fast - matter of 10th of second
Do - make tests easy to read and understand class Acceptance_Product_ProductListTest extends Test_TestCase { public function testReturnsErrorAboutNoProductsUserCanBuyWhenHeHasNoProductsRelatedToMedia() { $this->user() ->logIn() ->withoutProducts(array( $this->svodProductId, $this->premiumSvodProductId )) ->withoutOrderedMedia($this->mediaId); $this->shop()->respondsJsonAboutNoProductsAvailable($this->mediaId); }
Do - make tests easy to read and understand (2) Draft class SortingTest extends XsCanalDigitalTestStbAdbTestCase { /** * @test */ public function mediaListWithoutFixedSortingCanBeSortedByPressingRedButton() { $this->user() ->onPage($this->stbApp()->page('sort_fixed')); $initialAppState = $this->stbApp()->getState('maincovers'); $this->user()->pressedButton(Key::BUTTON_RED); $appStateAfterPressingRedButton = $this->stbApp()->getState('maincovers'); // TODO: better way of comparing items sorting $this->assertNotEquals( $initialAppState, $appStateAfterPressingRedButton, 'Pressing red button should change page sorting' ); }
Summary 1. Use TDD 2. Unit tests should test what you've written / proper unit and nothing more 3. Do not test too much in one test 4. Make your tests easy to maintain - proper naming etc. 5. Try to use data provider to test a lot of different cases without code duplication
Thank you @see Test Driven Development @see PHPUnit @see Behavioral Driven Development @see Behat @see Growing Object-Oriented Software, Guided by Tests @see www.jenkins-ci.org

Php tests tips

  • 1.
    PHP Tests tips Hownot to shoot yourself in the foot? Damian Sromek damiansromek.pl 2012-06
  • 2.
    Agenda ● What's aboutthose tests? ● What are the tests about? ● What to do and not to do?
  • 3.
    Test types 1. Acceptance(end-to-end) Test as if end user would use the whole system/feature. 2. Integration Test how different parts of system work together - eg. couple of classes or your class using some web service 3. Unit Test single unit (eg. class). Mock all dependencies, eg. web service, file system.
  • 4.
    Tests will saveyou time & troubles 1. Prove you've done your work 2. Help you check much faster if you're work is done 3. Protect you from breaking things - regresion 4. Help you make better code design - easier to maintain 5. Let you apply changes with less worries - refactoring will improve things without breaking anything
  • 5.
    How to behappy about tests? 1. If you're new to testing try with writing tests after writing the code 2. Once you're confident about testing try to write tests before the code - TDD 3. Start with acceptance tests, than integration and unit tests when your functionality is implemented 4. Run tests often - after every change and before any commit 5. Use continuous integration server
  • 6.
    You will regretyou've got a lot of tests if you: 1. make them complex and hard to read 2. duplicate the test code 3. do not use descriptive fail messages 4. do not run them often - eg. you don't have continuous integration server 5. do not make them run fast
  • 7.
    Behavioral tests Draft -Behat Feature: Log in and use the application In order to use the application As a user I need to log in using my smart card @javascript Scenario: Log in with inactive smart card Given I am inactive smart card user Given I am on "http://localhost/somethign.cool.php" When I log in Then I should see log in error message
  • 8.
    What's good testlike? 1. Finds as much bugs as possible 2. Tells you what the software can do and how to use it 3. Is easy to maintain 4. Independent from other tests and repeatable
  • 9.
    How to starttesting? 1. Write acceptance test for functionality you're implementing - it should cover main story flow 2. Make the test show meaningful fail message 3. Implement functionality so the test is passing 4. Add tests covering more story flows 5. Improve functionality code so all tests are passing again 6. Refactor (also tests) 7. Enjoy!
  • 10.
    How to starteven better? 1. "Wait" for a bug 2. Write a test confirming/reproducing that bug 3. Fix the code using the test you've got 4. Enjoy!
  • 11.
    TDD flow Repeat thisprocess 1. Write test 2. Make test failure message descriptive and easy to understand 3. Make the test pass - implement feature you've write the test for 4. Refactor
  • 12.
    Do not -test something irrelevant <?php // SHOULD TEST PLUGIN BUT IS TESTING DATABASE MODEL AND NOTHING MORE class Plugin_Smooth_Streaming_BoundariesTest extends TestCase_Plugin { ... public function testLowerBoundary() { $settings = Xstream_Loader::getModelHelper("SettingHelper"); $lowerLimit = $settings->setSetting('plugin_boundaries', 'lower_limit',1.0); $this->assertEquals(1.0, $settings->getSetting('plugin_boundaries', 'lower_limit')->value); } ... }
  • 13.
    Do not -test too much at once <?php // TRY NOT TO MAKE MORE THAN 2-3 ASSERTIONS IN ONE TEST class ResponseSetTest extends PHPUnit_Framework_TestCase { ... public function testAddItems() { $responseSet = new Xs_Monitor_Response_Set('test'); $this->assertEquals(Xs_Monitor_Interface::STATE_WARNING, $responseSet->getStateAsInteger()); $responseSet->addResponse(new Xs_Monitor_Response('service 1', Xs_Monitor_Interface::STATE_OK, 'detail of service 1')); $this->assertEquals(Xs_Monitor_Interface::STATE_OK, $responseSet->getStateAsInteger()); $this->assertNotEquals('detail of service 1', $responseSet->getDetails()); $responseSet->addResponse(new Xs_Monitor_Response('service 2', 3, 'detail of service 2')); $this->assertEquals(3, $responseSet->getStateAsInteger()); $this->assertEquals('UNKNOWN', $responseSet->getState()); $this->assertEquals('detail of service 2', $responseSet->getDetails()); $responseSet ->addResponse(new Xs_Monitor_Response('service 3', Xs_Monitor_Interface::STATE_WARNING, 'detail of service 3')); $this->assertEquals(Xs_Monitor_Interface::STATE_WARNING, $responseSet->getStateAsInteger()); $this->assertEquals('detail of service 3', $responseSet->getDetails()); $responseSet ->addResponse(new Xs_Monitor_Response('service 4', Xs_Monitor_Interface::STATE_CRITICAL, 'detail of service 4')); $this->assertEquals(Xs_Monitor_Interface::STATE_CRITICAL, $responseSet->getStateAsInteger()); $this->assertEquals('detail of service 4', $responseSet->getDetails()); $responseSet ->addResponse(new Xs_Monitor_Response('service 5', Xs_Monitor_Interface::STATE_CRITICAL, 'detail of service 5')); $this->assertEquals(Xs_Monitor_Interface::STATE_CRITICAL, $responseSet->getStateAsInteger()); $this->assertEquals('detail of service 4', $responseSet->getDetails());
  • 14.
    Do - usemeaningful test name Test name should be descriptive even if it has very long name. Try to avoid test names like "testAdd" etc. Better would be something like "testAddWillPutNewItemIntoContainer" or "testAddWillThrowExceptionIfContainerIsFull" /** * @test */ public function add() { ...
  • 15.
    Do - usedata providers <?php namespace XsTestUnit; class StringTest extends TestCase { /** * @param string $heystack * @param string $needle * @param boolean $expected * * @dataProvider endsWithDataProvider */ public function testEndsWith($heystack, $needle, $expected) { $this->assertEquals($expected, XsString::endsWith($heystack, $needle)); } public function endsWithDataProvider() { return array( array('asdf', 'f', true), array('asdf', 'df', true), array('asdf', 'asdf', true), array('asdf', 'd', false), array('asdf', 'xf', false), array('asdf', 'asd', false), ); } }
  • 16.
    Do - testone unit in unit test Unit test should NOT: 1. Test more than one unit/class 2. Touch database, filesystem, API/WebService etc. - you should mock all "externals" 3. Execute very fast - matter of 10th of second
  • 17.
    Do - maketests easy to read and understand class Acceptance_Product_ProductListTest extends Test_TestCase { public function testReturnsErrorAboutNoProductsUserCanBuyWhenHeHasNoProductsRelatedToMedia() { $this->user() ->logIn() ->withoutProducts(array( $this->svodProductId, $this->premiumSvodProductId )) ->withoutOrderedMedia($this->mediaId); $this->shop()->respondsJsonAboutNoProductsAvailable($this->mediaId); }
  • 18.
    Do - maketests easy to read and understand (2) Draft class SortingTest extends XsCanalDigitalTestStbAdbTestCase { /** * @test */ public function mediaListWithoutFixedSortingCanBeSortedByPressingRedButton() { $this->user() ->onPage($this->stbApp()->page('sort_fixed')); $initialAppState = $this->stbApp()->getState('maincovers'); $this->user()->pressedButton(Key::BUTTON_RED); $appStateAfterPressingRedButton = $this->stbApp()->getState('maincovers'); // TODO: better way of comparing items sorting $this->assertNotEquals( $initialAppState, $appStateAfterPressingRedButton, 'Pressing red button should change page sorting' ); }
  • 19.
    Summary 1. Use TDD 2.Unit tests should test what you've written / proper unit and nothing more 3. Do not test too much in one test 4. Make your tests easy to maintain - proper naming etc. 5. Try to use data provider to test a lot of different cases without code duplication
  • 20.
    Thank you @see TestDriven Development @see PHPUnit @see Behavioral Driven Development @see Behat @see Growing Object-Oriented Software, Guided by Tests @see www.jenkins-ci.org