Acceptance & Functional Testing with Codeception Joe Ferguson February 6th 2015
Who Am I? Joe Ferguson PHP Developer Twitter: @JoePFerguson Organizer of @MemphisPHP @NomadPHP Lightning Talks Passionate about Community
One late night coding session of fighting with tests…
I found
hey, I’m everyone…
Example Test Sample taken from http://codeception.com
You gotta be… …me!
Codeception • Selenium WebDriver integration • Elements matched by name, CSS, XPath • Symfony2, Laravel, Yii, Phalcon, Zend Framework • PageObjects and StepObjects included • BDD-style readable tests • Powered by PHPUnit • API testing: REST,SOAP,XML-RPC • Facebook API testing • Data Cleanup • HTML, XML, TAP, JSON reports • CodeCoverage and Remote CodeCoverage • Parallel Execution
Types of Testing Unit Functional Acceptance
Unit Testing In computer programming, unit testing is a software testing method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to determine whether they are fit for use. https://en.wikipedia.org/wiki/Unit_testing
Functional Testing Functional testing is a quality assurance (QA) process and a type of black box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered (not like in white-box testing). Functional testing usually describes what the system does. Functional testing does not imply that you are testing a function (method) of your module or class. Functional testing tests a slice of functionality of the whole system https://en.wikipedia.org/wiki/Functional_testing
Acceptance Testing In engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met. In software testing the ISTQB defines acceptance as: formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system. Acceptance testing is also known as user acceptance testing (UAT), end-user testing, operational acceptance testing (OAT) or field (acceptance) testing. https://en.wikipedia.org/wiki/Acceptance_testing
How I think about types Unit tests test your functions and methods. Functional tests test parts of your application. Acceptance tests test your interactions.
Isn’t unit testing enough?
It might be, how high is your coverage?
How do you know methods work together?
Functional tests allow you to test individual scenarios in your application
Acceptance tests allow you to test the interactions in your application
Where do we start?
Install Codeception
Eh…wait a minute…
Better way to install* *http://blog.doh.ms/2014/10/13/installing-composer-packages
Add to PATH
Bootstrap Codeception
Configure Codeception
Run Codeception
Add Laravel5 Helper
Codecept build
Our First Test
Login Form
Login Form Template
Generate Functional Test
Writing our test
Click on what? Click on ‘Copy CSS Path’
How to run tests codecept run // runs all tests codecept run functional // run functional tests codecept run functional tests/functional/ AdminCanLoginCept.php // run single test Can also add -vv or -vvv for detailed output
Run our test
What are we testing? The route ‘auth/login’ shows us a login form Submitting login form performs some action Dashboard (after successful login) displays Some success text exists “Hello admin!”
What are we NOT testing? Navigation on the page “Remember Me” functionality works Password Reset works User is actually an admin
Functional testing tests a slice of functionality of the whole system
We don’t care about Navigation on the page “Remember Me” functionality works Password Reset works User is actually an admin
We care about $I->wantTo('ensure an admin can log in');
What if our test fails?
Update our password
Run Our test
Failed tests output
Test will pass when… the user exists in the database the routes (get & post) work login form view renders properly dashboard view renders properly
Framework Modules
Using Laravel5 Module amOnAction is translated to the /auth/login url
Test Passes
Check the DB for user seeRecord() will fail if the user does not exist
User is in the database
Check for errors Useful for checking that error messages are returned dontSee() ensures text does not exist on view
Testing your API
Generate API Suite api.suite.yml
Test API Create User http://codeception.com/docs/10-WebServices
Now we know slices of our application work
Acceptance Testing
Testing Real(ish) things
Using a real(ish) browser
Add PhantomJS
Configure acceptance.suite.yml
Export your Database php artisan migrate php artisan db:seed mysqldump -u homestead -psecret homestead > tests/_data/dump.sql
Our 1st acceptance test
Test passes
That looks a lot like our functional test!
Functional Test runs via PhpBrowser
Acceptance Test runs via PhantomJS
Testing Elements via XPath
Use XPath to submit form
/html/body/div/div/div/div/ div[2]/form/div[4]/div/button
Even more specific
Checking specific elements
Using XPath for form fields isn’t very easy to read. XPath isn’t very easy to read…
When to use XPath? When you can’t add an ID or Class on an element When you want to ensure styles/formatting (error messages, flash messages, etc) When testing navigation elements When testing any dynamic elements
Acceptance failures come with screenshots!
Log in and look for content that doesn’t exist
Test Failed!
We see our content does not exist on the page
Some issues… PhantomJS pretty broken on Travis-CI Acceptance tests are SLOW (and memory hogs) Acceptance tests aren’t unit tests, it can be hard to find WHAT is causing the tests to fail Functional & Acceptance tests aren’t replacements for unit tests, but can be easier to write if your app isn’t written to be testable
Codeception Resources http://www.codeception.com http://chrislema.com/acceptance-testing- wordpress-codeception Writing Acceptance Tests: https://vimeo.com/113466213
Testing Resources https://leanpub.com/grumpy-testinghttps://leanpub.com/grumpy-phpunit http://grumpy-learning.com
Questions?
Feedback! https://joind.in/talk/4c449 Joe Ferguson Twitter: @JoePFerguson Email: joe@joeferguson.me Freenode: joepferguson Contact Info:

Acceptance & Functional Testing with Codeception - SunshinePHP 2016