Downloaded 76 times
![Reliable end to end tests with Selenium WebDriver Pawel Pabich, Senior Consultant at Readify blog: www.pabich.eu/blog twitter: @pawelpabich email: [email_address]](https://image.slidesharecdn.com/reliabletestswithseleniumwebdriver-110726073511-phpapp02/75/Reliable-tests-with-selenium-web-driver-1-2048.jpg)
Reliable tests with selenium web driver
- 1. Reliable end toend tests with Selenium WebDriver Pawel Pabich, Senior Consultant at Readify blog: www.pabich.eu/blog twitter: @pawelpabich email: [email_address]
- 2. My definition of end to end (e2e) test s Automated Often start at the UI Same as user acceptance test Often used to test user stories Often expressed using Given/When/Then Can describe what the system does Often slower than unit/integration test s
- 3. Reliable e2etests provide a lot of value Assure the code is shippable Provide solid safety net and in this way encourage constant refactoring Let QA focus on creative, exploratory testing
- 4. Unreliable e2etests are harmful Take a lot of time to investigate Make the development process slower than if there were no e2e tests at all Are extremely frustrating for all team members
- 5. How to write reliable e2e tests 1/2 Do NOT use Thread.Sleep. It is NEVER a good idea ALWAYS start with unit tests and fall back to e2e tests only when unit testing is not the right tool Focus on the happy path Treat test code as if it was Production code Do NOT share state between tests Do NOT use test recorders
- 6. How to write reliable e2e tests 2/2 Navigate directly to the screen under test Test only code you own , eg. all external systems are mocked Create a separate set of tests for integration with live external systems Always make sure the system is in a known initial state Test only one scenario at a time
- 7.
- 8. Summary If youcan't make it reliable do NOT write it at all. You will regret it. Links: Code: https://github.com/pawelpabich/Reliable-e2e-tests-with-Selenium-Webdriver Selenium: http://code.google.com/p/selenium/ Our own James Crisp : http://jamescrisp.org/2011/05/30/automated-testing-and-the-test-pyramid/
Editor's Notes
- #2 Selenium – Thoughtworks , WebDriver Google – Selenium/WebDriver a hopefully good combination of both
- #3 1. Well writtend, high level e2e test can play the role of live documentation.
- #4 1. Every time your CI build finishes succesfully a new piece of code can be deployed to production. So let’s say you don’t how e2e tests in place. Get the latests, the app is broken, figure out if it was you, not you, then who, what needs to be done to fix it. Now let’s imagine that all 20 guys on your team got the latest at the same time. 2. Accross the board
- #5 It takes more time to investiage a broken e2e test than a unit test so every false alarm is costly.
- #6 Happy path is easy, use unit tests for testing edge cases There is this attitue that tests are not deployed to Production so they don’t need the same attention as prouduction code. Wrong. Does not matter C#, Test or JS Test Recorders – not need to use them and then clean up the mess they create. I would rather start with clean code. It’s faster.
- #7 it’s a faster, if it takes 5 seconds to load the homepage and every tests needs to load it so each tests is 5 second longer than it needs to be External system, you don’t controll them so they are down your tests fail, you can have integration tests but keep them separate We all know that but with e2e it is even more important
- #8 Show the app first and explain DB Start with simple test (simple) Add ajax, show how to use (simpleajax) – works and doesn’t, every now and then Add 2s delay ,Debugging, TakeScreenshot (debugging) Fix the code with Thread.Sleep (fixedajax) – what’s wrong with this code, increase to 30s, waste of time Fix properly (reliableajax) (and decrease controller Thread.Sleep) show how it affects test execution time Change controller T.Sleep to 7 s and show TimeoutException, False/TE, bad API Fix with (betterretry) and show with 7s Optional: visibility (anotehr test) Long page load (hardcoded to 60 sec) but sometimes need (eg. Ncover), extensions method We’ve see some problems with finding elelements using CSS selectors If API not good enough, use access to IJavaScriptExecutor