Exploratory Testing on Computer interfaces

@maaretp http://maaretp.com Exploratory Testing of Computer Interfaces by Maaret Pyhäjärvi

@maaretp http://maaretp.com https://github.com/maaretp/exploring-interfaces Maaret Pyhäjärvi Attribution (Finland) http://creativecommons.org/licenses/by/1.0/fi/ http://creativecommons.org/licenses/by/1.0/fi/deed.en

@maaretp http://maaretp.com Exploratory testing is a systematic approach for discovering risks using rigorous analysis techniques coupled with testing heuristics. - Elisabeth Hendrickson

@maaretp http://maaretp.com Exploratory Testing: Frame of Management ”A day’s work” Vision (“Sandbox”) Current Charter Other Charters Details Bug Reports Perception of quality and coverage Quality ReportDebriefing Tester Test Manager Past Results Obstacles Outlook Feelings ? # xCharter backlog of the future testing Out of budget Next in importance! #, ?, x, + 20:20:60 Session sheets of the past testing Idea of exploration Metrics summary Coaching 4 Playbooks Coverage outlines

@maaretp http://maaretp.com An application programming interface (API) is a set of routines, protocols, and tools for building software and applications.

@maaretp http://maaretp.com (Exploratory) Tester Product is my external imagination

@maaretp http://maaretp.com Strong-Style Navigation “For an idea to go from your head to the computer it must go through someone else’s hands” *http://llewellynfalco.blogspot.fi/2014/06/llewellyns-strong-style-pairing.html

@maaretp http://maaretp.com Situation: You are in a real job interview for a QE/SQE/Tester position and they ask you following question. Do not use several minutes to think the answer, instead try to answer as you would answer in real interview situation (start answering in some 10-20 seconds).

@maaretp http://maaretp.com How WouldYou Test a Text Field?

@maaretp http://maaretp.com Junior tester or not a tester Category definition:They know little or nothing about testing Example answer types: •  Would give less that 5 type of inputs: e.g. only characters, numbers and funny character like "%&$ •  Would assume that this is a unit test •  Would test it Automatically (and nothing else is said) •  Would talk about hardware requirements •  Would separate test design and executions as own tasks and even to dedicated people •  Would think this is really big task and requires several people to do it •  Would think that text field is only for showing the text (not looking like field on screen)

@maaretp http://maaretp.com Tester / QE with functional testing experience Category definition: They can quickly list basic things that need to be covered, but stop there and They can follow given tasks and requirements, but don't really search for all the facts Example answer types: •  Lists 5 or more things from following testing ideas: •  alphabetic + number; special characters; double-byte characters •  fill up the field •  try to copy/paste to and from the field •  character limit •  correct character's shown •  erasing text •  does text fit to the field •  leave blank? •  line change with Enter? •  HTML •  SQL •  Lists 5 from above and then starts talking about processes (which this question was not about) like: •  Returning things to developers •  Entering bugs •  Retesting after developer has fixed the bugs

@maaretp http://maaretp.com Excellent functional tester / close to real senior QE Category definition: They can come up with huge amount of different ideas that would need to be tested, and they realize that there isn't necessarily time to do it AND They at some point notice that they actually want to know what this text field is? •  Example answer types: •  Instead of just listing the things in above category also understands things like: •  equivalence classes, boundary tests, positive and negative testing, fractions, ... •  finding requirements and limitations •  is there time/value/reason to test it more? •  different testing levels (e.g. for unit testing they would need to see the code) •  security testing (e.g. SQL injections, XSS, ...) •  Very long answers (almost like a book) that covers everything (well good that you know that, but there is shorter and better answer there) •  Answer includes talking to developers, product owners, customers, ... •  In addition to one or more of the 3 above, they at some point noticed that they actually need to figure out what this text field is all about

@maaretp http://maaretp.com The real senior QE Category definition: They answer back with questions and they can question everything first: ideas, requirements, business needs, assumptions, … Example answer types: •  What is this text field about? •  My first thought would be "With the information given - in no way". But as this wouldn't be a very polite answer, I would start asking for more information about the context

@maaretp http://maaretp.com This_box Text Field

@maaretp http://maaretp.com "All the brilliant people working on the same thing, at the same time, in the same space, on the same computer." -- Woody Zuill Lets Test!

@maaretp http://maaretp.com Gilded Rose By Emily Bache (available in 32 languages…) https://github.com/emilybache/GildedRose-Refactoring-Kata

@maaretp http://maaretp.com Disposable test automation Test automation as documentation

@maaretp http://maaretp.com Exploratory Testing Test Automation Exploratory Tester

@maaretp http://maaretp.com Test Automation Exploratory Testing Test Automation Specialist

@maaretp http://maaretp.com Exploratory Tester Test Automation Specialist Focus on the 1st (always different) test execution because repeating narrows scope! Focus on the repeating text execution to capture it. Repeating exactly is valuable!

@maaretp http://maaretp.com REST APIs Libraries Frameworks Languages Methods CLI Interfaces

@maaretp http://maaretp.com DX Developer Experience

@maaretp http://maaretp.com Building it the First Time Debugging and Maintaining

@maaretp http://maaretp.com How to Explore •  Do something with it •  Find out what is the common thing to do with it •  Find out what you could do with it •  Reflect after anything and everything – make notes –  What do we know from other connections? –  What do we know from empirical evidence? –  How do we turn it all into empirical evidence?

@maaretp http://maaretp.com REST API

@maaretp http://maaretp.com http://qa-matters.com/2016/07/30/vader-a-rest-api-test-heuristic/ Verbs Authorization/authentication Data Errors Responsiveness

@maaretp http://maaretp.com Methods HTTP GET - R HTTP POST - CU HTTP PUT - U HTTP DELETE - D HTTP PATCH - U Status Codes 1xx: Informational 2xx: Success 3xx: Redirection 4xx: Client Error 5xx: Server Error

@maaretp http://maaretp.com LIBRARY

@maaretp http://maaretp.com Structure Function Data Platform Operations Time SFDPOT heuristics from James Bach / Michael Bolton, Rapid SoftwareTesting

@maaretp http://maaretp.com Approvers do •  Formatting •  Sorting •  File Extensions •  Scrubbing (removing common inconsistencies) •  Serialization(saving to a file) •  Mocking •  Proxying •  Rendering •  Execution (e.g. retrieve the URL) •  Aggregating test cases •  File naming •  PRINCIPLE: ”Every time you handle this type of object, you do these things to it.” I LEARNED ABOUT FUNCTIONS

@maaretp http://maaretp.com Reporters do •  Waiting •  Scrubbing (removing common inconsistencies) •  Execution •  Launching •  Serialization •  Decompilation •  Chain of responsibility •  Creating Approved file •  Environmental awareness I LEARNED ABOUT FUNCTIONS

@maaretp http://maaretp.comI LEARNED ABOUTTHE ENVIRONMENT

@maaretp http://maaretp.com Approvers do •  Formatting •  Sorting •  File Extensions •  Scrubbing (removing common inconsistencies) •  Serialization (saving to a file) •  Mocking •  Proxying •  Rendering •  Execution (e.g. retrieve the URL) •  Aggregating test cases •  File naming •  PRINCIPLE: ”Every time you handle this type of object, you do these things to it.” I LEARNED ABOUT FUNCTIONS

@maaretp http://maaretp.com Patterns of Exploration for APIs Lessons Distilled

@maaretp http://maaretp.com Working with limited understanding: Focus!

@maaretp http://maaretp.com Calls and Operations. Inputs and Outputs. Exceptions.

@maaretp http://maaretp.com Target of your testing vs. the environment it depends on

@maaretp http://maaretp.com Specific user with a specific purpose

@maaretp http://maaretp.com Why would anyone use this?

@maaretp http://maaretp.com Think lifecycle Versioning and Deprecation

@maaretp http://maaretp.com Google for Concepts like “Conservative Overloading Strategy”

@maaretp http://maaretp.com Fast-track your understanding: collaborate

@maaretp http://maaretp.com Documentation Matters a Lot for APIs

@maaretp http://maaretp.com Explore to create documentation

@maaretp http://maaretp.com Some patterns become visible with repetition

@maaretp http://maaretp.com Summing It Up For Exploratory Testing Key Concepts

@maaretp http://maaretp.com What Testing gives Us TestAutomation ExploratoryTesting SPEC FEEDBACK REGRESSION GRANULARITY GUIDANCE UNDERSTANDING MODELS SERENDIPITYTesting as artifact creation Testing as performance

@maaretp http://maaretp.com Before implementing While implementing Before production While in production Testing as artifact creation Testing as performance (exploration) Ways to think of testing peak at different moments with regards to adding a capability TOTALS Testing is Everywhere!

@maaretp http://maaretp.com Exploratory Testing: Frame of Management ”A day’s work” Vision (“Sandbox”) Current Charter Other Charters Details Bug Reports Perception of quality and coverage Quality ReportDebriefing Tester Test Manager Past Results Obstacles Outlook Feelings ? # xCharter backlog of the future testing Out of budget Next in importance! #, ?, x, + 20:20:60 Session sheets of the past testing Idea of exploration Metrics summary Coaching 60 Playbooks Coverage outlines

@maaretp http://maaretp.com Exploration Skills Source:Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory Testing Dynamics. v.2.2. 2009 Self- management Developing ideas Examining product Done To Do Issues Coverage All sources available Best use of time – effective and efficient work Making models Tool support – creative solutions Risk-based testing – scientific approach Keeping one’s eyes open 61

@maaretp http://maaretp.com 1.00^365 = 1.00 1.01^365 = 37.8

@maaretp http://maaretp.com Experts are not the one ones who know the most but ones who learn the fastest

@maaretp http://maaretp.com Maaret Pyhäjärvi Email: maaret@iki.fi Twitter: @maaretp Web: maaretp.com Blog: visible-quality.blogspot.fi (please connect with me through Twitter or LinkedIn)

Exploratory Testing on Computer interfaces

More Related Content

Similar to Exploratory Testing on Computer interfaces

More from Maaret Pyhäjärvi

Recently uploaded

Exploratory Testing on Computer interfaces