Exploring with Automation James Thomas. 2022-01-14. EC Utbildning @qahiccupps
www.associationforsoftwaretesting.org www.ada.com @qahiccupps
Automation Using a tool that, largely unattended, replaces or extends the human ability to do a job Exploratory Testing @qahiccupps
The Information Space Known Unknown Knowns Unknowns Data generation Data analysis Data processing Repeated tasks Boring tasks Parallel tasks Documenting @qahiccupps
Today’s Scenario We’ve joined the Best Buy web team and want to get familiar with their API Today’s Mission Explore the Best Buy API using any tool that’s helpful to bootstrap our knowledge Today’s Caveats This is not a tutorial in tools or testing @qahiccupps
Testing an API Products, categories, and delivery costs. What do you know about? USB speakers? Here’s a list: … HTTP + JSON GET /actions/ { actions: [ product, categories, delivery ] } GET /product?type=USB_speaker [ {id:5, name: Akai}, {id:7, name: Sony},...] @qahiccupps
@qahiccupps
@qahiccupps
What Does a Product Look Like? @qahiccupps
What Do Product Descriptions Look Like? @qahiccupps
Can we do Better? Can we Automate? @qahiccupps
Is This Really Automation? @qahiccupps
Can we See More Results? @qahiccupps
Can we Get All of the Results? @qahiccupps
What are the Categories? @qahiccupps
What are the Categories in Use? @qahiccupps
Why so many Prices? @qahiccupps
@qahiccupps
What Did We Do? Explored and learned ● Observing ● Asking questions ● Finding expedient methods Found potential issues ● Product (server errors) ● Data (typos) ● Metadata (collision in category names) Built little by little ● Postman - curl ● Single command - pipes ● One-shot - scripts @qahiccupps
@qahiccupps
What Could We Do Next? Revisit interesting places Explore areas we didn’t look at Find another lens, e.g. database, unit tests Document some of the learning as regression tests @qahiccupps
Summary Automation ● isn’t only for the known-knowns ● can be exploited in exploration ● can give us new lenses to view a system through ● can amplify our capacity to explore ● can be used once and discarded ● we really just want it to help us to answer our questions @qahiccupps
Questions? @qahiccupps
Related Articles From my Blog Not Sure About Uncertainty Tools, Take Your Pick Stuck in the Middle Test Automation is not Automated Testing Don’t Just Check Please Use the Force Multiplier Exploratory Tooling Exploring to the Choir You Can Tidy the Data Bog-standard Testing @qahiccupps
Selected References TestPyramid, Martin Fowler Known Knowns, Wikipedia Best Buy API Playground (and others at Elizabeth Zagroba, Automation Panda) Exploratory Testing Elisabeth Hendrickson Exploratory Testing on Computer Interfaces Maaret Pyhäjärvi Image Credits Robot Clip art Map of London @qahiccupps
The Tools Postman Shell (pipe, redirect, variables, script) jq curl Linux utilities (grep, sort, uniq, wc, diff) Google Sheets (pivot table) @qahiccupps

Exploring with Automation

  • 1.
    Exploring with Automation JamesThomas. 2022-01-14. EC Utbildning @qahiccupps
  • 2.
  • 3.
    Automation Using a toolthat, largely unattended, replaces or extends the human ability to do a job Exploratory Testing @qahiccupps
  • 4.
    The Information Space Known Unknown KnownsUnknowns Data generation Data analysis Data processing Repeated tasks Boring tasks Parallel tasks Documenting @qahiccupps
  • 5.
    Today’s Scenario We’ve joinedthe Best Buy web team and want to get familiar with their API Today’s Mission Explore the Best Buy API using any tool that’s helpful to bootstrap our knowledge Today’s Caveats This is not a tutorial in tools or testing @qahiccupps
  • 6.
    Testing an API Products,categories, and delivery costs. What do you know about? USB speakers? Here’s a list: … HTTP + JSON GET /actions/ { actions: [ product, categories, delivery ] } GET /product?type=USB_speaker [ {id:5, name: Akai}, {id:7, name: Sony},...] @qahiccupps
  • 7.
  • 8.
  • 9.
    What Does aProduct Look Like? @qahiccupps
  • 10.
    What Do ProductDescriptions Look Like? @qahiccupps
  • 11.
    Can we doBetter? Can we Automate? @qahiccupps
  • 12.
    Is This ReallyAutomation? @qahiccupps
  • 13.
    Can we SeeMore Results? @qahiccupps
  • 14.
    Can we GetAll of the Results? @qahiccupps
  • 15.
    What are theCategories? @qahiccupps
  • 16.
    What are theCategories in Use? @qahiccupps
  • 17.
    Why so manyPrices? @qahiccupps
  • 18.
  • 19.
    What Did WeDo? Explored and learned ● Observing ● Asking questions ● Finding expedient methods Found potential issues ● Product (server errors) ● Data (typos) ● Metadata (collision in category names) Built little by little ● Postman - curl ● Single command - pipes ● One-shot - scripts @qahiccupps
  • 20.
  • 21.
    What Could WeDo Next? Revisit interesting places Explore areas we didn’t look at Find another lens, e.g. database, unit tests Document some of the learning as regression tests @qahiccupps
  • 22.
    Summary Automation ● isn’t onlyfor the known-knowns ● can be exploited in exploration ● can give us new lenses to view a system through ● can amplify our capacity to explore ● can be used once and discarded ● we really just want it to help us to answer our questions @qahiccupps
  • 23.
  • 24.
    Related Articles Frommy Blog Not Sure About Uncertainty Tools, Take Your Pick Stuck in the Middle Test Automation is not Automated Testing Don’t Just Check Please Use the Force Multiplier Exploratory Tooling Exploring to the Choir You Can Tidy the Data Bog-standard Testing @qahiccupps
  • 25.
    Selected References TestPyramid, MartinFowler Known Knowns, Wikipedia Best Buy API Playground (and others at Elizabeth Zagroba, Automation Panda) Exploratory Testing Elisabeth Hendrickson Exploratory Testing on Computer Interfaces Maaret Pyhäjärvi Image Credits Robot Clip art Map of London @qahiccupps
  • 26.
    The Tools Postman Shell (pipe,redirect, variables, script) jq curl Linux utilities (grep, sort, uniq, wc, diff) Google Sheets (pivot table) @qahiccupps