The Key to DevOps? Testing Early in the Pipeline
Welcome Joe Lust Software Engineer mabl
“Development cycles are getting shorter and new features are coming faster than ever…There simply isn’t enough time to test.” - Fortune 250 QA Leader
DevOps is disrupting software testing Change happens infrequently Weeks or months to write tests Weeks to execute tests Pre-DevOps Change is constant Hours to write tests Minutes to execute tests DevOps
Outdated testing tools and processes can’t keep up Testing Development Ops
Poll question How often does your team deploy?
2 80 releases every week The new normal
Compute requirements 50 scripts 3 scenarios (average 8 browser variations 3 targets 3 deploys runs/day 10,800 runs/day 7 mins/run compute hours/day Tests add up quickly A simple case - web testing
...But it’s bursty concurrency Concurrency needed over the course of a day - by test window
“While test automation is not new, past automation approaches have not helped app dev teams keep up with increased demands of modern delivery.” - The Forrester WaveTM for Continuous Functional Test Automation Suites, Q2 2020
The era of intelligent testing is here 12 ✔ Tests run in the cloud ✔ Tests are easy to create ✔ Tests adapt seamlessly to change ✔ Tests produce insightful output ✔ Testing solutions are part of CI/CD pipeline
Shift-left enables continuous testing No time for defects and broken tests in DevOps Code PR Master QA Staging Prod AttentiontoQuality Shift-Left Model Traditional Model
Benefits to testing earlier in the pipeline
The benefits of testing earlier in development Saving money Better team collaboration Saving time
Save time and increase test coverage MaestroQA Case Study
Poll question What do you think the relative cost of a bug in production is?
Relative time and cost to fix bugs throughout development
Create a culture of quality ■ Integrated release process ■ Effective communication ■ Joint issue tracking The value of collaboration? More efficient teams that move faster.
Go to the source: shift quality left Minimize business impacting bugs by shifting quality testing to the left
How mabl does shift-left testing
Shift-left workflow with mabl Local Branch Environment Preview Environment QA Environment Prod Environment Proposed Changes Master
Shift-left workflow with mabl Local Branch Environment QA Environment Prod Environment Start testing here Preview Environment
mabl CLI (Command Line Interface) A command-line tool that allows users to create, edit and run tests without ever opening the mabl UI ■ Stay in your workflow ■ Ensure code changes don’t introduce new issues
Branching Branches helps users make isolated changes to a test without affecting the master version of that test. ■ Version control ■ Ensure your changes work ■ Snapshotting
Headless runner As part of the mabl CLI, it allows users to easily run and debug tests on their local machine, including support for headless mode ■ Get lightning fast feedback ■ Save time while reducing risk ■ These local tests are FREE to run
Jest unit test runner Write mabl UI tests as standard Javascript/Typescript unit tests ■ Run mabl tests w/ your unit tests ■ Easily run from your IDE ■ Store test setup in source control
Shift-left workflow with mabl Local Branch Environment Preview Environment Test here QA Environment Prod Environment
Preview environments ■ On demand, short live environments ■ Used to validate changes and deployments ■ No/minimal impact if broken ▲ Single use ■ Also known as: ▲ Review app ▲ Ephemeral environments
Harness Your Preview Environments ■ Run your end to end automation against the preview ■ Link your merge requests to the preview ■ Test merges to main before you merge
31#GitLabCommit Slide option 1 of 2 This seems like a completely innocuous MR, especially if you don’t speak CSS.
32#GitLabCommit Slide option 1 of 2 But what if we could SEE the changes in a Preview Environment?
33#GitLabCommit Slide option 1 of 2 Oh, no! This certainly isn’t what I meant to approve!
Shift-left workflow with mabl Local Branch Environment Preview Environment QA Environment Prod Environment Test here Proposed Changes Master Test here
Test early. Test often. Test everywhere.
Intelligent Test Automation for Your Entire Team Reliable Tests: Easily create and maintain automated UI tests Unified Platform: Increase test coverage across your applications with a single platform Comprehensive Results: Gain actionable insights across the entire development lifecycle Customer-Centric Testing: Create end-to-end tests from your end user’s perspective
Questions? Learn more at mabl.com
The Key to DevOps? Testing Early in the Pipeline

The Key to DevOps? Testing Early in the Pipeline

  • 1.
    The Key toDevOps? Testing Early in the Pipeline
  • 2.
  • 3.
    “Development cycles aregetting shorter and new features are coming faster than ever…There simply isn’t enough time to test.” - Fortune 250 QA Leader
  • 4.
    DevOps is disruptingsoftware testing Change happens infrequently Weeks or months to write tests Weeks to execute tests Pre-DevOps Change is constant Hours to write tests Minutes to execute tests DevOps
  • 5.
    Outdated testing toolsand processes can’t keep up Testing Development Ops
  • 6.
    Poll question How oftendoes your team deploy?
  • 7.
    2 80 releases everyweek The new normal
  • 8.
  • 9.
    ...But it’s burstyconcurrency Concurrency needed over the course of a day - by test window
  • 10.
    “While test automationis not new, past automation approaches have not helped app dev teams keep up with increased demands of modern delivery.” - The Forrester WaveTM for Continuous Functional Test Automation Suites, Q2 2020
  • 12.
    The era ofintelligent testing is here 12 ✔ Tests run in the cloud ✔ Tests are easy to create ✔ Tests adapt seamlessly to change ✔ Tests produce insightful output ✔ Testing solutions are part of CI/CD pipeline
  • 13.
    Shift-left enables continuoustesting No time for defects and broken tests in DevOps Code PR Master QA Staging Prod AttentiontoQuality Shift-Left Model Traditional Model
  • 14.
    Benefits to testingearlier in the pipeline
  • 15.
    The benefits oftesting earlier in development Saving money Better team collaboration Saving time
  • 16.
    Save time andincrease test coverage MaestroQA Case Study
  • 17.
    Poll question What doyou think the relative cost of a bug in production is?
  • 18.
    Relative time andcost to fix bugs throughout development
  • 19.
    Create a cultureof quality ■ Integrated release process ■ Effective communication ■ Joint issue tracking The value of collaboration? More efficient teams that move faster.
  • 20.
    Go to thesource: shift quality left Minimize business impacting bugs by shifting quality testing to the left
  • 21.
    How mabl doesshift-left testing
  • 22.
    Shift-left workflow withmabl Local Branch Environment Preview Environment QA Environment Prod Environment Proposed Changes Master
  • 23.
    Shift-left workflow withmabl Local Branch Environment QA Environment Prod Environment Start testing here Preview Environment
  • 24.
    mabl CLI (CommandLine Interface) A command-line tool that allows users to create, edit and run tests without ever opening the mabl UI ■ Stay in your workflow ■ Ensure code changes don’t introduce new issues
  • 25.
    Branching Branches helps usersmake isolated changes to a test without affecting the master version of that test. ■ Version control ■ Ensure your changes work ■ Snapshotting
  • 26.
    Headless runner As partof the mabl CLI, it allows users to easily run and debug tests on their local machine, including support for headless mode ■ Get lightning fast feedback ■ Save time while reducing risk ■ These local tests are FREE to run
  • 27.
    Jest unit testrunner Write mabl UI tests as standard Javascript/Typescript unit tests ■ Run mabl tests w/ your unit tests ■ Easily run from your IDE ■ Store test setup in source control
  • 28.
    Shift-left workflow withmabl Local Branch Environment Preview Environment Test here QA Environment Prod Environment
  • 29.
    Preview environments ■ Ondemand, short live environments ■ Used to validate changes and deployments ■ No/minimal impact if broken ▲ Single use ■ Also known as: ▲ Review app ▲ Ephemeral environments
  • 30.
    Harness Your PreviewEnvironments ■ Run your end to end automation against the preview ■ Link your merge requests to the preview ■ Test merges to main before you merge
  • 31.
    31#GitLabCommit Slide option 1of 2 This seems like a completely innocuous MR, especially if you don’t speak CSS.
  • 32.
    32#GitLabCommit Slide option 1of 2 But what if we could SEE the changes in a Preview Environment?
  • 33.
    33#GitLabCommit Slide option 1of 2 Oh, no! This certainly isn’t what I meant to approve!
  • 34.
    Shift-left workflow withmabl Local Branch Environment Preview Environment QA Environment Prod Environment Test here Proposed Changes Master Test here
  • 35.
  • 36.
    Intelligent Test Automationfor Your Entire Team Reliable Tests: Easily create and maintain automated UI tests Unified Platform: Increase test coverage across your applications with a single platform Comprehensive Results: Gain actionable insights across the entire development lifecycle Customer-Centric Testing: Create end-to-end tests from your end user’s perspective
  • 37.