DEV Community

Cover image for 🚀 Building Playwright Framework Step By Step - Implementing CI/CD
idavidov13
idavidov13

Posted on • Edited on • Originally published at idavidov.eu

🚀 Building Playwright Framework Step By Step - Implementing CI/CD

#playwright #testing #automation #tutorial

🎯 Introduction

Integrating testing frameworks within CI/CD pipelines embodies a transformative approach to software delivery, enhancing both speed and quality! 🌟 Continuous Integration (CI) and Continuous Delivery (CD) practices facilitate the automatic execution of tests at various stages of the development lifecycle, from initial code commit through to production deployment. By embedding a testing framework into CI/CD workflows, teams can automatically run unit, integration, UI, and API tests. This not only accelerates feedback loops but also significantly reduces the risk of defects making their way into production! 🛡️

🎯 Why CI/CD Integration?: Automated testing in CI/CD pipelines ensures consistent quality, faster releases, and early bug detection - it's the foundation of modern DevOps!

🌟 Key Benefits:

  • 🔍 Early Bug Detection: Automated tests within CI/CD identify issues early, when they are easier and less costly to fix
  • ✅ Quality Assurance: Consistent testing ensures that code changes meet quality standards before they are merged or released
  • ⚡ Release Velocity: Automating the testing process eliminates bottlenecks in the deployment pipeline, enabling faster release cycles
  • 🛡️ Reliability and Confidence: Automated, repeatable testing ensures that software releases are more reliable, increasing confidence in the development and deployment process

💡 Modern Development: Incorporating a testing framework into CI/CD pipelines is not just a best practice; it's a cornerstone of modern, efficient, and reliable software development processes!

✅ Prerequisites

This article builds directly on the concepts from previous ones. To get the most out of it, you should have:

🔧 Implementing GitHub Actions Workflow

📋 Workflow Details

Our comprehensive CI/CD pipeline includes multiple stages and advanced features! 🎯

  • 🔐 Environment Variables: All workflows use environment variables defined in GitHub Secrets
  • 🧪 Test Stages: The Pipeline includes stages for setup and smoke test, and testing stage (sanity tests, API tests, and regression tests)
  • 📊 Reports: Test reports are uploaded as artifacts for review
  • 🔗 Merge Report (Optional): All uploaded reports are downloaded, merged in one, and merged report is uploaded as artifact
  • 🏗️ Build Report (Optional): Merged report is downloaded and github-pages are generated and uploaded as artifact
  • 🌐 Deploy Report (Optional): Uploaded github-pages is deployed in GitHub Pages for the workflow

⚠️ Important Note: Due to GitHub consideration of url to consist secrets, the workaround was to hardcode the url for the GitHub Pages, so it appears just below the job name and it is easily accessible for everyone.

📝 TO DO: If the job is implemented, the URL should be updated.

🧪 Add failingTest.spec.ts file in the tests/clientSite folder of the project for demonstration purposes 

import { test, expect } from '../../fixtures/pom/test-options'; test('Failing Sanity Test', { tag: '@Sanity' }, async ({ homePage }) => { await homePage.navigateToHomePageUser(); expect(2).toEqual(3); }); test('Failing API Test', { tag: '@Api' }, async ({ homePage }) => { await homePage.navigateToHomePageUser(); expect(2).toEqual(3); }); test('Failing Regression Test', { tag: '@Regressions' }, async ({ homePage }) => { await homePage.navigateToHomePageUser(); expect(2).toEqual(3); });
Enter fullscreen mode Exit fullscreen mode

🔧 Create .github/actions folder in the root directory of the project and add playwright-report/action.yml file to it and playwright-setup/action.yml file to it

Playwright Report Action:

name: 'Playwright Report' description: 'Upload Playwright Report' inputs: test-step-outcome: description: 'Outcome of the test step (success or failure)' required: true blob-report-upload: description: 'Wheather to always upload blob report as artifact or only on failure' required: false default: 'false' blob-report-retention-days: description: 'Retention days for blob report' required: false default: '3' blob-report-path: description: 'Path for blob report' required: false default: 'blob-report' runs: using: 'composite' steps: - name: Upload BLOB Report uses: actions/upload-artifact@v4 if: ${{ inputs.blob-report-upload == 'true' || inputs.test-step-outcome == 'failure' }} with: name: blob-report-${{ github.job }} path: ${{ inputs.blob-report-path }} retention-days: ${{ inputs.blob-report-retention-days }}
Enter fullscreen mode Exit fullscreen mode

Playwright Setup on Runner Action:

name: 'Playwright Setup on Runner' description: 'Setup Playwright on Runner by getting the code, setup Node, getting and caching dependencies and getting and caching Playwright Browsers' inputs: caching-dependencies: description: 'Wheather to cache dependencies or not' required: false default: 'true' caching-browsers: description: 'Wheather to cache browsers or not' required: false default: 'true' runs: using: 'composite' steps: - name: Setup Node uses: actions/setup-node@v4 with: node-version: lts/* - name: Cache Dependencies if: inputs.caching-dependencies == 'true' id: cache-dependencies uses: actions/cache@v4 with: path: node_modules key: deps-node-modules-${{ hashFiles('**/package-lock.json') }} - name: Install dependencies if: steps.cache-dependencies.outputs.cache-hit != 'true' || inputs.caching-dependencies != 'true' run: npm ci shell: bash - name: Cache Playwright Browsers if: inputs.caching-browsers == 'true' id: cache-browsers uses: actions/cache@v4 with: path: ~/.cache/ms-playwright key: playwright-browsers-${{ runner.os }} - name: Install Playwright Browsers if: steps.cache-browsers.outputs.cache-hit != 'true' || inputs.caching-browsers != 'true' run: npx playwright install --with-deps chromium shell: bash
Enter fullscreen mode Exit fullscreen mode

🔧 Create 'playwright-custom-runner.yml' file in the .github/workflows folder of the project 

name: Playwright Tests on Custom Runner Extended on: push: branches: [main, master] pull_request: branches: [main, master] workflow_dispatch: env: URL: ${{ secrets.URL }} API_URL: ${{ secrets.API_URL }} USER_NAME: ${{ secrets.USER_NAME }} EMAIL: ${{ secrets.EMAIL }} PASSWORD: ${{ secrets.PASSWORD }} permissions: contents: read pages: write id-token: write concurrency: group: 'pages' cancel-in-progress: false jobs: setup-and-smoke-test: name: Setup and Smoke Test timeout-minutes: 60 runs-on: ubuntu-latest outputs: smoke_outcome: ${{ steps.smoke.outcome }} steps: - name: Checkout code uses: actions/checkout@v4 - name: Playwright Setup on Runner uses: ./.github/actions/playwright-setup - name: Run Smoke tests id: smoke run: npm run smoke - name: Upload Playwright Report if: always() uses: ./.github/actions/playwright-report with: test-step-outcome: ${{ steps.smoke.outcome }} sanity-test: name: Sanity Test needs: setup-and-smoke-test timeout-minutes: 60 runs-on: ubuntu-latest outputs: sanity_outcome: ${{ steps.sanity.outcome }} steps: - name: Checkout code uses: actions/checkout@v4 - name: Playwright Setup on Runner uses: ./.github/actions/playwright-setup - name: Run Sanity tests id: sanity run: npm run sanity - name: Upload Playwright Report if: always() uses: ./.github/actions/playwright-report with: test-step-outcome: ${{ steps.sanity.outcome }} api-test: name: API Test needs: setup-and-smoke-test timeout-minutes: 60 runs-on: ubuntu-latest outputs: api_outcome: ${{ steps.api.outcome }} steps: - name: Checkout code uses: actions/checkout@v4 - name: Playwright Setup on Runner uses: ./.github/actions/playwright-setup - name: Run API tests id: api run: npm run api - name: Upload Playwright Report if: always() uses: ./.github/actions/playwright-report with: test-step-outcome: ${{ steps.api.outcome }} regression-test: name: Regression Test needs: setup-and-smoke-test timeout-minutes: 60 runs-on: ubuntu-latest outputs: regression_outcome: ${{ steps.regression.outcome }} steps: - name: Checkout code uses: actions/checkout@v4 - name: Playwright Setup on Runner uses: ./.github/actions/playwright-setup - name: Run Regression tests id: regression run: npm run regression - name: Upload Playwright Report if: always() uses: ./.github/actions/playwright-report with: test-step-outcome: ${{ steps.regression.outcome }} merge-report: name: Merge Report if: always() needs: [setup-and-smoke-test, sanity-test, api-test, regression-test] timeout-minutes: 60 runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v4 - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: lts/* - name: Install Dependencies run: npm ci - name: Download blob reports from GitHub Actions Artifacts uses: actions/download-artifact@v4 with: path: all-blob-reports pattern: blob-report-* merge-multiple: true - name: Merge into HTML Report run: npx playwright merge-reports --reporter html ./all-blob-reports - name: Upload HTML report uses: actions/upload-artifact@v4 with: name: Merged HTML Reports path: playwright-report retention-days: 3 build-report: name: Build Report if: always() needs: merge-report runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v4 - name: Download Merged HTML Reports uses: actions/download-artifact@v4 with: name: Merged HTML Reports path: ./_site - name: Upload artifact uses: actions/upload-pages-artifact@v3 with: path: ./_site depoloy-report: name: Deploy Report if: always() needs: build-report environment: name: github-pages url: 'https://idavidov13.github.io/Playwright-Framework/' runs-on: ubuntu-latest steps: - name: Deploy to GitHub Pages id: deployment uses: actions/deploy-pages@v4
Enter fullscreen mode Exit fullscreen mode

🔐 Set Up GitHub Actions Secrets

Secure configuration is essential for protecting sensitive information! 🛡️

🔒 Security Best Practice: Never hardcode sensitive information in your workflows - always use GitHub Secrets for credentials and URLs.

Steps to configure secrets:

  • 📂 Navigate to repository setting in GitHub
  • 🔐 Go to Secrets and variables
  • ⚙️ Go to Actions
  • ➕ Click on New repository secret
  • 📝 Add the following secrets:
    • URL - Your application URL
    • API_URL - Your API base URL
    • USER_NAME - Test user name
    • EMAIL - Test user email
    • PASSWORD - Test user password

🌐 Enable GitHub Pages

GitHub Pages deployment provides easy access to your test reports! 📊

🎯 Public Accessibility: GitHub Pages makes your test reports accessible to the entire team without requiring repository access.

Configuration steps:

  • 📂 Navigate to repository setting in GitHub
  • 📄 Go to Pages
  • ⚙️ Select Build and deployment source GitHub Actions

🎯 What's Next?

With the implementation of CI/CD, we've accomplished a major part of the work! 🎉 From this point forward, you're equipped with all the essential tools needed to develop an automation framework for various applications. Continue to experiment and expand your knowledge! 🚀

💬 Community: Please feel free to initiate discussions on this topic, as every contribution has the potential to drive further refinement.

✨ Congratulations! You've built a comprehensive Playwright testing framework with full CI/CD integration. Keep exploring and building amazing test automation solutions! 🌟

🙏🏻 Thank you for reading! Building robust, scalable automation frameworks is a journey best taken together. If you found this article helpful, consider joining a growing community of QA professionals 🚀 who are passionate about mastering modern testing.

Join the community and get the latest articles and tips by signing up for the newsletter.

Top comments (0)