DEV Community

Abhinav
Abhinav

Posted on

Building Robust CI/CD Pipeline

#python #opensource #cicd

In this post, I'll share my experience implementing a comprehensive CI/CD pipeline using GitHub Actions, writing tests for an existing codebase, and the valuable lessons learned along the way.

Setting Up GitHub Actions: From Zero to Production-Ready

The Challenge:

When I first approached this project, I was working with repo-contextr, a Python CLI tool that analyzes git repositories and packages their content for sharing with Large Language Models (LLMs). The project already had a solid foundation but lacked automated testing and continuous integration.

Designing the CI Workflow

In this, I made sure that I opt a robust approach to maintain the following:

  • Multi-platform compatibility (Linux, Windows, macOS)
  • Code quality through linting and formatting
  • Type safety with static analysis
  • Comprehensive test coverage
  • Successful package building

Here's the GitHub Actions workflow I implemented:

name: CI on: push: branches: [ main, develop ] pull_request: branches: [ main, develop ] jobs: test: runs-on: ${{ matrix.os }} strategy: fail-fast: false matrix: os: [ubuntu-latest, windows-latest, macos-latest] python-version: ['3.12'] steps: - uses: actions/checkout@v4 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} cache: 'pip' - name: Install dependencies run: | python -m pip install --upgrade pip pip install "ruff>=0.8.0" "mypy>=1.17.1" "pytest>=8.4.2" "pytest-cov>=6.0.0" pip install --editable . - name: Verify installation run: | python -c "import contextr; print('Package imported successfully')" python -c "from contextr.cli import app; print('CLI imported successfully')" - name: Run ruff linting run: | ruff check src tests - name: Run ruff formatting check run: | ruff format --check src tests - name: Run mypy type checking run: | mypy src continue-on-error: true - name: Run tests with coverage run: | pytest --cov=src --cov-report=xml --cov-report=term build: runs-on: ubuntu-latest needs: test steps: - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: '3.12' - name: Install build dependencies run: | python -m pip install --upgrade pip pip install build - name: Build package run: python -m build - name: Upload artifacts uses: actions/upload-artifact@v4 with: name: dist-packages path: dist/
Enter fullscreen mode Exit fullscreen mode

Writing Tests for Someone Else's Code: A Different Perspective

Understanding the perspective:

The Repo-Contextor already had a modular architecture as displayed below:

src/contextr/ ├── commands/ # CLI command implementations ├── config/ # Configuration management ├── discovery/ # File discovery logic ├── git/ # Git operations ├── processing/ # File reading and processing └── statistics/ # Token counting and stats
Enter fullscreen mode Exit fullscreen mode

My Approach:

  1. File Discovery Module: The file discovery system needed comprehensive testing for various scenarios: 
def test_discover_files_with_pattern(self, temp_dir): """Test discovering files with include pattern.""" (temp_dir / "file1.py").write_text("# python") (temp_dir / "file2.js").write_text("// javascript") (temp_dir / "file3.py").write_text("# python") result = discover_files([temp_dir], include_pattern="*.py") assert len(result) == 2 assert all(f.suffix == ".py" for f in result)
Enter fullscreen mode Exit fullscreen mode
  1. Git Operations: Git integration required careful mocking to test various scenarios: 
def test_get_git_info_valid_repo(self, sample_git_repo): """Test getting git info from valid repository.""" result = get_git_info(sample_git_repo) assert result is not None assert isinstance(result, dict) assert "commit" in result assert "branch" in result assert "author" in result assert "date" in result
Enter fullscreen mode Exit fullscreen mode
  1. CLI Interface: Testing the CLI required understanding the Typer framework and proper mocking: 
def test_basic_execution(self, mock_config, mock_package): """Test basic CLI execution without errors.""" # Mock configuration mock_config_obj = Mock() mock_config_obj.paths = ["."] mock_config_obj.include = None mock_config_obj.recent = False mock_config_obj.output = None
Enter fullscreen mode Exit fullscreen mode

Outcome:

  • Test count: Increased from 108 to 160 tests (+48% increase)
  • Code coverage: Improved from 35.72% to 78.03% (+42.31% improvement)
  • Module coverage: Several modules went from 0% to 95%+ coverage.

Conclusion:

Implementing CI/CD isn't just about automation—it's about adopting a mindset of continuous improvement and quality assurance. The process taught me several valuable lessons: Quality Gates Matter, Fast Feedback Loops, Documentation Through Code, Confidence in Changes.

Top comments (0)