Skip to content

Conversation

@iduartgomez
Copy link
Collaborator

Summary

Add CI workflow that automatically detects performance regressions in the transport layer by running level0 benchmarks on PRs.

How it works

  1. When main branch gets a push: Runs benchmarks and saves baseline to cache
  2. When PR is opened/updated: Restores main's baseline, runs benchmarks, compares
  3. If regression detected: Posts comment on PR with details

What it tests (level0 benchmarks)

Benchmark What it measures
level0/crypto/encrypt AES-GCM encryption throughput
level0/crypto/decrypt AES-GCM decryption throughput
level0/crypto/nonce Nonce generation speed
level0/serialization Packet serialization overhead
level0/packet_creation Full packet creation pipeline
level0/memcpy Memory copy baseline

These are pure logic benchmarks - deterministic, no I/O, no async runtime overhead.

Key design decisions

  • Non-blocking: PRs can still merge even if regressions detected (informational only)
  • Criterion comparison: Uses Criterion's built-in baseline comparison with 10% noise threshold
  • Cache strategy: Each main commit creates a new baseline; PRs use restore-keys to find most recent
  • Self-hosted runner: Runs on self-hosted for consistent hardware

Test plan

  • Merge this PR to establish initial baseline
  • Open a test PR touching transport code
  • Verify workflow runs and compares against baseline
  • Verify PR comment appears if regression introduced

🤖 Generated with Claude Code

Add CI workflow that runs level0 transport benchmarks on PRs and reports regressions compared to main branch baseline. Key features: - Runs level0 benchmarks (pure logic: AES-GCM, serialization, nonce gen) - Compares against cached baseline from main branch - Posts PR comment when regressions detected - Non-blocking: PRs can still merge even with regressions - Saves new baseline on each main branch commit The workflow uses Criterion's built-in comparison with GitHub Actions cache to track performance over time. Level0 benchmarks are deterministic and measure the transport layer's core operations without I/O dependencies. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
@iduartgomez iduartgomez added this pull request to the merge queue Dec 11, 2025
Merged via the queue into main with commit 2b31f08 Dec 11, 2025
10 checks passed
@iduartgomez iduartgomez deleted the ci/transport-perf-regression branch December 11, 2025 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants