Skip to content

Conversation

@steveny91
Copy link
Contributor

@steveny91 steveny91 commented Dec 22, 2025

What does this PR do?

This PR also introduces a criterion that our setup checks before running the e2e tests. The tests are currently broken on master. I addressed version ≥ 5.0 in #22176. However, after merging that change, master was still failing the e2e tests, even though the Aerospike tests for the failing environments were passing.

The hypothesis is that this happens because the tests in the PR were running in parallel. When run in parallel, each environment spins up its own runner and isolated environment and needs to pull its own agent. This additional download time gives the environment enough time to calculate latency. When the tests run sequentially, however, the runner already has the Datadog Agent image cached, which speeds up environment spin-up.

Logs when running in sequence (currently failing):

master-py3: Pulling from datadog/agent-dev Digest: sha256:0f8115077ada10351e0a7683f7063e971697918c275e6fb0090002994db87750 Status: Image is up to date for datadog/agent-dev:master-py3 

Logs when running in parallel:

master-py3: Pulling from datadog/agent-dev a4fe053ff13d: Pulling fs layer a4fe053ff13d: Verifying Checksum a4fe053ff13d: Download complete a4fe053ff13d: Pull complete Digest: sha256:0f8115077ada10351e0a7683f7063e971697918c275e6fb0090002994db87750 Status: Downloaded newer image for datadog/agent-dev:master-py3 

Misc:
Output when it's not ready:

docker exec -it aerospike asinfo -v 'latencies:' batch-index:;{test}-read:;{test}-write:;{test}-udf:;{test}-query: 

Output when ready:

batch-index:msec,0.0,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00;{test}-read:msec,0.0,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00;{test}-write:msec,0.0,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00;{test}-udf:;{test}-query: 
@codecov
Copy link

codecov bot commented Dec 22, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.03%. Comparing base (4672754) to head (9422e8f).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
Copy link
Contributor

@iliakur iliakur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌 thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment