ci: pin elastic-agent version

[pkoutsovasilis]

What does this PR do?

This PR temporarily pins AGENT_VERSION and BEAT_VERSION (probably for historical reasons we need both, ty for the BEAT_VERSION hint @pchila ) to 9.1.0-SNAPSHOT in CI pipelines. This is an effort to mitigate breakages caused by the recent version bump to 9.2.0 which point to unavailable snapshot artifacts for 9.2.0-SNAPSHOT in the unified release.

In particular, this patch:

• Pins AGENT_VERSION, AGENT_PACKAGE_VERSION, and BEAT_VERSION across all affected pipelines.
• Skips upgrade_broken_package_test.go since it fails due to the version mismatch until the pinning is removed.
• Wraps the parsing logic of version.go in Bash and PowerShell scripts to only run when the AGENT_VERSION is not explicitly set.
• Updates cloud:push and cloud:load mage targets to respect the pinned AGENT_VERSION, replacing previously hardcoded fallback to version.go.

Why is it important?

Pinning the version ensures a consistent and known state for the CI runs until the 9.2.0 snapshot artifacts are fully available via DRA and unified stack release.

In addition, there's another issue introduced by #8352, which pins the ESS stack version used in CI to the latest stable version. This means that even if the 9.2.0 DRA snapshot was published, the CI would still be failing due to version incompatibility between the provisioned ESS stack and the elastic-agent binary under test, until the pinned ESS version point to a 9.2.0 stack.

Also, the cloud:push and cloud:load mage targets - introduced by #8035 - did not originally respect version overrides and required additional code changes to support environment-based overrides, similar to the rest of the codebase.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

None expected. These changes only affect CI behavior and not end-user installations or interactions.

How to test this PR locally

This changes regard only CI pipelines

Related issues

N/A

[dliappis]

Thanks for the quick action! So far, I am leaving a comment only as this is a draft, and will follow with an LGTM once the CI run is green.

We'll need to be careful with the backports of this PR!

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 9b75415

History

• 💔 Build #23257 failed cdd1f7c
• 💔 Build #23256 failed a22c6ef

cc @pkoutsovasilis

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[pchila]

Just a small correction on a comment. Not blocking for merge.
Thank you for the prompt action.
LGTM

[dliappis]

LGTM