[9.1] (backport #9992) fix: always clear the coordinator overridden state on err inside upgrade of coordinator

[mergify]

What does this PR do?

This PR fixes a regression introduced by #9634 where the coordinator’s overrideState could remain set if an upgrade attempt failed early (e.g. agent not upgradeable, capability check denied, or pre-upgrade callback returned an error).

Specifically, this PR:

• Adds calls to ClearOverrideState() before returning from all early failure branches inside Coordinator.Upgrade.
• Extends the coordinator test suite to assert that overrideState is reset to nil after a failing preUpgradeCallback, preventing stale state from leaking into subsequent upgrade attempts.

Why is it important?

Without this change, failed upgrades could leave the coordinator in a state that incorrectly reflects an ongoing upgrade. This blocks future upgrade attempts until the Elastic Agent is restarted, which is disruptive and operationally undesirable.

By clearing the override state on failure, we ensure the coordinator always returns to a clean state, enabling subsequent upgrade attempts to proceed without requiring a restart.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

Previously, users would need to manually restart the Elastic Agent after a failed upgrade attempt in order to retry an upgrade.
With this fix, the agent automatically clears the override state, removing the need for manual intervention.

How to test this PR locally

Run mage unitTest and confirm that all tests pass, including the updated coordinator tests.

Related issues

• Relates fix: send upgrade action to units before adding it to bkgActions #9634

---

This is an automatic backport of pull request #9992 done by [Mergify](https://mergify.com).

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
66.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 27089ef

Failed CI Steps

• x86_64:sudo: fleet

cc @pkoutsovasilis