Skip to content

[ML] Change the auditor to write via an alias #120064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 20 commits into from
Jan 29, 2025
Merged

[ML] Change the auditor to write via an alias #120064

merged 20 commits into from
Jan 29, 2025
+1,015 −581

Conversation

davidkyle
Copy link
Member

ML and Transforms have an Auditor that write significant operational messages to an index. For v9 upgrades the notifications indices where the messages are stored may need upgrading, the first step of upgrade is to rollover the index which requires a alias to the index.

Previously the auditor wrote to the concrete index directly, this PR switches to using an alias

  • If the notifications index does not exist then create the index and the alias
  • If the index exists then create the alias if it does not exist (this scenario is covered in the upgrade test MlMappingsUpgradeIT)
  • Added a version number comparison when checking that the index templates is installed. If the template changes (as has happened here) the new one needs to be installed
@davidkyle davidkyle added >upgrade :ml Machine learning auto-backport Automatically create backport pull requests when merged v9.0.0 v8.18.0 labels Jan 13, 2025
@elasticsearchmachine elasticsearchmachine added the Team:ML Meta label for the ML team label Jan 13, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)
@elasticsearchmachine
Copy link
Collaborator

Hi @davidkyle, I've created a changelog YAML for you.
@davidkyle davidkyle mentioned this pull request Jan 17, 2025
Merged
przemekwitek
przemekwitek approved these changes Jan 21, 2025
View reviewed changes
Copy link
Contributor

@przemekwitek przemekwitek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@davidkyle davidkyle enabled auto-merge (squash) January 29, 2025 12:06
@davidkyle davidkyle mentioned this pull request Jan 29, 2025
Closed
@davidkyle davidkyle merged commit a52c26a into elastic:main Jan 29, 2025
15 of 16 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
❌ 8.x Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 120064
davidkyle added a commit to davidkyle/elasticsearch that referenced this pull request Jan 29, 2025
@davidkyle
[ML] Change the auditor to write via an alias (elastic#120064)
0120d72 
Changes the ml and transform auditor classes to write through an alias. The alias is required to rollover the index which required for upgrades # Conflicts: #	muted-tests.yml #	x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/MachineLearning.java #	x-pack/qa/rolling-upgrade/src/test/java/org/elasticsearch/upgrades/MlMappingsUpgradeIT.java
@davidkyle davidkyle mentioned this pull request Jan 29, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged backport pending :ml Machine learning Team:ML Meta label for the ML team >upgrade v8.18.0 v9.0.0

3 participants

@davidkyle @elasticsearchmachine @przemekwitek