Suppress merge-on-recovery for older indices #113462
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
There may be many older indices in need of merging, but today we do not throttle this work across shards so an upgrade could lead to an overwhelming spike in merges. With this commit we make it so that the automatic merge-on-recovery behaviour only applies to newly-created indices.
DaveCTurner added >enhancement :Distributed Indexing/CRUD 
elasticsearchmachine added the Team:Distributed (Obsolete) 
| Pinging @elastic/es-distributed (Team:Distributed) |
| Hi @DaveCTurner, I've created a changelog YAML for you. |
DaveCTurner commented Sep 24, 2024
| | ||
| static { | ||
| final var propertyValue = System.getProperty("es.trigger_merge_after_recovery"); | ||
| final var propertyValue = System.getProperty("es.trigger_merge_after_recovery_8_515_00_0"); |
There was a problem hiding this comment.
Renaming the kill-switch here to nullify the overrides that are in place today and automatically re-enable this feature when upgrading. We can rename it back again later.
DaveCTurner added the auto-merge-without-approval DaveCTurner deleted the 2024/09/24/merge-on-recovery-new-indices-only branch DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this pull request Sep 24, 2024
Most of elastic#113462 only applies to `main`, but for now we must keep the index versions aligned in `8.x`. This commit backports the new `IndexVersion` constant.
elasticsearchmachine pushed a commit that referenced this pull request Sep 24, 2024
Most of #113462 only applies to `main`, but for now we must keep the index versions aligned in `8.x`. This commit backports the new `IndexVersion` constant.
DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this pull request Feb 7, 2025
In elastic#113462 we introduced a version bound on post-recovery merges to avoid a spike of merge activity after an upgrade. At the time there was no `IndexVersion` unique to the 9.x series, but we have such a version now, so this commit properly restricts post-recovery merges to only apply to indices created by versions that support them. Relates ES-9620
elasticsearchmachine pushed a commit that referenced this pull request Feb 18, 2025
In #113462 we introduced a version bound on post-recovery merges to avoid a spike of merge activity after an upgrade. At the time there was no `IndexVersion` unique to the 9.x series, but we have such a version now, so this commit properly restricts post-recovery merges to only apply to indices created by versions that support them. Relates ES-9620
DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this pull request Feb 18, 2025
In elastic#113462 we introduced a version bound on post-recovery merges to avoid a spike of merge activity after an upgrade. At the time there was no `IndexVersion` unique to the 9.x series, but we have such a version now, so this commit properly restricts post-recovery merges to only apply to indices created by versions that support them. Relates ES-9620
elasticsearchmachine pushed a commit that referenced this pull request Feb 18, 2025
In #113462 we introduced a version bound on post-recovery merges to avoid a spike of merge activity after an upgrade. At the time there was no `IndexVersion` unique to the 9.x series, but we have such a version now, so this commit properly restricts post-recovery merges to only apply to indices created by versions that support them. Relates ES-9620
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
There may be many older indices in need of merging, but today we do not
throttle this work across shards so an upgrade could lead to an
overwhelming spike in merges. With this commit we make it so that the
automatic merge-on-recovery behaviour only applies to newly-created
indices.