Skip to content

Conversation

@eyalkoren
Copy link
Contributor

@eyalkoren eyalkoren commented Apr 25, 2023

A fix for two issues that were introduced with the addition of ignore_missing_component_templates:

  1. A component template is not allowed to be removed even if it is specified as ignore_missing_component_templates. Such templates can be removed, as they are not required for creation of new indices.
  2. A preconfigured composable template cannot be created if any of the component templates it is composed of is missing, even if it is specified as ignore_missing_component_templates. This is in contradiction to manually registering a composable template with missing non-required component templates, as demonstrated in an existing test case.
@eyalkoren eyalkoren added >bug Team:Data Management Meta label for data/management team v8.8.0 labels Apr 25, 2023
@eyalkoren eyalkoren requested review from dakrone and ruflin April 25, 2023 10:11
@eyalkoren eyalkoren self-assigned this Apr 25, 2023
@elasticsearchmachine elasticsearchmachine added external-contributor Pull request authored by a developer outside the Elasticsearch team needs:triage Requires assignment of a team area label and removed Team:Data Management Meta label for data/management team labels Apr 25, 2023
@eyalkoren eyalkoren added the Team:Data Management Meta label for data/management team label Apr 25, 2023
@elasticsearchmachine elasticsearchmachine removed the Team:Data Management Meta label for data/management team label Apr 25, 2023
@eyalkoren eyalkoren added the :Data Management/Data streams Data streams and their lifecycles label Apr 25, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Apr 25, 2023
@elasticsearchmachine
Copy link
Collaborator

Hi @eyalkoren, I've created a changelog YAML for you.

@elasticsearchmachine elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Apr 25, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@ruflin
Copy link
Contributor

ruflin commented Apr 25, 2023

A composable template cannot be created if any of the component templates it is composed of is missing, even if it is specified as ignore_missing_component_templates.

Is this really the case? I thought I had tested this and should also be covered by this test case? https://github.com/elastic/elasticsearch/pull/92436/files#diff-b5daa47f1a90ec68657df8ffb1bc5bd5c65561c66be53edd2088d6f79767d1a6R307 blue is missing there.

@felixbarny felixbarny requested a review from jbaiera April 25, 2023 11:48
Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I left one comment about removing the unnecessary test utility class

Comment on lines 18 to 26
public class TestUtils {
public static <T> T assetNoException(Callable<T> callable, @Nullable String msg) {
try {
return callable.call();
} catch (Throwable th) {
throw new AssertionError(((msg != null) ? msg + "; original error: " : "unexpected error: ") + th.getMessage());
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any exception thrown will fail the test, rather than having a construct explicitly catching exceptions and causing an assertion. We don't need to add this for now (we try to be very conservative with utility classes since we have so many.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are benefits with proper assertions, rather than letting a test just fail. Mainly, the ability to print a more indicative message on what may be the actual regression leading to future failures. In addition, anything that expects assertion failures to be the same would benefit from that. For example, in IntelliJ an assertion failure will have a different icon from any other uncaught exception, which I think is preferable for troubleshooting. This was a very simple attempt to offer a functionality similar to assertDoesNotThrow() that was introduced in JUnit 5.

we try to be very conservative with utility classes since we have so many

No problem, removing

@dakrone dakrone added the v8.7.2 label Apr 25, 2023
@gmarouli gmarouli added v8.9.0 and removed v8.8.0 labels Apr 26, 2023
@felixbarny
Copy link
Member

@gmarouli I'm re-adding the v8.8.0 label so that this bug fix gets backported.

@eyalkoren
Copy link
Contributor Author

eyalkoren commented Apr 27, 2023

Is this really the case? I thought I had tested this and should also be covered by this test case? https://github.com/elastic/elasticsearch/pull/92436/files#diff-b5daa47f1a90ec68657df8ffb1bc5bd5c65561c66be53edd2088d6f79767d1a6R307 blue is missing there.

@ruflin right, I saw this test and should have been more specific- such failures occur when an inherent composable template gets added to the registry through the automatic application of such, based on cluster state. See the testMissingNonRequiredTemplates() test case I added to reproduce that.
The fix is in IndexTeplateRegistry#componentTemplatesExist().

I changed the original PR description accordingly.

Copy link
Member

@felixbarny felixbarny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

@eyalkoren eyalkoren merged commit 37d2da1 into elastic:main Apr 27, 2023
@eyalkoren eyalkoren deleted the fix-missing-template-issues branch April 27, 2023 06:48
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.7 Commit could not be cherrypicked due to conflicts
The backport operation could not be completed due to the following error:
An unhandled error occurred. Please consult the logs 

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 95527

@eyalkoren
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.7

Questions ?

Please refer to the Backport tool documentation

eyalkoren added a commit to eyalkoren/elasticsearch that referenced this pull request Apr 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>bug :Data Management/Data streams Data streams and their lifecycles external-contributor Pull request authored by a developer outside the Elasticsearch team Team:Data Management Meta label for data/management team v8.7.2 v8.8.0 v8.9.0

6 participants