Integration Name

Elasticsearch [elasticsearch]

Dataset Name

n/a

Integration Version

1.19.0

Agent Version

n/a

Agent Output Type

elasticsearch

Elasticsearch Version

8.18.4

OS Version and Architecture

ECH

Software/API Version

No response

Error Message

No response

Event Original

No response

What did you do?

In order to test a critical deprecation fix, I upgraded the Elasticsearch integration from 1.18.1 to 1.19.0

The Critical Deprecation:

GET kbn:/api/upgrade_assistant/es_deprecations 

{ "totalCriticalDeprecations": 1, "migrationsDeprecations": [ { "index": "logs-elasticsearch.index_pivot-template@package", "type": "templates", "details": "Configuring source mode in mappings is deprecated and will be removed in future versions. Use [index.mapping.source.mode] index setting instead.", "message": "Configuring source mode in mappings is deprecated.", "url": "https://ela.st/migrate-source-mode", "isCritical": true, "resolveDuringUpgrade": false } ], "totalCriticalHealthIssues": 0, "enrichedHealthIndicators": [] } 

What did you see?

After upgrading the package from the UI, I noticed the deprecation remained and found it was due to the culprit component template not getting upgraded.

What did you expect to see?

Upgrading a package should update all assets.

Anything else?

I found I had to either delete/reinstall the package, or upgrade using the API with "force":true

Walkthrough...

1. install older version of ES package

   POST kbn:/api/fleet/epm/packages/elasticsearch/1.18.1 { "force": true } 

2. Check a template. logs-elasticsearch.index_pivot-template@packagewill include:

   • "default_pipeline": "1.18.1-monitoring_indices",
   • "_source": {"mode": "synthetic"

   GET _component_template/logs-elasticsearch.index_pivot-template@package 

3. upgrade the package from the integrations UI or API

   POST kbn:/api/fleet/epm/packages/elasticsearch/1.19.0 

4. recheck the component template will include the previous settings indicating it was not updated:

   • "default_pipeline": "1.18.1-monitoring_indices",
   • "_source": {"mode": "synthetic"

   GET _component_template/logs-elasticsearch.index_pivot-template@package 

5. rerun package upgrade with force: true

   POST kbn:/api/fleet/epm/packages/elasticsearch/1.19.0 { "force": true } 

6. recheck the component template. The template is updated:

   • "default_pipeline": "1.19.0-monitoring_indices",
   • deprecated _source mapping is removed

   GET _component_template/logs-elasticsearch.index_pivot-template@package 

I noticed the version for the index_pivot assets did not change from 1.18.1 to 1.19.0. Not sure if that has any bearing in the asset upgrade logic.

 { "id": "logs-elasticsearch.index_pivot-template", "type": "index_template", "version": "0.3.0" }, { "id": "logs-elasticsearch.index_pivot-template@custom", "type": "component_template", "version": "0.3.0" }, { "id": "logs-elasticsearch.index_pivot-template@package", "type": "component_template", "version": "0.3.0" }