[ML] Review and remove deprecated API usage #117607
Description
Internal ML code should not be reliant upon deprecated APIs:
- Locate the instance where deprecated APIs are being called
- Suggest our outline any mitigation
Procedure
Suites of tests targeting Anomaly Detection, transforms, Dataframe Analytics and Inference were run, which give extensive code coverage. The deprecation logs were then examined directly.
Results
Anomaly Detection
{ "@timestamp": "2024-11-26T22:23:00.715Z", "log.level": "WARN", "data_stream.dataset": "deprecation.elasticsearch", "data_stream.namespace": "default", "data_stream.type": "logs", "elasticsearch.event.category": "indices", "event.code": "dot-prefix", "message": "Index [.monitoring-logstash-8-*] name begins with a dot (.), which is deprecated, and will not be allowed in a future Elasticsearch version.", "ecs.version": "1.2.0", "service.name": "ES_ECS", "event.dataset": "deprecation.elasticsearch", "process.thread.name": "elasticsearch[node-0][generic][T#10]", "log.logger": "org.elasticsearch.deprecation.validation.DotPrefixValidator", "elasticsearch.cluster.uuid": "zQhT3EFyR8-mjAzKndwR8A", "elasticsearch.node.id": "Q0xSJ4nsQpSNzq1DjDqSGQ", "elasticsearch.node.name": "node-0", "elasticsearch.cluster.name": "default-8.18.0-8.x-378" } Transforms
{ "@timestamp": "2024-11-26T21:15:03.172Z", "log.level": "WARN", "data_stream.dataset": "deprecation.elasticsearch", "data_stream.namespace": "default", "data_stream.type": "logs", "elasticsearch.event.category": "indices", "event.code": "dot-prefix", "message": "Index [.monitoring-ent-search-8-*] name begins with a dot (.), which is deprecated, and will not be allowed in a future Elasticsearch version.", "ecs.version": "1.2.0", "service.name": "ES_ECS", "event.dataset": "deprecation.elasticsearch", "process.thread.name": "elasticsearch[node-0][generic][T#9]", "log.logger": "org.elasticsearch.deprecation.validation.DotPrefixValidator", "elasticsearch.cluster.uuid": "m087BeDMQbqK4aC-arpSxQ", "elasticsearch.node.id": "cfdaTPSjT9KkRDIgBdrRow", "elasticsearch.node.name": "node-0", "elasticsearch.cluster.name": "default-8.18.0-8.x-378" } Dataframe Analytics
{ "@timestamp": "2024-11-26T21:31:57.556Z", "log.level": "WARN", "data_stream.dataset": "deprecation.elasticsearch", "data_stream.namespace": "default", "data_stream.type": "logs", "elasticsearch.event.category": "indices", "event.code": "dot-prefix", "message": "Index [.monitoring-ent-search-8-*] name begins with a dot (.), which is deprecated, and will not be allowed in a future Elasticsearch version.", "ecs.version": "1.2.0", "service.name": "ES_ECS", "event.dataset": "deprecation.elasticsearch", "process.thread.name": "elasticsearch[node-0][generic][T#10]", "log.logger": "org.elasticsearch.deprecation.validation.DotPrefixValidator", "elasticsearch.cluster.uuid": "2HtZFHXHRvqjE8NjdmbxcQ", "elasticsearch.node.id": "ZJEKhG7oTpG9yqyH1hCgcQ", "elasticsearch.node.name": "node-0", "elasticsearch.cluster.name": "default-8.18.0-8.x-378" } { "@timestamp": "2024-11-26T21:37:57.209Z", "log.level": "WARN", "data_stream.dataset": "deprecation.elasticsearch", "data_stream.namespace": "default", "data_stream.type": "logs", "elasticsearch.event.category": "api", "event.code": "reindex_sort", "message": "The sort option in reindex is deprecated. Instead consider using query filtering to find the desired subset of data.", "ecs.version": "1.2.0", "service.name": "ES_ECS", "event.dataset": "deprecation.elasticsearch", "process.thread.name": "elasticsearch[node-0][clusterApplierService#updateTask][T#1]", "log.logger": "org.elasticsearch.deprecation.reindex.ReindexValidator", "elasticsearch.cluster.uuid": "2HtZFHXHRvqjE8NjdmbxcQ", "elasticsearch.node.id": "ZJEKhG7oTpG9yqyH1hCgcQ", "elasticsearch.node.name": "node-0", "elasticsearch.cluster.name": "default-8.18.0-8.x-378" } Inference
{ "@timestamp": "2024-11-26T20:47:25.651Z", "log.level": "WARN", "data_stream.dataset": "deprecation.elasticsearch", "data_stream.namespace": "default", "data_stream.type": "logs", "elasticsearch.event.category": "indices", "event.code": "dot-prefix", "message": "Index [.monitoring-beats-8-*] name begins with a dot (.), which is deprecated, and will not be allowed in a future Elasticsearch version.", "ecs.version": "1.2.0", "service.name": "ES_ECS", "event.dataset": "deprecation.elasticsearch", "process.thread.name": "elasticsearch[node-0][generic][T#10]", "log.logger": "org.elasticsearch.deprecation.validation.DotPrefixValidator", "elasticsearch.cluster.uuid": "VeA6W3rSQFO8sPXBBO3s4g", "elasticsearch.node.id": "bnTtdQRSSfaHey6-Ggbljg", "elasticsearch.node.name": "node-0", "elasticsearch.cluster.name": "default-8.18.0-8.x-377" } Summary
| Deprecation warning | Mitigation |
|---|---|
| dot (.) prefixes in index names | out of scope of this exercise and will be addressed elsewhere. |
| sort option in reindex | Remove (potentially unneeded) sort in dataframe reindex step - #117606 |