Skip to content

Conversation

joegallo
Copy link
Contributor

@joegallo joegallo commented Sep 16, 2025

_type isn't special in ingest conditional scripts anymore, and we shouldn't warn about it as if it is.

With this pipeline:

PUT _ingest/pipeline/underscore_type { "processors": [ { "script": { "source": "ctx._foo = ctx._bar" # this is a field } }, { "script": { "source": "ctx._baz = ctx._type" # no warning from this, it's not a _conditional_ } }, { "set": { "if": "ctx._type == null", # this warns on `main` but won't anymore after this PR "field": "message_1", "value": "test" } }, { "set": { "if": "ctx._missing == null", # this is just another field "field": "message_2", "value": "some other test" } } ] } 

This _simulate call:

POST _ingest/pipeline/underscore_type/_simulate { "docs": [ { "_index": "index", "_id": "id", "_source": { "hello": "world", "_bar": "quux" } } ] } 

gives this result on main:

// Warning: 299 Elasticsearch-9.2.0-494dc7075a299cb5edb06c36681601550c4fcdae "[types removal] Looking up doc types [_type] in scripts is deprecated." { "docs" : [ { "doc" : { "_index" : "index", "_version" : "-3", "_id" : "id", "_source" : { "_baz" : null, "message_2" : "some other test", "message_1" : "test", "_bar" : "quux", "hello" : "world", "_foo" : "quux" }, "_ingest" : { "timestamp" : "2025-09-16T21:36:01.143913Z" } } } ] } 

The result would be the same after this PR, there just wouldn't be a warning anymore -- that is, _type is just a regular old non-special field, like _foo or _bar or whatever (edit: that is, at least in terms of reads).

Related to #134816, in that I'm pulling a part of that out as its own PR for searchability and accounting purposes.

from conditional processor scripts
@joegallo joegallo added >enhancement :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP Team:Data Management Meta label for data/management team v9.2.0 labels Sep 16, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@elasticsearchmachine
Copy link
Collaborator

Hi @joegallo, I've created a changelog YAML for you.

@joegallo
Copy link
Contributor Author

@masseyke for the code review (maybe I need more tests?), and @dakrone for the process review (that is, "yes, this is a good change to make").

Copy link
Member

@masseyke masseyke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines -43 to -50
private static final Map<String, Function<Object, Object>> FUNCTIONS = Map.of("_type", value -> {
deprecationLogger.warn(
DeprecationCategory.INDICES,
"conditional-processor__type",
"[types removal] Looking up doc types [_type] in scripts is deprecated."
);
return value;
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason we wouldn't do:

FUNCTIONS = Map.of("_type", ignored -> "_doc");

and just hardcode it to return _doc?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My guess would be because it already does that in Painless?

Copy link
Contributor Author

@joegallo joegallo Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'd be a change in the behavior. The behavior has been (for a while, I could track down the PR that changed it, but I'm 95% sure I didn't write it so I don't know what I'm looking for off the top of my head) that the _type is null (in both the conditional and the ordinary script context):

POST _ingest/pipeline/_simulate { "pipeline": { "processors": [ { "script": { "source": "ctx.type_from_script = ctx._type" } }, { "script": { "if": "ctx._type == null", "source": "ctx.type_from_conditional_was_null = true" } } ] }, "docs": [ { "_index": "index", "_id": "id", "_source": { "hello": "world" } } ] } 

results in:

{ "docs" : [ { "doc" : { "_index" : "index", "_version" : "-3", "_id" : "id", "_source" : { "type_from_script" : null, "type_from_conditional_was_null" : true, "hello" : "world" }, "_ingest" : { "timestamp" : "2025-09-17T11:33:43.056173Z" } } } ] } 

Before and after this PR the above script works the same (that is, _type is null), it's just that prior to this PR a deprecation would be generated only for the conditional part of the one script. All this PR does is remove the deprecation warning, other than that doesn't change the behavior of the scripts in question.

Copy link
Contributor Author

@joegallo joegallo Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't found a PR that clearly is the one that makes the _type be null for these, but I have confirmed that if you run 8.0.0 the behavior is the same as I show above. So this has been the behavior since at least February 2022.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh okay, I hadn't realized it was null, so removing it makes sense. Thanks for the clarification.

@joegallo joegallo requested a review from dakrone September 17, 2025 11:40
Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@joegallo joegallo merged commit 44cbbe3 into elastic:main Sep 17, 2025
34 checks passed
@joegallo joegallo deleted the remove-type-warning-conditional-scripts branch September 17, 2025 15:52
gmjehovich pushed a commit to gmjehovich/elasticsearch that referenced this pull request Sep 18, 2025
phananh1010 added a commit to phananh1010/elasticsearch that referenced this pull request Sep 23, 2025
BASE=f6967fec35dafe147432f1259d7a2b7cf29b1d68 HEAD=ba5d8ae076c5643a926506259ee289d5ab7993db Branch=main
phananh1010 added a commit to phananh1010/elasticsearch that referenced this pull request Oct 2, 2025
BASE=f6967fec35dafe147432f1259d7a2b7cf29b1d68 HEAD=ba5d8ae076c5643a926506259ee289d5ab7993db Branch=main
phananh1010 added a commit to phananh1010/elasticsearch that referenced this pull request Oct 7, 2025
BASE=f6967fec35dafe147432f1259d7a2b7cf29b1d68 HEAD=ba5d8ae076c5643a926506259ee289d5ab7993db Branch=main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement Team:Data Management Meta label for data/management team v9.2.0

4 participants