NestedAggregation in co.elastic.clients:elasticsearch-java

Elastic Stack Elasticsearch
1

Hi,

I'm currently migrating my project to use the new client co.elastic.clients:elasticsearch-java:7.17.22. Previously, with org.elasticsearch:elasticsearch:7.8.1, I created nested aggregations in my JUnit tests like this:

TermsAggregationBuilder expectedTerms = new TermsAggregationBuilder("agg"); expectedTerms.field("data.Nested.NestedInner.NestedInnerField"); expectedTerms.size(1000); NestedAggregationBuilder expectedInnerNested = new NestedAggregationBuilder("nested", "data.Nested.NestedInner"); expectedInnerNested.subAggregation(expectedTerms); NestedAggregationBuilder expectedParentNested = new NestedAggregationBuilder("nested", "data.Nested"); expectedParentNested.subAggregation(expectedInnerNested);

This would return a JSON structure like:

{ "nested": { "nested": { "path": "data.Nested" }, "aggregations": { "nested": { "nested": { "path": "data.Nested.NestedInner" }, "aggregations": { "agg": { "terms": { "field": "data.Nested.NestedInner.NestedInnerField", "size": 1000, "min_doc_count": 1, "shard_min_doc_count": 0, "show_term_doc_count_error": false, "order": [ { "_count": "desc" }, { "_key": "asc" } ] } } } } } } }

Now, with the new client, I tried to create the same nested aggregation using:

String jsonString = "{\"nested\":{\"nested\":{\"path\":\"data.Nested\"},\"aggregations\":{\"nested\":{\"nested\":{\"path\":\"data.Nested.NestedInner\"},\"aggregations\":{\"agg\":{\"terms\":{\"field\":\"data.Nested.NestedInner.NestedInnerField\",\"size\":1000,\"min_doc_count\":1,\"shard_min_doc_count\":0,\"show_term_doc_count_error\":false,\"order\":[{\"_count\":\"desc\"},{\"_key\":\"asc\"}]}}}}}}}"; return new NestedAggregation.Builder() .withJson(stringReader) .build();

or

new Aggregation .Builder() .withJson(stringReader) .build();

However, I received the following error:

co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch._types.aggregations.NestedAggregation: Unknown field 'nested' (JSON path: nested) (line no=1, column no=10, offset=9)

or

Caused by: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch._types.aggregations.NestedAggregation: Unknown field 'nested' (JSON path: nested.nested) (line no=1, column no=20, offset=19)	at co.elastic.clients.json.ObjectDeserializer.parseUnknownField

Can anyone help me understand what I'm doing wrong? How can I properly create a NestedAggregation with sub-aggregations using the new client?

Thanks!

2

Hello and welcome!

The first part of the json structure you sent:

 "nested": { "nested": { "path": "data.Nested" },

isn't part of the aggregation structure, that's why the new client doesn't recognize it. I was trying to understand what it actually is by sending the request to the server using the Kibana dev tools, but it doesn't seem to be a query, could you give me more information on this?

In the meantime, this is how to write a nested aggregation with the 7.17.21 version of the java client:

 esClient.search(s -> s.index("my-index") .aggregations("nested", a -> a .nested(n -> n.path("data.Nested.NestedInner")) .aggregations("agg", ag -> ag .terms(t -> t .field("data.Nested.NestedInner.NestedInnerField") .size(1000) .minDocCount(1) .showTermDocCountError(false) .order(NamedValue.of("_count", SortOrder.Desc),NamedValue.of("_key",SortOrder.Asc))))) , Object.class);

There's one field missing which is shard_min_doc_count, we just recently noticed it was missing from TermsAggregation, we already merged a fix for it which will be available in the 7.17.23 version coming out in a few days.

3

Hi,

We have a junit test to create nestedAggregations + subAggregations:

TermsAggregationBuilder expectedTerms = new TermsAggregationBuilder("agg"); expectedTerms.field("data.Nested.NestedInner.NestedInnerField"); expectedTerms.size(1000); NestedAggregationBuilder expectedInnerNested = new NestedAggregationBuilder("nested", "data.Nested.NestedInner"); expectedInnerNested.subAggregation(expectedTerms); NestedAggregationBuilder expectedParentNested = new NestedAggregationBuilder("nested1", "data.Nested"); expectedParentNested.subAggregation(expectedInnerNested);

expectedParentNested will have

{ "nested":{ "nested":{ "path":"data.Nested" }, "aggregations":{ "nested":{ "nested":{ "path":"data.Nested.NestedInner" }, "aggregations":{ "agg":{ "terms":{ "field":"data.Nested.NestedInner.NestedInnerField", "size":1000, "min_doc_count":1, "shard_min_doc_count":0, "show_term_doc_count_error":false, "order":[ { "_count":"desc" }, { "_key":"asc" } ] } } } } } } }

As far as I understand this test, we wanted to have

"nested":{ "path":"data.Nested", "name": "nested" }

instead of

 "nested":{ "nested":{ "path":"data.Nested" }

In java docs, I found `NestedAggregationBuilder(name, path), name - is name of Aggregation but not a node 'Nested'

A new client:

Aggregation.of(builder -> builder .nested(n -> n.path("data.Nested")) .aggregations("nested", n -> n.nested(s ->s.path("data.Nested.NestedInner").name("nested1"))).aggregations("agg", a -> a.terms(t -> t.field("data.Nested.NestedInner.NestedInnerField").size(100)))); 
nested:{ "path": data.Nested.NestedInner "name: nested1 }

I think it is OK.

4

Hope it helped! If you're still having issues with aggregations here are some other examples of nested/sub aggregations:

1 Like
5

Thank you :wink: