Invalid query generated when a composite aggregation contains a terms aggregation with an order

[Scylanth]

Java API client version

7.17.0

Java version

17

Elasticsearch Version

7.17.0

Problem description

Composite aggregations with a Terms aggregation source fails when it contains an order.

Code sample
new co.elastic.clients.elasticsearch.core.SearchRequest.Builder() .index(Arrays.asList(getIndexes(getIndexPrefixesFromParameters()))) .size(0) .aggregations( "composite", a -> a.composite( c -> c.sources(Collections.singletonMap("source1", new CompositeAggregationSource.Builder().terms( t -> t.field("test").order(Collections.singletonMap("_key", SortOrder.Asc)) ).build()) )));

Generated query
"aggregations": { "composite": { "composite": { "sources": [{ "source1": { "terms": { "field": "test", "order": [{ "_key": "asc" } ] } } } ] } } }, "size": 0 }

The order field is an array, which contradicts the documentation

It is possible to define the direction of the sort for each value source by setting order to asc (default value) or desc (descending order) directly in the value source definition

Note : The order method requires a map in that case, which works well when doing a terms aggregation, but a terms aggregation as a source in a composite aggregation does not seems to accept the same json structure.

[wilbertpol]

This problem is still present in the latest version of the official client.

It would be nice to have an official client that actually generates valid requests.

[kemusag]

This problem is still occuring in version 8.5.0 of the client. Is there any plan to fix this soon?

[kylelyk]

This is still a problem a year later. If you’re going to deprecate the HighLevelRestClient, you need to have a replacement client that actually works.

[ramyogi7283]

we are facing same issue. we are trying to get some help from forum but no response.
https://discuss.elastic.co/t/elasticsearch-java-api-client-8-7-1-no-option-available-to-generate-the-correct-format-for-source-ordering-for-composition-aggregation/337477

[criself]

This continues to be an issue in 8.12.2. Trying to go around this issue thusly:

String aggJson = """
{"aggregations": {"myCompositeAgg":{"composite":{"size":10,"sources":[{"myTerm":{"terms":{"field":"myTerm","order":"desc"}}}]}}}}""";
new SearchRequest.Builder().withJson(new StringReader(aggJson));

Also results in an exception:

co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch._types.aggregations.TermsAggregation: co.elastic.clients.json.UnexpectedJsonEventException: Unexpected JSON event 'VALUE_STRING' instead of '[START_ARRAY, START_OBJECT]' (JSON path: aggregations.myCompositeAgg.composite.sources[0].myTerm.terms.order) 

[l-trotta]

Hello, sorry for the long wait! This seems to be an issue with the API specification used to produce the Java code. I tested this using version 8.12.2 and it seems to be working, here's how I'm calling the API:

        SearchResponse<Object> compositeagg = esClient.search(s -> s
                .index("test")
                .size(3)
                .query(q -> q
                    .match(m -> m
                        .field("field_name")
                        .query("field_value")))
                .aggregations("cmp", a -> a
                    .composite(c -> c
                        .size(10)
                        .sources(Map.of("example", new CompositeAggregationSource.Builder()
                            .terms(ts -> ts
                                .field("field_name.keyword")
                                .missingBucket(false)
                                .order(SortOrder.Desc))
                            .build()))))
            , Object.class);

The fix still has to be backported to version 7.17, it will be done as soon as we can and then the java client code will be updated to solve this. Thank you for your patience :)

[l-trotta]

fixed with #759 , the issue will be solved in the next release.