Skip to content

Conversation

@tmgordeeva
Copy link
Contributor

Enables min score on time series aggregation.

Enables min score on time series aggregation.
@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label v8.9.0 labels Jun 15, 2023
@tmgordeeva
Copy link
Contributor Author

@martijnvg might need some more checkstyle cleanup, but should essentially be ready for review.

@tmgordeeva tmgordeeva added Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :StorageEngine/TSDB You know, for Metrics and removed needs:triage Requires assignment of a team area label labels Jun 15, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this.
I left a few comments.

Scorer scorer = weight.scorer(leaf);
if (scorer != null) {
LeafWalker leafWalker = new LeafWalker(leaf, scorer, bucketCollector, () -> tsidOrd[0]);
LeafWalker leafWalker = new LeafWalker(leaf, scorer, minimumScore, bucketCollector, () -> tsidOrd[0]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about wrapping the scorer here with a MinScoreScorer wrapper?
The DocIdSetIterator it produces in the LeafWalker constructor only returns documents that have at least the minimum score.

This way LeafWalker class doesn't need to be changed and there shouldn't be any overhead for users not using the minimum score feature?

this.timestampReverse = TIME_SERIES_SORT[1].getOrder() == SortOrder.DESC;
}

public void setMinimumScore(float minimumScore) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does need to be invoked from AggregationPhase otherwise it minimum score functionality can't be used?

@tmgordeeva
Copy link
Contributor Author

@martijnvg should we ready to go now! I'll be around for a while on slack if there's any last minute changes we need.

@martijnvg
Copy link
Member

should we ready to go now! I'll be around for a while on slack if there's any last minute changes we need.

This looks almost ready. Can you address this comment: https://github.com/elastic/elasticsearch/pull/96878/files#r1234034640

I think we just need to wrap the scorer before we create the LeafWalker in a MinScoreScorer instance.
Then I think the LeafWalker class doesn't need any modifications.

dir.close();
}

static class StaticScoreFunctionBuilder extends ScoreFunctionBuilder<StaticScoreFunctionBuilder> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this inner class can be removed? Because it is no longer used?

@elasticsearchmachine
Copy link
Collaborator

Hi @tmgordeeva, I've created a changelog YAML for you.

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one comment, otherwise LGTM

private final SortedNumericDocValues timestamps; // TODO can we have this just a NumericDocValues?
private final BytesRefBuilder scratch = new BytesRefBuilder();

private final Scorer scorer;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this field is no longer used and can be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I missed this comment - I'll make a tiny cleanup PR later.

@tmgordeeva tmgordeeva merged commit 59c6621 into elastic:main Jun 20, 2023
@tmgordeeva tmgordeeva deleted the feature/tsdb-min-score branch June 20, 2023 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>bug :StorageEngine/TSDB You know, for Metrics Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.9.0

3 participants