Skip to content

Add _metric_names_hash field to OTel metric mappings #120952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 18, 2025
Merged

Add _metric_names_hash field to OTel metric mappings #120952

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a4876aa
Add _metric_names_hash field to OTel metric mappings
felixbarny Jan 27, 2025
64f168c
Update docs/changelog/120952.yaml
felixbarny Jan 27, 2025
eda60af
Merge branch 'main' into otel-metric-names-hash
felixbarny Feb 17, 2025
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/changelog/120952.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 120952
summary: Add `_metric_names_hash` field to OTel metric mappings
area: Data streams
type: bug
issues: []
4 changes: 4 additions & 0 deletions x-pack/plugin/otel-data/src/main/resources/component-templates/metrics-otel@mappings.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ template:
type: passthrough
dynamic: true
priority: 10
# workaround for https://github.com/elastic/elasticsearch/issues/99123
_metric_names_hash:
type: keyword
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: will a number be more lightweight, as you're using a 8 digit hex anyway?
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment, numbers can't leverage run-length encoding. So it's actually lighter to use a keyword here as all dimensions are incorporated into the _tsid, which we sort by. Therefore, all values for the same tsid are equal and can be compressed very efficiently.
time_series_dimension: true
unit:
type: keyword
time_series_dimension: true
Expand Down
23 changes: 23 additions & 0 deletions x-pack/plugin/otel-data/src/yamlRestTest/resources/rest-api-spec/test/20_metrics_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -302,3 +302,26 @@ Metrics with different scope names are not duplicates:
metrics:
foo.bar: 42
- is_false: errors

---
Metrics with different metric name hashes are not duplicates:
- do:
bulk:
index: metrics-generic.otel-default
refresh: true
body:
- create: {"dynamic_templates":{"metrics.foo":"counter_long"}}
- "@timestamp": 2024-07-18T14:00:00Z
attributes:
foo: bar
metrics:
foo: 42
_metric_names_hash: a9f37ed7
- create: {"dynamic_templates":{"metrics.bar":"counter_long"}}
- "@timestamp": 2024-07-18T14:00:00Z
attributes:
foo: bar
metrics:
bar: 42
_metric_names_hash: 76b77d1a
- is_false: errors