Skip to content

Commit 39eea2a

Browse files
authored
[azure_application_insights] [app_state] Add dimensions, metric_type to the app_state data stream (#7550)
* add dimensions and metric types * update docs * bump package version * fix pr id * bump kibana version * remove metrics from package fields * Revert "bump kibana version" This reverts commit 987dd5e. * build package * fix metric types * update docs
1 parent cc7f38a commit 39eea2a

File tree

8 files changed

+224
-164
lines changed

8 files changed

+224
-164
lines changed

packages/azure_application_insights/changelog.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
1+
- version: "1.2.1"
2+
changes:
3+
- description: Add dimension and metric_type mappings to the app_state datastream
4+
type: enhancement
5+
link: https://github.com/elastic/integrations/pull/7550
16
- version: 1.2.0
27
changes:
38
- description: Update the package format_version to 3.0.0.

packages/azure_application_insights/data_stream/app_state/fields/agent.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@
4848
ignore_above: 1024
4949
description: Region in which this host is running.
5050
example: us-east-1
51+
dimension: true
5152
- name: project.id
5253
type: keyword
5354
description: Name of the project in Google Cloud.

packages/azure_application_insights/data_stream/app_state/fields/ecs.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,6 @@
1515
external: ecs
1616
- name: host
1717
external: ecs
18+
- name: agent.id
19+
external: ecs
20+
dimension: true

packages/azure_application_insights/data_stream/app_state/fields/fields.yml

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,91 +16,108 @@
1616
1717
- name: requests_count.sum
1818
type: float
19+
metric_type: gauge
1920
description: >
2021
Request count
2122
2223
- name: requests_failed.sum
2324
type: float
25+
metric_type: gauge
2426
description: >
2527
Request failed count
2628
2729
- name: users_count.unique
2830
type: float
31+
metric_type: gauge
2932
description: >
3033
User count
3134
3235
- name: sessions_count.unique
3336
type: float
37+
metric_type: gauge
3438
description: >
3539
Session count
3640
3741
- name: users_authenticated.unique
3842
type: float
43+
metric_type: gauge
3944
description: >
4045
Authenticated users count
4146
4247
- name: browser_timings_network_duration.avg
4348
type: float
49+
metric_type: gauge
4450
description: >
4551
Browser timings network duration
4652
4753
- name: browser_timings_send_duration.avg
4854
type: float
55+
metric_type: gauge
4956
description: >
5057
Browser timings send duration
5158
5259
- name: browser_timings_receive_duration.avg
5360
type: float
61+
metric_type: gauge
5462
description: >
5563
Browser timings receive duration
5664
5765
- name: browser_timings_processing_duration.avg
5866
type: float
67+
metric_type: gauge
5968
description: >
6069
Browser timings processing duration
6170
6271
- name: browser_timings_total_duration.avg
6372
type: float
73+
metric_type: gauge
6474
description: >
6575
Browser timings total duration
6676
6777
- name: exceptions_count.sum
6878
type: float
79+
metric_type: gauge
6980
description: >
7081
Exception count
7182
7283
- name: exceptions_browser.sum
7384
type: float
85+
metric_type: gauge
7486
description: >
7587
Exception count at browser level
7688
7789
- name: exceptions_server.sum
7890
type: float
91+
metric_type: gauge
7992
description: >
8093
Exception count at server level
8194
8295
- name: performance_counters_memory_available_bytes.avg
8396
type: float
97+
metric_type: gauge
8498
description: >
8599
Performance counters memory available bytes
86100
87101
- name: performance_counters_process_private_bytes.avg
88102
type: float
103+
metric_type: gauge
89104
description: >
90105
Performance counters process private bytes
91106
92107
- name: performance_counters_process_cpu_percentage_total.avg
93108
type: float
109+
metric_type: gauge
94110
description: >
95111
Performance counters process cpu percentage total
96112
97113
- name: performance_counters_process_cpu_percentage.avg
98114
type: float
115+
metric_type: gauge
99116
description: >
100117
Performance counters process cpu percentage
101118
102119
- name: performance_counters_processiobytes_per_second.avg
103120
type: float
121+
metric_type: gauge
104122
description: >
105-
Performance counters process IO bytes per second
106-
123+
Performance counters process IO bytes per second

packages/azure_application_insights/data_stream/app_state/fields/package-fields.yml

Lines changed: 33 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
fields:
55
- name: timegrain
66
type: keyword
7+
dimension: true
78
description: >
89
The Azure metric timegrain
910
@@ -25,6 +26,7 @@
2526
2627
- name: id
2728
type: keyword
29+
dimension: true
2830
description: >
2931
The id of the resource
3032
@@ -40,6 +42,7 @@
4042
4143
- name: namespace
4244
type: keyword
45+
dimension: true
4346
description: >
4447
The namespace selected
4548
@@ -54,14 +57,33 @@
5457
The application ID
5558
5659
- name: dimensions
57-
type: flattened
58-
description: >
59-
Azure metric dimensions.
60-
61-
- name: metrics.*.*
62-
type: object
63-
object_type: float
64-
object_type_mapping_type: "*"
65-
description: >
66-
Metrics returned.
67-
60+
type: group
61+
fields:
62+
- name: cloud_role_instance
63+
type: keyword
64+
dimension: true
65+
description: The unique identifier of the cloud instance where the application is running.
66+
- name: cloud_role_name
67+
type: keyword
68+
dimension: true
69+
description: The name of the role that the cloud instance is performing.
70+
- name: exception_type
71+
type: keyword
72+
dimension: true
73+
description: The type of exception that was thrown.
74+
- name: browser_timing_url_host
75+
type: keyword
76+
dimension: true
77+
description: The host part of the URL that the browser was accessing when timings were captured.
78+
- name: browser_timing_url_path
79+
type: keyword
80+
dimension: true
81+
description: The path part of the URL that the browser was accessing when timings were captured.
82+
- name: request_url_host
83+
type: keyword
84+
dimension: true
85+
description: The host part of the URL that was requested.
86+
- name: request_name
87+
type: keyword
88+
dimension: true
89+
description: The name of the request that was made.

packages/azure_application_insights/docs/README.md

Lines changed: 81 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -147,81 +147,87 @@ An example event for `app_insights` looks as following:
147147

148148
**Exported fields**
149149

150-
| Field | Description | Type |
151-
|---|---|---|
152-
| @timestamp | Event timestamp. | date |
153-
| azure.app_state.browser_timings_network_duration.avg | Browser timings network duration | float |
154-
| azure.app_state.browser_timings_processing_duration.avg | Browser timings processing duration | float |
155-
| azure.app_state.browser_timings_receive_duration.avg | Browser timings receive duration | float |
156-
| azure.app_state.browser_timings_send_duration.avg | Browser timings send duration | float |
157-
| azure.app_state.browser_timings_total_duration.avg | Browser timings total duration | float |
158-
| azure.app_state.end_date | The end date | date |
159-
| azure.app_state.exceptions_browser.sum | Exception count at browser level | float |
160-
| azure.app_state.exceptions_count.sum | Exception count | float |
161-
| azure.app_state.exceptions_server.sum | Exception count at server level | float |
162-
| azure.app_state.performance_counters_memory_available_bytes.avg | Performance counters memory available bytes | float |
163-
| azure.app_state.performance_counters_process_cpu_percentage.avg | Performance counters process cpu percentage | float |
164-
| azure.app_state.performance_counters_process_cpu_percentage_total.avg | Performance counters process cpu percentage total | float |
165-
| azure.app_state.performance_counters_process_private_bytes.avg | Performance counters process private bytes | float |
166-
| azure.app_state.performance_counters_processiobytes_per_second.avg | Performance counters process IO bytes per second | float |
167-
| azure.app_state.requests_count.sum | Request count | float |
168-
| azure.app_state.requests_failed.sum | Request failed count | float |
169-
| azure.app_state.sessions_count.unique | Session count | float |
170-
| azure.app_state.start_date | The start date | date |
171-
| azure.app_state.users_authenticated.unique | Authenticated users count | float |
172-
| azure.app_state.users_count.unique | User count | float |
173-
| azure.application_id | The application ID | keyword |
174-
| azure.dimensions | Azure metric dimensions. | flattened |
175-
| azure.metrics.\*.\* | Metrics returned. | object |
176-
| azure.namespace | The namespace selected | keyword |
177-
| azure.resource.group | The resource group | keyword |
178-
| azure.resource.id | The id of the resource | keyword |
179-
| azure.resource.name | The name of the resource | keyword |
180-
| azure.resource.tags | Azure resource tags. | flattened |
181-
| azure.resource.type | The type of the resource | keyword |
182-
| azure.subscription_id | The subscription ID | keyword |
183-
| azure.timegrain | The Azure metric timegrain | keyword |
184-
| cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword |
185-
| cloud.availability_zone | Availability zone in which this host is running. | keyword |
186-
| cloud.image.id | Image ID for the cloud instance. | keyword |
187-
| cloud.instance.id | Instance ID of the host machine. | keyword |
188-
| cloud.instance.name | Instance name of the host machine. | keyword |
189-
| cloud.machine.type | Machine type of the host machine. | keyword |
190-
| cloud.project.id | Name of the project in Google Cloud. | keyword |
191-
| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword |
192-
| cloud.region | Region in which this host is running. | keyword |
193-
| container.id | Unique container id. | keyword |
194-
| container.image.name | Name of the image the container was built on. | keyword |
195-
| container.labels | Image labels. | object |
196-
| container.name | Container name. | keyword |
197-
| container.runtime | Runtime managing this container. | keyword |
198-
| data_stream.dataset | Data stream dataset name. | constant_keyword |
199-
| data_stream.namespace | Data stream namespace. | constant_keyword |
200-
| data_stream.type | Data stream type. | constant_keyword |
201-
| dataset.name | Dataset name. | constant_keyword |
202-
| dataset.namespace | Dataset namespace. | constant_keyword |
203-
| dataset.type | Dataset type. | constant_keyword |
204-
| ecs.version | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
205-
| host | A host is defined as a general computing instance. ECS host.\* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes. | group |
206-
| host.architecture | Operating system architecture. | keyword |
207-
| host.containerized | If the host is a container. | boolean |
208-
| host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
209-
| host.hostname | Hostname of the host. It normally contains what the `hostname` command returns on the host machine. | keyword |
210-
| host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`. | keyword |
211-
| host.ip | Host ip addresses. | ip |
212-
| host.mac | Host mac addresses. | keyword |
213-
| host.name | Name of the host. It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword |
214-
| host.os.build | OS build information. | keyword |
215-
| host.os.codename | OS codename, if any. | keyword |
216-
| host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword |
217-
| host.os.kernel | Operating system kernel version as a raw string. | keyword |
218-
| host.os.name | Operating system name, without the version. | keyword |
219-
| host.os.name.text | Multi-field of `host.os.name`. | text |
220-
| host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword |
221-
| host.os.version | Operating system version as a raw string. | keyword |
222-
| host.type | Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword |
223-
| service.address | Service address | keyword |
224-
| service.type | The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, `service.type` would be `elasticsearch`. | keyword |
150+
| Field | Description | Type | Metric Type |
151+
|---|---|---|---|
152+
| @timestamp | Event timestamp. | date | |
153+
| agent.id | Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id. | keyword | |
154+
| azure.app_state.browser_timings_network_duration.avg | Browser timings network duration | float | gauge |
155+
| azure.app_state.browser_timings_processing_duration.avg | Browser timings processing duration | float | gauge |
156+
| azure.app_state.browser_timings_receive_duration.avg | Browser timings receive duration | float | gauge |
157+
| azure.app_state.browser_timings_send_duration.avg | Browser timings send duration | float | gauge |
158+
| azure.app_state.browser_timings_total_duration.avg | Browser timings total duration | float | gauge |
159+
| azure.app_state.end_date | The end date | date | |
160+
| azure.app_state.exceptions_browser.sum | Exception count at browser level | float | gauge |
161+
| azure.app_state.exceptions_count.sum | Exception count | float | gauge |
162+
| azure.app_state.exceptions_server.sum | Exception count at server level | float | gauge |
163+
| azure.app_state.performance_counters_memory_available_bytes.avg | Performance counters memory available bytes | float | gauge |
164+
| azure.app_state.performance_counters_process_cpu_percentage.avg | Performance counters process cpu percentage | float | gauge |
165+
| azure.app_state.performance_counters_process_cpu_percentage_total.avg | Performance counters process cpu percentage total | float | gauge |
166+
| azure.app_state.performance_counters_process_private_bytes.avg | Performance counters process private bytes | float | gauge |
167+
| azure.app_state.performance_counters_processiobytes_per_second.avg | Performance counters process IO bytes per second | float | gauge |
168+
| azure.app_state.requests_count.sum | Request count | float | gauge |
169+
| azure.app_state.requests_failed.sum | Request failed count | float | gauge |
170+
| azure.app_state.sessions_count.unique | Session count | float | gauge |
171+
| azure.app_state.start_date | The start date | date | |
172+
| azure.app_state.users_authenticated.unique | Authenticated users count | float | gauge |
173+
| azure.app_state.users_count.unique | User count | float | gauge |
174+
| azure.application_id | The application ID | keyword | |
175+
| azure.dimensions.browser_timing_url_host | The host part of the URL that the browser was accessing when timings were captured. | keyword | |
176+
| azure.dimensions.browser_timing_url_path | The path part of the URL that the browser was accessing when timings were captured. | keyword | |
177+
| azure.dimensions.cloud_role_instance | The unique identifier of the cloud instance where the application is running. | keyword | |
178+
| azure.dimensions.cloud_role_name | The name of the role that the cloud instance is performing. | keyword | |
179+
| azure.dimensions.exception_type | The type of exception that was thrown. | keyword | |
180+
| azure.dimensions.request_name | The name of the request that was made. | keyword | |
181+
| azure.dimensions.request_url_host | The host part of the URL that was requested. | keyword | |
182+
| azure.namespace | The namespace selected | keyword | |
183+
| azure.resource.group | The resource group | keyword | |
184+
| azure.resource.id | The id of the resource | keyword | |
185+
| azure.resource.name | The name of the resource | keyword | |
186+
| azure.resource.tags | Azure resource tags. | flattened | |
187+
| azure.resource.type | The type of the resource | keyword | |
188+
| azure.subscription_id | The subscription ID | keyword | |
189+
| azure.timegrain | The Azure metric timegrain | keyword | |
190+
| cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword | |
191+
| cloud.availability_zone | Availability zone in which this host is running. | keyword | |
192+
| cloud.image.id | Image ID for the cloud instance. | keyword | |
193+
| cloud.instance.id | Instance ID of the host machine. | keyword | |
194+
| cloud.instance.name | Instance name of the host machine. | keyword | |
195+
| cloud.machine.type | Machine type of the host machine. | keyword | |
196+
| cloud.project.id | Name of the project in Google Cloud. | keyword | |
197+
| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword | |
198+
| cloud.region | Region in which this host is running. | keyword | |
199+
| container.id | Unique container id. | keyword | |
200+
| container.image.name | Name of the image the container was built on. | keyword | |
201+
| container.labels | Image labels. | object | |
202+
| container.name | Container name. | keyword | |
203+
| container.runtime | Runtime managing this container. | keyword | |
204+
| data_stream.dataset | Data stream dataset name. | constant_keyword | |
205+
| data_stream.namespace | Data stream namespace. | constant_keyword | |
206+
| data_stream.type | Data stream type. | constant_keyword | |
207+
| dataset.name | Dataset name. | constant_keyword | |
208+
| dataset.namespace | Dataset namespace. | constant_keyword | |
209+
| dataset.type | Dataset type. | constant_keyword | |
210+
| ecs.version | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword | |
211+
| host | A host is defined as a general computing instance. ECS host.\* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes. | group | |
212+
| host.architecture | Operating system architecture. | keyword | |
213+
| host.containerized | If the host is a container. | boolean | |
214+
| host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword | |
215+
| host.hostname | Hostname of the host. It normally contains what the `hostname` command returns on the host machine. | keyword | |
216+
| host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`. | keyword | |
217+
| host.ip | Host ip addresses. | ip | |
218+
| host.mac | Host mac addresses. | keyword | |
219+
| host.name | Name of the host. It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword | |
220+
| host.os.build | OS build information. | keyword | |
221+
| host.os.codename | OS codename, if any. | keyword | |
222+
| host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword | |
223+
| host.os.kernel | Operating system kernel version as a raw string. | keyword | |
224+
| host.os.name | Operating system name, without the version. | keyword | |
225+
| host.os.name.text | Multi-field of `host.os.name`. | text | |
226+
| host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword | |
227+
| host.os.version | Operating system version as a raw string. | keyword | |
228+
| host.type | Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword | |
229+
| service.address | Service address | keyword | |
230+
| service.type | The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, `service.type` would be `elasticsearch`. | keyword | |
225231

226232

227233

0 commit comments

Comments
 (0)