Skip to content

Try out constant_keyword value fields in system log integrations #1211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jun 29, 2021
Merged

Try out constant_keyword value fields in system log integrations #1211

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
9d023aa
try out constant_keyword values in system logs
fearful-symmetry Jun 24, 2021
a020a49
add fields wanted by security
fearful-symmetry Jun 28, 2021
2ddb0d3
update all data streams
fearful-symmetry Jun 28, 2021
0f58d85
Merge remote-tracking branch 'upstream/master' into constant_keyword_…
fearful-symmetry Jun 28, 2021
321e536
add changelog, fix security and docs
fearful-symmetry Jun 28, 2021
dd5bfe1
update version
fearful-symmetry Jun 28, 2021
f2a5911
tinker with data_stream values
fearful-symmetry Jun 28, 2021
00569e1
build again
fearful-symmetry Jun 28, 2021
27d37a1
fix changelog
fearful-symmetry Jun 28, 2021
75b0af5
remove datastream types
fearful-symmetry Jun 29, 2021
0bdaf9c
Merge remote-tracking branch 'upstream/master' into constant_keyword_…
fearful-symmetry Jun 29, 2021
64cac9f
update version
fearful-symmetry Jun 29, 2021
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/system/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "0.13.6"
changes:
- description: Use event.dataset and event.module
type: enhancement
link: https://github.com/elastic/integrations/pull/1211
- version: "0.13.5"
changes:
- description: Add support for Splunk authorization tokens
Expand Down
8 changes: 8 additions & 0 deletions packages/system/data_stream/application/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.application
9 changes: 9 additions & 0 deletions packages/system/data_stream/auth/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
- name: data_stream.type
type: constant_keyword
description: Data stream type.
value: logs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice addition we likely should add anywhere. @mtojek I remember we had some discussion around this in the past, maybe there is even an issue for it. Later on we should make sure it is in sync with the type in the manifest.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you referring to validation only? If so, then I understand that this property will ALWAYS be in sync with the type (any overriding forbidden)?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, can't think of a use case where the two would not be in sync.
- name: data_stream.dataset
type: constant_keyword
description: Data stream dataset.
Expand All @@ -10,3 +11,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.auth
- name: event.module
type: constant_keyword
description: Event module
value: system
8 changes: 8 additions & 0 deletions packages/system/data_stream/core/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.core
8 changes: 8 additions & 0 deletions packages/system/data_stream/cpu/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.cpu
8 changes: 8 additions & 0 deletions packages/system/data_stream/diskio/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.diskio
8 changes: 8 additions & 0 deletions packages/system/data_stream/filesystem/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.filesystem
8 changes: 8 additions & 0 deletions packages/system/data_stream/fsstat/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.fsstat
8 changes: 8 additions & 0 deletions packages/system/data_stream/load/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.load
8 changes: 8 additions & 0 deletions packages/system/data_stream/memory/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.memory
8 changes: 8 additions & 0 deletions packages/system/data_stream/network/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.network
8 changes: 8 additions & 0 deletions packages/system/data_stream/process/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.process
8 changes: 8 additions & 0 deletions packages/system/data_stream/process_summary/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.process.summary
18 changes: 9 additions & 9 deletions packages/system/data_stream/security/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
- name: data_stream.type
type: constant_keyword
description: Data stream type.
value: logs
- name: data_stream.dataset
type: constant_keyword
description: Data stream dataset name.
- name: data_stream.namespace
type: constant_keyword
description: Data stream namespace.
- name: dataset.type
type: constant_keyword
description: Dataset type.
- name: dataset.name
type: constant_keyword
description: Dataset name.
- name: dataset.namespace
type: constant_keyword
description: Dataset namespace.
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.security
- name: tags
description: List of keywords used to tag each event.
example: '["production", "env2"]'
Expand Down
8 changes: 8 additions & 0 deletions packages/system/data_stream/socket_summary/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.socket_summary
9 changes: 9 additions & 0 deletions packages/system/data_stream/syslog/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
- name: data_stream.type
type: constant_keyword
description: Data stream type.
value: logs
- name: data_stream.dataset
type: constant_keyword
description: Data stream dataset.
Expand All @@ -10,3 +11,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.syslog
- name: event.module
type: constant_keyword
description: Event module
value: system
8 changes: 8 additions & 0 deletions packages/system/data_stream/system/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.system
8 changes: 8 additions & 0 deletions packages/system/data_stream/uptime/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@
- name: '@timestamp'
type: date
description: Event timestamp.
- name: event.module
type: constant_keyword
description: Event module
value: system
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: system.uptime
Loading