Add support for constant_keyword "value" in field definition

[andrewkroh]

For the constant_keyword field type Elasticsearch allows for the value to be specified in the mapping.
If the value is not specified in the mapping then the value is set based on the first document that is
indexed. This adds support for specifying a constant_keyword "value" in a field definition.

Package data streams will be able to explicitly declare a constant_keyword value and will not have to
include the field with every document sent. It can be used optimize storage for an integration since
you could omit the constant_keyword field entirely an event's _source.

Depends on: elastic/package-spec#194 (this requires a spec update)

package-spec update: elastic/package-spec@b358e97

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: Pull request #386 updated

• Start Time: 2021-06-28T14:25:46.559+0000

• Duration: 23 min 8 sec

• Commit: cb873b3

Test stats 🧪

Test	Results
Failed	0
Passed	315
Skipped	2
Total	317

Trends 🧪

[mtojek]

There are more changes required than this as we enabled fields dependency management.

It's related to the use case I mentioned in https://github.com/elastic/package-spec/pull/194/files#r656795658 (import field definition and override some properties). If we want to enable properties overriding. We have to modify the logic calling fdm.ImportFields (see:

elastic-package/internal/fields/dependency_manager.go

Line 175 in 841b4f0

func (dm *DependencyManager) ImportField(schemaName, fieldPath string) (FieldDefinition, error) {

) to not only import a field, but also override properties. I think there are 3 places in the codebase using it (docs, builder, fields validator). The simplest way to find them would adding a sample field definition in some test package (e.g. https://github.com/elastic/elastic-package/blob/master/test/packages/nginx/data_stream/access/fields/ecs.yml#L6).

[andrewkroh]

Good point, I'll look over the places you mention and start adding support for overriding. I can see a use-case for a field like data_stream.dataset where you want to import from ECS, but also add value: aws.vpcflow.

[mtojek]

@andrewkroh Do you plan to finalize this PR or split into multiple chunks? If it's ready for review, please unmark the draft mode.

[andrewkroh]

I will implement the ability to set attributes for external fields in a separate PR. The issue to track this feature is #392.

[mtojek]

Ship it! Please also update the dependency in the elastic/integrations.