Skip to content

[system tests] Validate fields are documented with dynamic templates if needed #2207

@mrodm

Description

@mrodm

Follows #2206

In order to try to be agnostic to the structure of the documents ingested to run the validation in system tests. It would be helpful to run validations comparing the mapping definitions instead.

For those mappings that can not be validated against the preview mappings in #2206, it is also needed to validate if they match with any of the dynamic templates found in the data stream.

Mappings and dynamic templates can be retrieved from these APIs:

  • Mappings and dynamic templates installed by Fleet before ingesting any doc.
    • These preview mappings can be retrieved using this simulated API:
      POST /_index_template/_simulate/<index_template_name> # Example POST /_index_template/_simulate/logs-microsoft_dhcp.log 
  • Mappings and dynamic templates that are present after ingesting the docs as part of the system tests.
    • These mappings can be retrieved using this API:
      GET /<data_stream_test>/_mapping/ 

For now, the errors that could be raised as part of this validation should not make fail the validation process, therefore:

  • Enable via environment variable this mapping or show just just warning messages.

To be tested:

  • Run these validations in stack 7.x
  • Run these validations in stack 8.x
  • Run these validations in input and integration packages.
  • Run these validations in Stacks with LogsDB enabled (synthetics).

Metadata

Metadata

Assignees

Labels

Team:EcosystemLabel for the Packages Ecosystem team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions