Skip to content

sei: multiple packages with duplicated field definitions #4398

New issue
New issue
Closed
Closed
sei: multiple packages with duplicated field definitions#4398
Labels
Team:Cloud-MonitoringLabel for the Cloud Monitoring teambugSomething isn't working, use only for issuesintegrationLabel used for meta issues tracking each integration
@efd6

Description

@efd6
efd6
opened on Oct 6, 2022

While preparing packages for the ECS updates I found that duplicated field definitions now caused indeterminate outcomes from elastic-package build. This prevented tests from passing while doing the updates and so I fixed the packages that were failing making use of new elastic-package behaviour for vetting v2 packages for duplicate field definitions. At the time, elastic-package did not continue to vet packages beyond failures in the manifest checks, which meant that it was necessary to fix those before being able to investigate duplications.

elastic-package now proceeds to check duplications even when there are manifest vet failures (here and here), and it has become clear that the packages that I found in the original pass based on test failures did not find all cases (for example google_workspace fixed here).

So here is a complete list of non-deprecated SEI packages that have duplicated field definitions found using the script at the footer of this issue. The script requires elastic-package v0.65.0 and was run on the tree at 28fecbd from the packages directory.

for p in *; do	grep 'elastic/security-external-integrations' ${p}/manifest.yml >/dev/null || continue	grep '^description: Deprecated' ${p}/manifest.yml >/dev/null && continue	gsed -i -e 's/^format_version: 1.0.0/format_version: 2.0.0/' -e '/^license: .*/d' ${p}/manifest.yml;	(	cd $p	m="$(elastic-package build 2>&1 | grep 'defined multiple')"	if [ "$m" != "" ]; then	echo "- [ ] $p"	echo $m \	| gsed -r 's|^ +[0-9]+\. field "(.*)" is defined multiple times for data stream (.*), found in:| \2: `\1`|g' \	| gsed -r 's|/[^ ]+/data_stream/[^ ]+/fields/||g' \	| sort	fi	)	git reset --hard >/dev/null done

Metadata

Metadata

Assignees

No one assigned

    Labels

    Team:Cloud-MonitoringLabel for the Cloud Monitoring teambugSomething isn't working, use only for issuesintegrationLabel used for meta issues tracking each integration

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions