Avoid trying to adjust dropped events on test results

[jsoriano]

Pipelines can drop events, and they appear as null values in the test result, avoid trying to convert these null events into common.MapStr, producing errors like:

ERROR: verifying test result failed: comparing test results failed: can't adjust test results: can't unmarshal event: unexpected end of JSON input 

[mtojek]

It smells like a workaround, but in general all runners (test framework) require refactoring. If it doesn't harm other cases, I'm good with this change. Maybe just drop comment on this line, why is it necessary.

[elasticmachine]

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: Branch indexing

• Start Time: 2021-06-24T22:28:27.270+0000

• Duration: 32 min 58 sec

• Commit: 87e58f7

Test stats 🧪

Test	Results
Failed	1
Passed	275
Skipped	2
Total	278

Trends 🧪

Test errors

Expand to view the tests failures

system test: tcp – multiinput.test

• no error details
Expand to view the stacktrace

 could not find hits in logs-multiinput.test-ep data stream 

Steps errors

Expand to view the steps failures

Check

• Took 29 min 24 sec . View more details on here
• Description: make check

Log output

Expand to view the last 100 lines of log output

[2021-06-24T23:00:24.014Z] Done [2021-06-24T23:00:24.014Z] + for d in test/packages/*/ [2021-06-24T23:00:24.014Z] + cd test/packages/kubernetes/ [2021-06-24T23:00:24.014Z] + elastic-package clean -v [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Enable verbose logging [2021-06-24T23:00:24.014Z] Clean used resources [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Clean build resources [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Build directory for integration: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/kubernetes [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Remove directory (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/kubernetes) [2021-06-24T23:00:24.014Z] Build resources removed: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/kubernetes [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Clean built packages from the development stack [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Remove folder (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/stack/development/kubernetes) [2021-06-24T23:00:24.014Z] Package removed from the development stack: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/stack/development/kubernetes [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Clean all service logs [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Remove folder content (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/tmp/service_logs) [2021-06-24T23:00:24.014Z] Temporary service logs removed: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/tmp/service_logs [2021-06-24T23:00:24.014Z] Done [2021-06-24T23:00:24.014Z] + for d in test/packages/*/ [2021-06-24T23:00:24.014Z] + cd test/packages/log/ [2021-06-24T23:00:24.014Z] + elastic-package clean -v [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Enable verbose logging [2021-06-24T23:00:24.014Z] Clean used resources [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Clean build resources [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Build directory for integration: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/log [2021-06-24T23:00:24.014Z] 2021/06/24 23:00:23 DEBUG Remove directory (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/log) [2021-06-24T23:00:24.015Z] Build resources removed: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/log [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Clean built packages from the development stack [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Remove folder (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/stack/development/log) [2021-06-24T23:00:24.015Z] Package removed from the development stack: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/stack/development/log [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Clean all service logs [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Remove folder content (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/tmp/service_logs) [2021-06-24T23:00:24.015Z] Temporary service logs removed: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/tmp/service_logs [2021-06-24T23:00:24.015Z] Done [2021-06-24T23:00:24.015Z] + for d in test/packages/*/ [2021-06-24T23:00:24.015Z] + cd test/packages/multiinput/ [2021-06-24T23:00:24.015Z] + elastic-package clean -v [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Enable verbose logging [2021-06-24T23:00:24.015Z] Clean used resources [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Clean build resources [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Build directory for integration: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/multiinput [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Remove directory (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/multiinput) [2021-06-24T23:00:24.015Z] Build resources removed: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/multiinput [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Clean built packages from the development stack [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Remove folder (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/stack/development/multiinput) [2021-06-24T23:00:24.015Z] Package removed from the development stack: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/stack/development/multiinput [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Clean all service logs [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Remove folder content (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/tmp/service_logs) [2021-06-24T23:00:24.015Z] Temporary service logs removed: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/tmp/service_logs [2021-06-24T23:00:24.015Z] Done [2021-06-24T23:00:24.015Z] + for d in test/packages/*/ [2021-06-24T23:00:24.015Z] + cd test/packages/nginx/ [2021-06-24T23:00:24.015Z] + elastic-package clean -v [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Enable verbose logging [2021-06-24T23:00:24.015Z] Clean used resources [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Clean build resources [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Build directory for integration: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/nginx [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Remove directory (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/nginx) [2021-06-24T23:00:24.015Z] Build resources removed: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/nginx [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Clean built packages from the development stack [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Remove folder (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/stack/development/nginx) [2021-06-24T23:00:24.015Z] Package removed from the development stack: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/stack/development/nginx [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Clean all service logs [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Remove folder content (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/tmp/service_logs) [2021-06-24T23:00:24.015Z] Temporary service logs removed: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/tmp/service_logs [2021-06-24T23:00:24.015Z] Done [2021-06-24T23:00:24.015Z] + for d in test/packages/*/ [2021-06-24T23:00:24.015Z] + cd test/packages/pipeline_tests/ [2021-06-24T23:00:24.015Z] + elastic-package clean -v [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Enable verbose logging [2021-06-24T23:00:24.015Z] Clean used resources [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Clean build resources [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Build directory for integration: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/pipeline_tests [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Remove directory (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/pipeline_tests) [2021-06-24T23:00:24.015Z] Build resources removed: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package/build/integrations/pipeline_tests [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Clean built packages from the development stack [2021-06-24T23:00:24.015Z] 2021/06/24 23:00:23 DEBUG Remove folder (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/stack/development/pipeline_tests) [2021-06-24T23:00:24.016Z] Package removed from the development stack: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/stack/development/pipeline_tests [2021-06-24T23:00:24.016Z] 2021/06/24 23:00:23 DEBUG Clean all service logs [2021-06-24T23:00:24.016Z] 2021/06/24 23:00:23 DEBUG Remove folder content (path: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/tmp/service_logs) [2021-06-24T23:00:24.016Z] Temporary service logs removed: /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/.elastic-package/tmp/service_logs [2021-06-24T23:00:24.016Z] Done [2021-06-24T23:00:24.016Z] + exit 1 [2021-06-24T23:00:24.016Z] Makefile:37: recipe for target 'test-check-packages' failed [2021-06-24T23:00:24.016Z] make: *** [test-check-packages] Error 1 [2021-06-24T23:00:24.060Z] Post stage [2021-06-24T23:00:24.075Z] Running in /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384/src/github.com/elastic/elastic-package [2021-06-24T23:00:24.091Z] Archiving artifacts [2021-06-24T23:00:24.119Z] Archiving artifacts [2021-06-24T23:00:24.146Z] Archiving artifacts [2021-06-24T23:00:24.174Z] Archiving artifacts [2021-06-24T23:00:24.214Z] Recording test results [2021-06-24T23:00:24.535Z] [Checks API] No suitable checks publisher found. [2021-06-24T23:00:24.757Z] Running on Jenkins in /var/lib/jenkins/workspace/t-manager_elastic-package_PR-384 [2021-06-24T23:00:24.789Z] [INFO] getVaultSecret: Getting secrets [2021-06-24T23:00:24.821Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID [2021-06-24T23:00:25.397Z] + chmod 755 generate-build-data.sh [2021-06-24T23:00:25.397Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Ingest-manager/elastic-package/PR-384/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Ingest-manager/elastic-package/PR-384/runs/4 FAILURE 1917865 [2021-06-24T23:00:25.397Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Ingest-manager/elastic-package/PR-384/runs/4/steps/?limit=10000 -o steps-info.json [2021-06-24T23:00:25.397Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Ingest-manager/elastic-package/PR-384/runs/4/tests/?status=FAILED -o tests-errors.json [2021-06-24T23:00:25.397Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Ingest-manager/elastic-package/PR-384/runs/4/log/ -o pipeline-log.txt 

[mtojek]

@jsoriano Do you plan to continue working on this PR or should it be closed?

[jsoriano]

Let's close it and continue droping the events on the pipeline.

[jsoriano]

Oh, I closed this thinking that it was #380 facepalm reopening, this is needed for elastic/integrations#1145.

[jsoriano]

Reopened again as #397