crowdstrike: Return empty events array when no resources in alert, host.

[kcreddy]

Proposed commit message

Return empty events array when no resources in alert and host data-streams.

When there are no resources in first API call, current CEL code returns state.
But this state doesn't have events inside it. As per CEL docs, the events field
is necessary. Without this, the following errors appear and lead to restarting of input.

{"log.level":"error","@timestamp":"2024-08-21T06:26:37.252Z","message":"unexpected missing events array from evaluation","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"cel-default","type":"cel"},"log":{"source":"cel-default"},"input_source":"http://svc-crowdstrike-host:8090","input_url":"http://svc-crowdstrike-host:8090","ecs.version":"1.6.0","log.logger":"input.cel","log.origin":{"file.line":350,"file.name":"cel/input.go","function":"github.com/elastic/beats/v7/x-pack/filebeat/input/cel.input.run.func1"},"service.name":"filebeat","id":"cel-crowdstrike.host-20697e27-0353-4b36-8c17-18f5ea89ae90","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2024-08-21T06:26:37.252Z","message":"unregistering","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"cel-default","type":"cel"},"log":{"source":"cel-default"},"input_type":"cel","key":"cel-crowdstrike_host-20697e27-0353-4b36-8c17-18f5ea89ae90::http://svc-crowdstrike-host:8090","ecs.version":"1.6.0","log.logger":"metric_registry","log.origin":{"file.line":70,"file.name":"inputmon/input.go","function":"github.com/elastic/beats/v7/libbeat/monitoring/inputmon.NewInputRegistry.func1"},"service.name":"filebeat","id":"cel-crowdstrike.host-20697e27-0353-4b36-8c17-18f5ea89ae90::http://svc-crowdstrike-host:8090","uuid":"4ee4a37e-1dde-445d-9f17-2429c435d848","ecs.version":"1.6.0"} {"log.level":"error","@timestamp":"2024-08-21T06:26:37.252Z","message":"Input 'cel' failed with: input.go:130: input cel-crowdstrike.host-20697e27-0353-4b36-8c17-18f5ea89ae90 failed (id=cel-crowdstrike.host-20697e27-0353-4b36-8c17-18f5ea89ae90)\n\tunexpected type returned for evaluation events: <nil>","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"cel-default","type":"cel"},"log":{"source":"cel-default"},"log.logger":"input.cel","log.origin":{"file.line":132,"file.name":"compat/compat.go","function":"github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start.func1"},"service.name":"filebeat","id":"cel-crowdstrike.host-20697e27-0353-4b36-8c17-18f5ea89ae90","ecs.version":"1.6.0","ecs.version":"1.6.0"} 

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

By modifying "resources": [] in the config-alert.yml and config-host.yml:

Before: (example with host)
eval "$(elastic-package stack shellinit)" && elastic-package test system --generate -v --data-streams=host

{"log.level":"error","@timestamp":"2024-08-21T06:26:37.252Z","message":"unexpected missing events array from evaluation","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"cel-default","type":"cel"},"log":{"source":"cel-default"},"input_source":"http://svc-crowdstrike-host:8090","input_url":"http://svc-crowdstrike-host:8090","ecs.version":"1.6.0","log.logger":"input.cel","log.origin":{"file.line":350,"file.name":"cel/input.go","function":"github.com/elastic/beats/v7/x-pack/filebeat/input/cel.input.run.func1"},"service.name":"filebeat","id":"cel-crowdstrike.host-20697e27-0353-4b36-8c17-18f5ea89ae90","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2024-08-21T06:26:37.252Z","message":"unregistering","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"cel-default","type":"cel"},"log":{"source":"cel-default"},"input_type":"cel","key":"cel-crowdstrike_host-20697e27-0353-4b36-8c17-18f5ea89ae90::http://svc-crowdstrike-host:8090","ecs.version":"1.6.0","log.logger":"metric_registry","log.origin":{"file.line":70,"file.name":"inputmon/input.go","function":"github.com/elastic/beats/v7/libbeat/monitoring/inputmon.NewInputRegistry.func1"},"service.name":"filebeat","id":"cel-crowdstrike.host-20697e27-0353-4b36-8c17-18f5ea89ae90::http://svc-crowdstrike-host:8090","uuid":"4ee4a37e-1dde-445d-9f17-2429c435d848","ecs.version":"1.6.0"} {"log.level":"error","@timestamp":"2024-08-21T06:26:37.252Z","message":"Input 'cel' failed with: input.go:130: input cel-crowdstrike.host-20697e27-0353-4b36-8c17-18f5ea89ae90 failed (id=cel-crowdstrike.host-20697e27-0353-4b36-8c17-18f5ea89ae90)\n\tunexpected type returned for evaluation events: <nil>","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"cel-default","type":"cel"},"log":{"source":"cel-default"},"log.logger":"input.cel","log.origin":{"file.line":132,"file.name":"compat/compat.go","function":"github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start.func1"},"service.name":"filebeat","id":"cel-crowdstrike.host-20697e27-0353-4b36-8c17-18f5ea89ae90","ecs.version":"1.6.0","ecs.version":"1.6.0"} 

After:
No error messages in agent logs.

eval "$(elastic-package stack shellinit)" && elastic-package test system --generate -v --data-streams=host,alert

--- Test results for package: crowdstrike - START --- ╭─────────────┬─────────────┬───────────┬───────────┬────────┬───────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├─────────────┼─────────────┼───────────┼───────────┼────────┼───────────────┤ │ crowdstrike │ alert │ system │ common │ PASS │ 37.831208584s │ │ crowdstrike │ host │ system │ common │ PASS │ 36.294177167s │ ╰─────────────┴─────────────┴───────────┴───────────┴────────┴───────────────╯ --- Test results for package: crowdstrike - END --- Done 

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elasticmachine]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[chrisberkhout]

That changes a couple of other things:

• it doesn't return the calculated want_more and offset (maybe not important)
• it doesn't return the error event in the case of a GET failure (important)

I think the change should instead be to insert "events": [], below the ?"resources" line.

Same for both data streams.

[kcreddy]

Nice catch. I updated the CEL code with your suggestion and re-ran the system tests. No error messages in the logs, so its working.

[andrewkroh]

When there are no resources in first API call, current CEL code returns state

Can we mock this in our system test scenario? Like have it return no resources in a response before it returns resources.

[kcreddy]

Can we mock this in our system test scenario? Like have it return no resources in a response before it returns resources.

I was unable to mock the no resources scenario at the beginning, but at the end. i.e., first it returns resources and subsequent calls returns no resources. Commit for this change: eb72037.

The issue I had with making it at the beginning was that I am unable to increment offset because resources.size() is 0. Whatever offset that I start with, I end up with same value, thus unable to move to next page. https://github.com/elastic/integrations/blob/main/packages/crowdstrike/data_stream/alert/agent/stream/cel.yml.hbs#L55-L58

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: eb72037

History

• 💚 Build #14852 succeeded 9d1ee20
• 💚 Build #14820 succeeded 04b5269

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

Package crowdstrike - 1.39.1 containing this change is available at https://epr.elastic.co/search?package=crowdstrike