Adding Jamf Protect

[txhaflaire]

Adding an new integration for Jamf Protect

[cla-checker-service]

💚 CLA has been signed

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[txhaflaire]

CLA is internally reviewed at Jamf

[botelastic]

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[ebeahan]

/test

[kcreddy]

/test

[kcreddy]

CI is now failing from error found character that cannot start any token inside manifest.yml. Could you please change from
github: @elastic/security-service-integrations to github: elastic/security-service-integrations. Notice missing @.

Example: https://github.com/elastic/integrations/blob/main/packages/ti_cif3/manifest.yml#L42-L44

After that, can you run following command elastic-package build && elastic-package format && elastic-package lint && elastic-package check && elastic-package build and fix any resulting errors?

[kcreddy]

/test

[txhaflaire]

@kcreddy done - initially i've merged suggestions by @ebeahan. No errors after lint & format except.

`Error: linting package failed: found 8 validation errors:

1. item [test-jamf_protect-alerts-sample-logs.log] is not allowed in folder [/Users/thijs.xhaflaire/Documents/GitHub/Elastic/integrations/packages/jamf_protect/data_stream/alerts/_dev/test/pipeline]
2. item [test-jamf_protect-alerts-sample-logs.log-expected.json] is not allowed in folder [/Users/thijs.xhaflaire/Documents/GitHub/Elastic/integrations/packages/jamf_protect/data_stream/alerts/_dev/test/pipeline]
3. item [test-jamf_protect-telemetry-sample-logs.log] is not allowed in folder [/Users/thijs.xhaflaire/Documents/GitHub/Elastic/integrations/packages/jamf_protect/data_stream/telemetry/_dev/test/pipeline]
4. item [test-jamf_protect-telemetry-sample-logs.log-expected.json] is not allowed in folder [/Users/thijs.xhaflaire/Documents/GitHub/Elastic/integrations/packages/jamf_protect/data_stream/telemetry/_dev/test/pipeline]
5. item [test-jamf_protect-threat-sample-logs.log] is not allowed in folder [/Users/thijs.xhaflaire/Documents/GitHub/Elastic/integrations/packages/jamf_protect/data_stream/web_threat_events/_dev/test/pipeline]
6. item [test-jamf_protect-threat-sample-logs.log-expected.json] is not allowed in folder [/Users/thijs.xhaflaire/Documents/GitHub/Elastic/integrations/packages/jamf_protect/data_stream/web_threat_events/_dev/test/pipeline]
7. item [test-jamf_protect-traffic-sample-logs.log] is not allowed in folder [/Users/thijs.xhaflaire/Documents/GitHub/Elastic/integrations/packages/jamf_protect/data_stream/web_traffic_events/_dev/test/pipeline]
8. item [test-jamf_protect-traffic-sample-logs.log-expected.json] is not allowed in folder [/Users/thijs.xhaflaire/Documents/GitHub/Elastic/integrations/packages/jamf_protect/data_stream/web_traffic_events/_dev/test/pipeline]`

[kcreddy]

@txhaflaire

`Error: linting package failed: found 8 validation errors:

The issue seems to be coming from naming convention for package test files.
Related issue: elastic/package-spec#185

The convention is test-name-someothername.log
You could fix this issue by renaming jamf_protect in your test file names to jamf-protect

[kcreddy]

/test

[ebeahan]

Could you please change from
github: @elastic/security-service-integrations to github: elastic/security-service-integrations. Notice missing @.

Ah sorry @txhaflaire! And thanks @kcreddy for the catch!

[txhaflaire]

@kcreddy It seems it's failing, however i'm not able to review notes from buildkite - any tips?

[kcreddy]

I was running your tests locally with command eval "$(elastic-package stack shellinit)" && elastic-package test -v.

This is the error I get: [0] parsing field value failed: field "event.dataset"'s value "jamf_protect_telemetry.log" does not match the declared constant_keyword value "jamf_protect.telemetry"

You should just be using name of the package jamf_protect instead of jamf_protect_telemetry for event.module field here. Subsequently, you may need to rename multiple of the fields and ingest pipelines from jamf_protect_telemetry.log to jamf_protect.telemetry

Reference: https://github.com/elastic/integrations/blob/main/packages/jamf_compliance_reporter/data_stream/log/fields/base-fields.yml#L14-L17

[kcreddy]

/test

[kcreddy]

/test

[elasticmachine]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[txhaflaire]

@kcreddy hey!

Anything needed from my side to complete and merge this pr?

[kcreddy]

I reviewed alerts datastream, but most of the review points are applicable to other datastreams as well. If you see the review comment could be related to other datastreams, please change them accordingly as well.

[txhaflaire]

@kcreddy something odd is happening with local pipeline testing for related.ip in

integrations/packages/jamf_protect/data_stream/alerts/elasticsearch/ingest_pipeline/default.yml

Line 415 in 6a66d84

- append:

[2] parsing field value failed: the IP "{0=175.16.199.11}" is not one of the allowed test IPs (see: https://github.com/elastic/elastic-package/blob/main/internal/fields/_static/allowed_geo_ips.txt)

host.ip is an array of IPs, and related.ips also expects an array, but somehow it's giving an error.

[kcreddy]

host.ip is an array of IPs, and related.ips also expects an array, but somehow it's giving an error.

I think issue is with array being appended to another array.
If you have an array already, you can use foreach processor to append each value instead of whole array: https://github.com/elastic/integrations/blob/main/packages/tenable_io/data_stream/vulnerability/elasticsearch/ingest_pipeline/default.yml#L146-L153

[kcreddy]

/test

[kcreddy]

/test

[kcreddy]

@txhaflaire are the pipeline tests successful on your side? Looks like the CI is failing.

[txhaflaire]

@txhaflaire are the pipeline tests successful on your side? Looks like the CI is failing.

No fails, on the geo ip stuff. the suggested fixes are not working in my case and it seems that way of using two processors is not working.

- foreach: field: host.ip if: ctx.host?.ip != null processor: - append: field: related.ip value: '{{ host.ip }}' allow_duplicates: false

[txhaflaire]

@kcreddy

Please also review these errors, and provide suggested fixes. Tried implementing your suggestions but causing errors and before not.

--- Test results for package: jamf_protect - START --- FAILURE DETAILS: jamf_protect/alerts Verify sample_event.json: [0] parsing field value failed: field "event.module"'s value "Alerts" does not match the declared constant_keyword value "jamf_protect" jamf_protect/web_threat_events Verify sample_event.json: [0] parsing field value failed: field "event.module"'s value "Threat Events Stream" does not match the declared constant_keyword value "jamf_protect" jamf_protect/web_traffic_events Verify sample_event.json: [0] parsing field value failed: field "event.module"'s value "Network Traffic Stream" does not match the declared constant_keyword value "jamf_protect"

[kcreddy]

/test

[kcreddy]

/test

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 33573d5

History

• 💔 Build #10214 failed 460c4c8
• 💔 Build #10174 failed b98aad1
• 💔 Build #10169 failed 6a66d84
• 💚 Build #10130 succeeded 8274971
• 💔 Build #9915 failed 190deec

[elastic-sonarqube]

Quality Gate failed

Failed conditions

45.1% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

[kcreddy]

LGTM 👍🏼

[elasticmachine]

Package jamf_protect - 0.1.0 containing this change is available at https://epr.elastic.co/search?package=jamf_protect