[claroty_xdome] Initial release of Claroty xDome #13388

muskan-agarwal26 · 2025-04-02T04:54:36Z

Proposed commit message

Added alert, event, and vulnerability data stream.
Added data collection logic for all the data streams.
Added the ingest pipeline for all the data streams.
Mapped fields according to the ECS schema and added Fields metadata in the appropriate yml files.
Added dashboard and visualizations.
Added test for pipeline for the data streams.
Added system test cases for the data streams.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Clone integrations repo.
Install elastic package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/claroty_xdome directory.
Run the following command to run tests.

elastic-package test

elastic-package test Run asset tests for the package 2025/04/02 10:10:02 INFO License text found in "/root/github/integrations/LICENSE.txt" will be included in package --- Test results for package: claroty_xdome - START --- ╭───────────────┬───────────────┬───────────┬────────────────────────────────────────────────────────────────────────┬────────┬──────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├───────────────┼───────────────┼───────────┼────────────────────────────────────────────────────────────────────────┼────────┼──────────────┤ │ claroty_xdome │ │ asset │ dashboard claroty_xdome-1a6db475-9ab4-4970-9684-9a427321a765 is loaded │ PASS │ 1.757µs │ │ claroty_xdome │ │ asset │ dashboard claroty_xdome-9d8a86b9-6253-4aa6-8f5c-06f4dce86a59 is loaded │ PASS │ 268ns │ │ claroty_xdome │ │ asset │ dashboard claroty_xdome-9dc03d8d-e798-4bad-a368-c21468a5eeea is loaded │ PASS │ 261ns │ │ claroty_xdome │ alert │ asset │ index_template logs-claroty_xdome.alert is loaded │ PASS │ 194ns │ │ claroty_xdome │ alert │ asset │ ingest_pipeline logs-claroty_xdome.alert-0.1.0 is loaded │ PASS │ 237ns │ │ claroty_xdome │ event │ asset │ index_template logs-claroty_xdome.event is loaded │ PASS │ 191ns │ │ claroty_xdome │ event │ asset │ ingest_pipeline logs-claroty_xdome.event-0.1.0 is loaded │ PASS │ 165ns │ │ claroty_xdome │ vulnerability │ asset │ index_template logs-claroty_xdome.vulnerability is loaded │ PASS │ 294ns │ │ claroty_xdome │ vulnerability │ asset │ ingest_pipeline logs-claroty_xdome.vulnerability-0.1.0 is loaded │ PASS │ 243ns │ ╰───────────────┴───────────────┴───────────┴────────────────────────────────────────────────────────────────────────┴────────┴──────────────╯ --- Test results for package: claroty_xdome - END --- Done Run pipeline tests for the package --- Test results for package: claroty_xdome - START --- ╭───────────────┬───────────────┬───────────┬───────────────────────────────────────────────────┬────────┬──────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├───────────────┼───────────────┼───────────┼───────────────────────────────────────────────────┼────────┼──────────────┤ │ claroty_xdome │ alert │ pipeline │ (ingest pipeline warnings test-alert.log) │ PASS │ 692.726885ms │ │ claroty_xdome │ alert │ pipeline │ test-alert.log │ PASS │ 602.7538ms │ │ claroty_xdome │ event │ pipeline │ (ingest pipeline warnings test-event.log) │ PASS │ 570.372204ms │ │ claroty_xdome │ event │ pipeline │ test-event.log │ PASS │ 342.794831ms │ │ claroty_xdome │ vulnerability │ pipeline │ (ingest pipeline warnings test-vulnerability.log) │ PASS │ 527.419216ms │ │ claroty_xdome │ vulnerability │ pipeline │ test-vulnerability.log │ PASS │ 512.233912ms │ ╰───────────────┴───────────────┴───────────┴───────────────────────────────────────────────────┴────────┴──────────────╯ --- Test results for package: claroty_xdome - END --- Done Run policy tests for the package --- Test results for package: claroty_xdome - START --- No test results --- Test results for package: claroty_xdome - END --- Done Run static tests for the package --- Test results for package: claroty_xdome - START --- ╭───────────────┬───────────────┬───────────┬──────────────────────────┬────────┬──────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├───────────────┼───────────────┼───────────┼──────────────────────────┼────────┼──────────────┤ │ claroty_xdome │ alert │ static │ Verify sample_event.json │ PASS │ 226.445235ms │ │ claroty_xdome │ event │ static │ Verify sample_event.json │ PASS │ 194.122196ms │ │ claroty_xdome │ vulnerability │ static │ Verify sample_event.json │ PASS │ 253.400571ms │ ╰───────────────┴───────────────┴───────────┴──────────────────────────┴────────┴──────────────╯ --- Test results for package: claroty_xdome - END --- Done Run system tests for the package 2025/04/02 10:10:14 INFO License text found in "/root/github/integrations/LICENSE.txt" will be included in package 2025/04/02 10:12:40 INFO Write container logs to file: /root/github/integrations/build/container-logs/claroty_xdome-1743568960314953212.log 2025/04/02 10:12:44 INFO Write container logs to file: /root/github/integrations/build/container-logs/elastic-agent-1743568964069045092.log 2025/04/02 10:13:48 INFO Write container logs to file: /root/github/integrations/build/container-logs/claroty_xdome-1743569028341759408.log 2025/04/02 10:13:51 INFO Write container logs to file: /root/github/integrations/build/container-logs/elastic-agent-1743569031046424883.log 2025/04/02 10:14:56 INFO Write container logs to file: /root/github/integrations/build/container-logs/claroty_xdome-1743569096540570617.log 2025/04/02 10:15:00 INFO Write container logs to file: /root/github/integrations/build/container-logs/elastic-agent-1743569100091809715.log --- Test results for package: claroty_xdome - START --- ╭───────────────┬───────────────┬───────────┬───────────┬────────┬─────────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├───────────────┼───────────────┼───────────┼───────────┼────────┼─────────────────┤ │ claroty_xdome │ alert │ system │ default │ PASS │ 2m22.416409042s │ │ claroty_xdome │ event │ system │ default │ PASS │ 47.176099952s │ │ claroty_xdome │ vulnerability │ system │ default │ PASS │ 48.041577103s │ ╰───────────────┴───────────────┴───────────┴───────────┴────────┴─────────────────╯ --- Test results for package: claroty_xdome - END --- Done

Related issues

Closes [New Integration] Claroty XDome #13266

Screenshots

kcreddy · 2025-04-09T05:58:40Z

/test

elasticmachine · 2025-04-09T05:58:56Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

elastic-vault-github-plugin-prod · 2025-04-09T06:17:18Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

jamiehynds · 2025-04-15T09:02:13Z

packages/claroty_xdome/data_stream/event/_dev/test/pipeline/test-event.log-expected.json

+ "1000039"
+ ],
+ "source": {
+ "asset_id": "FZAATCV",


@tinnytintin10 this integration will pull in device/entity data from Claroty. Can you confirm which fields need to be populated in order to populate the asset inventory view? Ideally, we can populate them from the get go, and avoid any rework down the line. (cc @cpascale43)

Thanks for the ping @jamiehynds!

I see that there are vulnerability, alert, and event-related data streams, but not a dedicated asset/entity-centric data stream. @krishna-macharla can you confirm this is indeed the case?

Assuming this is correct, we need to ensure that each of the entities encountered (like hosts/devices) are properly mapped to the relevant ECS fieldsets they correspond to - such as host, user, device, etc.

For any entity that doesn't have an applicable ECS fieldset (ex., object store), you can leverage the generic entity fieldset that we're developing. You can see the details in the PR here. This generic fieldset will allow you to capture metadata for entities that don't fit into existing categories.

This ECS alignment is essential because the current architecture (which is under review) uses pivot transforms to extract and persist entity metadata from properly formatted documents. When the integration's data is correctly mapped to the relevant entity ECS fieldsets, our entity definitions will automatically identify these entities, extract their metadata, and persist them in our entity store. From there, they'll be surfaced in the inventory view.

I have asked engineering to prepare a guide. Once this guide is available, I will be sure to share it with you and the team.

cc @nick-alayil regarding the vuln datastream. Maybe a quick win to get this integration aligned so we don't have to revisit it later?

packages/claroty_xdome/img/claroty_xdome-alert.png

packages/claroty_xdome/img/claroty_xdome-vulnerability.png

packages/claroty_xdome/data_stream/alert/agent/stream/cel.yml.hbs

packages/claroty_xdome/data_stream/alert/sample_event.json

kcreddy · 2025-04-15T13:25:35Z

packages/claroty_xdome/data_stream/alert/_dev/test/pipeline/test-alert.log-expected.json

+ "family": "Autonomous Vehicle",
+ "value": "Autonomous Vehicle"
+ },
+ "uid": "06f488dd-40e9-4bd9-a1d6-7c515bb1a901",


Which value among the uid, id, asset_id is used by their API to query for hosts/devices?

Ideally that field should be used to populate host.id

There is no id field inside the device data, hence mapping the uid with the host.id.

Need API doc to review this.
Whatever field Claroty xDome is using inside their hosts API (uid/id/asset_id), that field needs to be extracted into host.id. It will be useful when performing actions on hosts.

#13388 (comment)

packages/claroty_xdome/_dev/build/docs/README.md

...roty_xdome/data_stream/vulnerability/_dev/test/pipeline/test-vulnerability.log-expected.json

packages/claroty_xdome/img/claroty_xdome-vulnerability.png

packages/claroty_xdome/manifest.yml

1. Added saved searches in alert and vulnerability datastream. 2. Add fields into related.user 3. Mapped required vulnerability fields. 4. Added category vulnerability_management.

kcreddy · 2025-04-20T11:22:30Z

/test

...roty_xdome/data_stream/vulnerability/_dev/test/pipeline/test-vulnerability.log-expected.json

packages/claroty_xdome/img/claroty_xdome-vulnerability.png

1. Mapped resource.id with host.id, resource.name to host.name

kcreddy · 2025-04-21T11:51:22Z

/test

kcreddy

LGTM.
Please wait for @jamiehynds comment to be addressed before merging.

elastic-sonarqube · 2025-04-21T12:14:54Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
98.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

elasticmachine · 2025-04-21T12:14:58Z

💚 Build Succeeded

Buildkite Build
Commit: 0c5c805

History

💚 Build #24912 succeeded 0c56e84
💚 Build #24579 succeeded aa6ce86

cc @piyush-elastic

jamiehynds · 2025-05-09T10:48:46Z

LGTM. Please wait for @jamiehynds comment to be addressed before merging.

Thanks @kcreddy. Confirmed with @piyush-elastic that the necessary ECS entity mappings are in place. As that fieldset evolves, there may be a need to add more mappings down the line. But for now, I'm fine with going ahead with the merge for this integration.

elastic-vault-github-plugin-prod · 2025-05-09T11:09:35Z

Package claroty_xdome - 0.1.0 containing this change is available at https://epr.elastic.co/package/claroty_xdome/0.1.0/

muskan-crest added 3 commits April 2, 2025 10:16

Initial release of Claroty xDome

f4a65d1

Update changelog entry

3f55bb1

Added entry in CODEOWNERS

aa6ce86

andrewkroh added New Integration Issue or pull request for creating a new integration package. dashboard Relates to a Kibana dashboard bug, enhancement, or modification. Crest Contributions from Crest developement team. labels Apr 2, 2025

kcreddy added the Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] label Apr 9, 2025

piyush-elastic self-assigned this Apr 15, 2025

kcreddy added Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] and removed Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Apr 15, 2025

kcreddy requested a review from a team April 15, 2025 08:58

jamiehynds reviewed Apr 15, 2025

View reviewed changes

kcreddy reviewed Apr 15, 2025

View reviewed changes

packages/claroty_xdome/manifest.yml Show resolved Hide resolved

Resolved comments by @kcreddy.

0c56e84

1. Added saved searches in alert and vulnerability datastream. 2. Add fields into related.user 3. Mapped required vulnerability fields. 4. Added category vulnerability_management.

muskan-agarwal26 requested a review from a team as a code owner April 17, 2025 09:43

muskan-agarwal26 requested a review from kcreddy April 17, 2025 09:43

kcreddy reviewed Apr 21, 2025

View reviewed changes

Resolved comments by @kcreddy.

0c5c805

1. Mapped resource.id with host.id, resource.name to host.name

muskan-agarwal26 requested a review from kcreddy April 21, 2025 11:36

kcreddy approved these changes Apr 21, 2025

View reviewed changes

kcreddy merged commit b6a44bd into elastic:main May 9, 2025
7 checks passed

andrewkroh added the Integration:claroty_xdome Claroty xDome label May 9, 2025

na0F mentioned this pull request May 27, 2025

[claroty_xdome]: the official release schedule for Claroty xDome support #14007

Closed

[claroty_xdome] Initial release of Claroty xDome #13388

[claroty_xdome] Initial release of Claroty xDome #13388

Uh oh!

Conversation

muskan-agarwal26 commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

How to test this PR locally

Related issues

Screenshots

kcreddy commented Apr 9, 2025

elasticmachine commented Apr 9, 2025

elastic-vault-github-plugin-prod bot commented Apr 9, 2025

🚀 Benchmarks report

jamiehynds Apr 15, 2025

Choose a reason for hiding this comment

tinnytintin10 Apr 23, 2025

Choose a reason for hiding this comment

tinnytintin10 Apr 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kcreddy Apr 15, 2025

Choose a reason for hiding this comment

kcreddy Apr 15, 2025

Choose a reason for hiding this comment

muskan-agarwal26 Apr 16, 2025

Choose a reason for hiding this comment

kcreddy Apr 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

kcreddy Apr 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kcreddy commented Apr 20, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kcreddy commented Apr 21, 2025

kcreddy left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

elastic-sonarqube bot commented Apr 21, 2025

Quality Gate passed

elasticmachine commented Apr 21, 2025

💚 Build Succeeded

History

jamiehynds commented May 9, 2025

Uh oh!

elastic-vault-github-plugin-prod bot commented May 9, 2025

Labels

8 participants

muskan-agarwal26 commented Apr 2, 2025 •

edited

Loading

kcreddy Apr 21, 2025 •

edited

Loading

kcreddy left a comment •

edited

Loading