Skip to content

Conversation

@kcreddy
Copy link
Contributor

@kcreddy kcreddy commented Aug 1, 2024

Proposed commit message

Fix ECS date mapping for threat fields. ecs@mappings component template is missing threat fields mapped as date. Example: fields such as first_seen, last_seen, modified_at are being mapped as keyword in transform's source datastream-backed indices. The transform's destination indices are not effected as they are not datastream-backed and mappings are explicitly defined as date. This causes field type conflicts. - Explicitly add ECS threat fields that are of type date into source data-stream backed fields. - Ensure fields are correctly mapped using system tests. 

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Before (Mapping Conflict)

Screenshot 2024-08-02 at 9 35 38 AM Screenshot 2024-08-02 at 9 36 03 AM Screenshot 2024-08-02 at 9 36 12 AM

After (No Mapping Conflict)

Screenshot 2024-08-02 at 9 46 58 AM Screenshot 2024-08-02 at 9 47 14 AM
@elasticmachine
Copy link

elasticmachine commented Aug 1, 2024

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@andrewkroh andrewkroh added Integration:ti_anomali Anomali Integration:ti_cif3 Collective Intelligence Framework v3 (Community supported) Integration:ti_crowdstrike CrowdStrike Falcon Intelligence Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Aug 2, 2024
@kcreddy kcreddy self-assigned this Aug 2, 2024
@elasticmachine
Copy link

💚 Build Succeeded

History

cc @kcreddy

@kcreddy kcreddy marked this pull request as ready for review August 2, 2024 07:35
@kcreddy kcreddy requested a review from a team as a code owner August 2, 2024 07:35
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@kcreddy kcreddy added bugfix Pull request that fixes a bug issue Integration:ti_cybersixgill Cybersixgill Integration:ti_eclecticiq EclecticIQ (Partner supported) Integration:ti_maltiverse Maltiverse (Partner supported) labels Aug 2, 2024
@kcreddy kcreddy added Integration:ti_eset ESET Threat Intelligence (Partner supported) Integration:ti_mandiant_advantage Mandiant Advantage (Partner supported) Integration:ti_otx AlienVault OTX Integration:ti_opencti OpenCTI Integration:ti_recordedfuture Recorded Future Integration:ti_rapid7_threat_command Rapid7 Threat Command (Partner supported) Integration:ti_threatq ThreatQuotient (Partner supported) Integration:ti_threatconnect ThreatConnect (Partner supported) labels Aug 2, 2024
@kcreddy kcreddy merged commit 3632d84 into elastic:main Aug 6, 2024
@elasticmachine
Copy link

Package ti_anomali - 1.22.1 containing this change is available at https://epr.elastic.co/search?package=ti_anomali

@elasticmachine
Copy link

Package ti_cif3 - 1.14.1 containing this change is available at https://epr.elastic.co/search?package=ti_cif3

@elasticmachine
Copy link

Package ti_crowdstrike - 1.1.3 containing this change is available at https://epr.elastic.co/search?package=ti_crowdstrike

@elasticmachine
Copy link

Package ti_cybersixgill - 1.30.1 containing this change is available at https://epr.elastic.co/search?package=ti_cybersixgill

@elasticmachine
Copy link

Package ti_eclecticiq - 1.2.1 containing this change is available at https://epr.elastic.co/search?package=ti_eclecticiq

@elasticmachine
Copy link

Package ti_eset - 1.2.2 containing this change is available at https://epr.elastic.co/search?package=ti_eset

@elasticmachine
Copy link

Package ti_maltiverse - 1.2.1 containing this change is available at https://epr.elastic.co/search?package=ti_maltiverse

@elasticmachine
Copy link

Package ti_mandiant_advantage - 1.3.1 containing this change is available at https://epr.elastic.co/search?package=ti_mandiant_advantage

@elasticmachine
Copy link

Package ti_opencti - 2.3.2 containing this change is available at https://epr.elastic.co/search?package=ti_opencti

@elasticmachine
Copy link

Package ti_otx - 1.25.1 containing this change is available at https://epr.elastic.co/search?package=ti_otx

@elasticmachine
Copy link

Package ti_rapid7_threat_command - 2.0.1 containing this change is available at https://epr.elastic.co/search?package=ti_rapid7_threat_command

@elasticmachine
Copy link

Package ti_recordedfuture - 1.26.1 containing this change is available at https://epr.elastic.co/search?package=ti_recordedfuture

@elasticmachine
Copy link

Package ti_threatconnect - 1.2.1 containing this change is available at https://epr.elastic.co/search?package=ti_threatconnect

@elasticmachine
Copy link

Package ti_threatq - 1.28.1 containing this change is available at https://epr.elastic.co/search?package=ti_threatq

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
Fix ECS date mapping for threat fields. ecs@mappings component template is missing threat fields mapped as date. Example: fields such as first_seen, last_seen, modified_at are being mapped as keyword in transform's source datastream-backed indices. The transform's destination indices are not effected as they are not datastream-backed and mappings are explicitly defined as date. This causes field type conflicts. - Explicitly add ECS threat fields that are of type date into source data-stream backed fields. - Ensure fields are correctly mapped using system tests.
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
Fix ECS date mapping for threat fields. ecs@mappings component template is missing threat fields mapped as date. Example: fields such as first_seen, last_seen, modified_at are being mapped as keyword in transform's source datastream-backed indices. The transform's destination indices are not effected as they are not datastream-backed and mappings are explicitly defined as date. This causes field type conflicts. - Explicitly add ECS threat fields that are of type date into source data-stream backed fields. - Ensure fields are correctly mapped using system tests.
@kcreddy kcreddy deleted the ti_fix_ecs_theat_date branch February 7, 2025 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue Integration:ti_anomali Anomali Integration:ti_cif3 Collective Intelligence Framework v3 (Community supported) Integration:ti_crowdstrike CrowdStrike Falcon Intelligence Integration:ti_cybersixgill Cybersixgill Integration:ti_eclecticiq EclecticIQ (Partner supported) Integration:ti_eset ESET Threat Intelligence (Partner supported) Integration:ti_maltiverse Maltiverse (Partner supported) Integration:ti_mandiant_advantage Mandiant Advantage (Partner supported) Integration:ti_opencti OpenCTI Integration:ti_otx AlienVault OTX Integration:ti_rapid7_threat_command Rapid7 Threat Command (Partner supported) Integration:ti_recordedfuture Recorded Future Integration:ti_threatconnect ThreatConnect (Partner supported) Integration:ti_threatq ThreatQuotient (Partner supported) Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

4 participants