elastic · efd6 · Jul 30, 2025 · Apr 7, 2025 · Apr 7, 2025 · May 19, 2025
@@ -4,7 +4,9 @@
 
 [Google Threat Intelligence](https://gtidocs.virustotal.com/) is a security solution that helps organizations detect, analyze, and mitigate threats. It leverages Google's global telemetry, advanced analytics, and vast infrastructure to provide actionable insights. Key features include threat detection, malware and phishing analysis, and real-time threat alerts.
 
-Google Threat Intelligence uses the **[Threat List API](https://gtidocs.virustotal.com/reference/get-hourly-threat-list)** to deliver hourly data chunks. The Threat Lists feature allows customers to consume **Indicators of Compromise (IOCs)** categorized by various threat types.
+Google Threat Intelligence integration offers support for two APIs:
+1. **[Threat List API](https://gtidocs.virustotal.com/reference/get-hourly-threat-list)** to deliver hourly data chunks. The Threat Lists feature allows customers to consume **Indicators of Compromise (IOCs)** categorized by various threat types.
+2. **[IOC Stream API](https://gtidocs.virustotal.com/reference/get-objects-from-the-ioc-stream)** to deliver various types of **Indicators of Compromise (IOCs)** originating from multiple sources. Depending on the source of the notification, different context-specific attributes are added to enrich the IOCs.
 
 ## Threat List API Feeds
 
@@ -35,7 +37,7 @@ Customers can access a subset of the available threat lists based on their **Goo
 
 ## Data Streams
 
-Data collection is available for all threat feed types: `cryptominer`, `first_stage_delivery_vectors`, `infostealer`, `iot`, `linux`, `malicious_network_infrastructure`, `malware`, `mobile`, `osx`, `phishing`, `ransomware`, `threat_actor`, `trending` and `vulnerability_weaponization`, each with a separate data stream. By default, **Ransomware** and **Malicious Network Infrastructure** is enabled. Users can enable additional data streams based on their GTI subscription tier. If a user enables data collection for a data stream they do not have access to, it will result in an error log on the **Discover** page.
+Data collection is available for all threat feeds and IOC Stream, each with a separate data stream. By default, **Ransomware** and **Malicious Network Infrastructure** is enabled. Users can enable additional data streams based on their GTI subscription tier. If a user enables data collection for a data stream they do not have access to, it will result in an error log on the **Discover** page.
 
 ## Requirements
 
@@ -68,7 +70,7 @@ Elastic Agent must be installed. For more details, check the Elastic Agent [inst
  - Access Token
  - Initial Interval
  - Interval
- - (Optional) Query to add custom query filtering on relationship, GTI score, and positives.
+ - (Optional) Query to add custom query filtering on relationship, GTI score, and positives. (not applicable to IOC Stream)
 6. Click on **Save and Continue** to save the integration.
 **Note:** Please make only the threat feed types you have the privilege to access are enabled.
 
@@ -78,7 +80,7 @@ To keep the collected data up to date, **Transforms** are used.
 
 Users can view the transforms by navigating to **Management > Stack Management > Transforms**.
 
-Follow **Steps to enable transforms** to enable transforms and populate `Threat Feed Overview` dashboard.
+Follow **Steps to enable transforms** to enable transforms and populate `Threat Feed Overview` and `IOC Stream Overview` dashboard.
 
 Here, users can see continuously running transforms and also view the latest transformed GTI data in the **Discover** section.
 
@@ -96,6 +98,10 @@ The following are four transforms along with their associated pipelines:
 | URL Transform (ID: `logs-ti_google_threat_intelligence.url_ioc`, Pipeline: `ti_google_threat_intelligence-latest_url_ioc-transform-pipeline`) | Keeps URL entity type data up to date. |
 | Domain Transform (ID: `logs-ti_google_threat_intelligence.domain_ioc`, Pipeline: `ti_google_threat_intelligence-latest_domain_ioc-transform-pipeline`) | Keeps Domain entity type data up to date. |
 | File Transform (ID: `logs-ti_google_threat_intelligence.file_ioc`, Pipeline: `ti_google_threat_intelligence-latest_file_ioc-transform-pipeline`) | Keeps File entity type data up to date. |
+| IP IOC Stream Transform (ID: `logs-ti_google_threat_intelligence.ip_ioc_st`, Pipeline: `ti_google_threat_intelligence-latest_ip_ioc_st-transform-pipeline`) | Keeps IP entity type data up to date for IOC Stream. |
+| URL IOC Stream Transform (ID: `logs-ti_google_threat_intelligence.url_ioc_st`, Pipeline: `ti_google_threat_intelligence-latest_url_ioc_st-transform-pipeline`) | Keeps URL entity type data up to date for IOC Stream. |
+| Domain IOC Stream Transform (ID: `logs-ti_google_threat_intelligence.domain_ioc_st`, Pipeline: `ti_google_threat_intelligence-latest_domain_ioc_st-transform-pipeline`) | Keeps Domain entity type data up to date for IOC Stream. |
+| File IOC Stream Transform (ID: `logs-ti_google_threat_intelligence.file_ioc_st`, Pipeline: `ti_google_threat_intelligence-latest_file_ioc_st-transform-pipeline`) | Keeps File entity type data up to date for IOC Stream. |
 
 For example:
 
@@ -128,22 +134,27 @@ To tailor a rule based on Elastic environment:
 
 Once saved, successfully executed rules will generate alerts. Users can view these alerts in the **Alerts** section.
 
-**Note:** A transform runs in the background to filter relevant data from alerts. The `data_stream.dataset: ti_google_threat_intelligence.enriched_ioc` field represents logs for enriched threat intelligence data, which can be analyzed in the **Discover** section.
+**Note:** Two transforms are available to filter relevant data from alerts. The `data_stream.dataset: ti_google_threat_intelligence.enriched_ioc` and `data_stream.dataset: ti_google_threat_intelligence.enriched_ioc_stream` field represents logs for enriched threat intelligence data, which can be analyzed in the **Discover** section.
 
-The following are the names of the four sample rules:
+The following are the names of the eight sample rules:
 
-| Sample Rule Name | Description |
-| ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
-| Google Threat Intelligence URL IOC Correlation | Detects and alerts on matches between URL IOCs collected by GTI data with user's selected Elastic environment data. |
-| Google Threat Intelligence Domain IOC Correlation | Detects and alerts on matches between Domain IOCs collected by GTI data with user's selected Elastic environment data. |
-| Google Threat Intelligence File IOC Correlation | Detects and alerts on matches between File IOCs collected by GTI data with user's selected Elastic environment data. |
-| Google Threat Intelligence IP Address IOC Correlation | Detects and alerts on matches between IP Address IOCs collected by GTI data with user's selected Elastic environment data. |
+| Sample Rule Name | Description |
+| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
+| Google Threat Intelligence URL IOC Correlation | Detects and alerts on matches between URL IOCs collected by GTI data with user's selected Elastic environment data. |
+| Google Threat Intelligence Domain IOC Correlation | Detects and alerts on matches between Domain IOCs collected by GTI data with user's selected Elastic environment data. |
+| Google Threat Intelligence File IOC Correlation | Detects and alerts on matches between File IOCs collected by GTI data with user's selected Elastic environment data. |
+| Google Threat Intelligence IP Address IOC Correlation | Detects and alerts on matches between IP Address IOCs collected by GTI data with user's selected Elastic environment data. |
+| Google Threat Intelligence URL IOC Stream Correlation | Detects and alerts on matches between URL IOCs collected by GTI IOC Stream data with user's selected Elastic environment data. |
+| Google Threat Intelligence Domain IOC Stream Correlation | Detects and alerts on matches between Domain IOCs collected by GTI IOC Stream data with user's selected Elastic environment data. |
+| Google Threat Intelligence File IOC Stream Correlation | Detects and alerts on matches between File IOCs collected by GTI IOC Stream data with user's selected Elastic environment data. |
+| Google Threat Intelligence IP Address IOC Stream Correlation | Detects and alerts on matches between IP Address IOCs collected by GTI IOC Stream data with user's selected Elastic environment data. |
 
-The following transform and its associated pipelines are used to filter relevant data from alerts. Follow **Steps to enable transforms** to enable these transforms and populate `Threat Intelligence` and `Adversary Intelligence` dashboards.
+The following are two transforms along with their associated pipelines to filter relevant data from alerts. Follow **Steps to enable transforms** to enable these transforms and populate `Threat Intelligence`, `Adversary Intelligence` and `IOC Stream Threat Intelligence` dashboards.
 
-| Transform Name | Description |
-| ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
-| Detected IOC Transform (ID: `logs-ti_google_threat_intelligence.rule`, Pipeline: `ti_google_threat_intelligence-correlation_detection_rule-pipeline`) | Filters and extracts necessary information from Detected IOCs from threat feed. |
+| Transform Name | Description |
+| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
+| Detected IOC Transform (ID: `logs-ti_google_threat_intelligence.rule`, Pipeline: `ti_google_threat_intelligence-correlation_detection_rule-pipeline`) | Filters and extracts necessary information from Detected IOCs from threat feed. |
+| Detected IOC from IOC stream Transform (ID: `logs-ti_google_threat_intelligence.rule_ioc_st`, Pipeline: `ti_google_threat_intelligence-correlation_detection_rule_ioc_st-pipeline`) | Filters and extracts necessary information from Detected IOCs from IOC stream. |
 
 ### Steps to enable transforms
 
@@ -156,7 +167,7 @@ The following transform and its associated pipelines are used to filter relevant
  - Prefix the pipeline name with the integration version.
  For example:
  ```
- {package_version}-ti_google_threat_intelligence-latest_ip_ioc-transform-pipeline
+ {package_version}-ti_google_threat_intelligence-latest_ip_ioc_st-transform-pipeline
  ```
  - Click **Update** to save the changes.
 5. Click the **three dots** again next to the transform and select **Start** to activate it.
@@ -170,10 +181,9 @@ The following transform and its associated pipelines are used to filter relevant
 
 ## Troubleshooting
 
-1. If you encounter a privilege error for a threat feed type, such as: `You are not authorized to perform the requested operation`, verify your privilege level and enable only the threat feeds you have access to.
-2. If you see an error like `Package 2025031310 is not available until 2025-03-13 at 11:00 UTC because of privacy policy.`, ensure that your initial interval and interval are set in hours and are greater than one hour.
-3. If events are not appearing in the transformed index, check if transforms are running without errors. If you encounter issues, refer to [Troubleshooting transforms](https://www.elastic.co/guide/en/elasticsearch/reference/current/transform-troubleshooting.html).
-4. If detection rules take longer to run, ensure you have specified index patterns and applied queries to make your source events more specific.
+1. If you see an error like `Package 2025031310 is not available until 2025-03-13 at 11:00 UTC because of privacy policy.`, ensure that your initial interval and interval are set in hours and are greater than one hour.
+2. If events are not appearing in the transformed index, check if transforms are running without errors. If you encounter issues, refer to [Troubleshooting transforms](https://www.elastic.co/guide/en/elasticsearch/reference/current/transform-troubleshooting.html).
+3. If detection rules take longer to run, ensure you have specified index patterns and applied queries to make your source events more specific.
  **Note:** More events in index patterns mean more time needed for detection rules to run.
 5. Ensure that relevant fields are correctly mapped in the **Indicator Mapping** section. Verify that fields in the specified index pattern are properly mapped, and ensure entity-specific fields (e.g., IP fields to IP fields, keyword fields like file hash SHA256 to corresponding file hash SHA256 fields) are accurately configured.
 6. If any transform is not in a **Healthy** state, try resetting it:
@@ -212,6 +222,16 @@ This is the `Infostealers` dataset.
 
 {{fields "infostealer"}}
 
+### IOC Stream
+
+This is the `IOC Stream` dataset.
+
+#### Example
+
+{{event "ioc_stream"}}
+
+{{fields "ioc_stream"}}
+
 ### Internet of Things
 
 This is the `Internet of Things` dataset.

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: 0.6.0
+ changes:
+ - description: Add data streams - IOC Stream.
+ type: enhancement
+ link: https://github.com/elastic/integrations/pull/13449
 - version: 0.5.0
  changes:
  - description: Add data streams - phishing, ransomware, threat_actor, trending and vulnerability_weaponization.

@@ -33,7 +33,7 @@ program: |
  ?"query": has(state.query) ? optional.of([state.query]) : optional.none(),
  "limit": ["4000"],
  "x-tool": ["Elastic"],
- "User-Agent": ["v0.5.0"], // Keep this in sync with 'version' in package level manifest.yml.
+ "User-Agent": ["v0.6.0"], // Keep this in sync with 'version' in package level manifest.yml.
  }.format_query()
  ).with({
  "Header": {

@@ -1,6 +1,9 @@
 ---
 description: Pipeline for processing Cryptominer events.
 processors:
+ - drop:
+ if: ctx.message == 'retry'
+ tag: drop_retry_events
  - set:
  field: ecs.version
  tag: set_ecs_version
@@ -9,9 +12,6 @@ processors:
  tag: data_collection_error
  if: ctx.error?.message != null && ctx.message == null && ctx.event?.original == null
  description: error message set and no data to process.
- - drop:
- if: ctx.message == 'retry'
- tag: drop_retry_events
  - remove:
  field:
  - organization

@@ -33,7 +33,7 @@ program: |
  ?"query": has(state.query) ? optional.of([state.query]) : optional.none(),
  "limit": ["4000"],
  "x-tool": ["Elastic"],
- "User-Agent": ["v0.5.0"], // Keep this in sync with 'version' in package level manifest.yml.
+ "User-Agent": ["v0.6.0"], // Keep this in sync with 'version' in package level manifest.yml.
  }.format_query()
  ).with({
  "Header": {

@@ -1,6 +1,9 @@
 ---
 description: Pipeline for processing First Stage Delivery Vectors events.
 processors:
+ - drop:
+ if: ctx.message == 'retry'
+ tag: drop_retry_events
  - set:
  field: ecs.version
  tag: set_ecs_version
@@ -9,9 +12,6 @@ processors:
  tag: data_collection_error
  if: ctx.error?.message != null && ctx.message == null && ctx.event?.original == null
  description: error message set and no data to process.
- - drop:
- if: ctx.message == 'retry'
- tag: drop_retry_events
  - remove:
  field:
  - organization

@@ -33,7 +33,7 @@ program: |
  ?"query": has(state.query) ? optional.of([state.query]) : optional.none(),
  "limit": ["4000"],
  "x-tool": ["Elastic"],
- "User-Agent": ["v0.5.0"], // Keep this in sync with 'version' in package level manifest.yml.
+ "User-Agent": ["v0.6.0"], // Keep this in sync with 'version' in package level manifest.yml.
  }.format_query()
  ).with({
  "Header": {

@@ -1,6 +1,9 @@
 ---
 description: Pipeline for processing Infostealer events.
 processors:
+ - drop:
+ if: ctx.message == 'retry'
+ tag: drop_retry_events
  - set:
  field: ecs.version
  tag: set_ecs_version
@@ -9,9 +12,6 @@ processors:
  tag: data_collection_error
  if: ctx.error?.message != null && ctx.message == null && ctx.event?.original == null
  description: error message set and no data to process.
- - drop:
- if: ctx.message == 'retry'
- tag: drop_retry_events
  - remove:
  field:
  - organization

@@ -0,0 +1,3 @@
+fields:
+ tags:
+ - preserve_original_event