Rebrand AbuseCH to abuse.ch and incorporate changes for new data streams

Author: kcreddy

packages/ti_abusech/_dev/build/docs/README.md

@@ -1,8 +1,8 @@
-# AbuseCH Integration for Elastic
+# abuse.ch Integration for Elastic
 
 ## Overview
 
-The AbuseCH integration for Elastic enables collection of logs from [abuse.ch](https://abuse.ch/). This integration facilitates the ingestion of threat intelligence indicators to be used for threat detection and event enrichment.
+The abuse.ch integration for Elastic enables collection of logs from [abuse.ch](https://abuse.ch/). This integration facilitates the ingestion of threat intelligence indicators to be used for threat detection and event enrichment.
 
 ### Compatibility
 This integration is compatible with `v1` version of URLhaus, MalwareBazaar, and ThreatFox APIs.
@@ -15,44 +15,31 @@ This integration periodically queries the abuse.ch APIs to retrieve threat intel
 
 This integration collects threat intelligence indicators into the following datasets:
 
+- `ja3_fingerprints`: Collects JA3 fingerprint based threat indicators identified by SSLBL via [SSLBL API endpoint](https://sslbl.abuse.ch/blacklist/ja3_fingerprints.csv).
 - `malware`: Collects malware payloads from URLs tracked by URLhaus via [URLhaus Bulk API](https://urlhaus-api.abuse.ch/#payloads-recent).
 - `malwarebazaar`: Collects malware payloads from MalwareBazaar via [MalwareBazaar API](https://bazaar.abuse.ch/api/#latest_additions).
+- `sslblacklist`: Collects SSL certificate based threat indicators blacklisted on SSLBL via [SSLBL API endpoint](https://sslbl.abuse.ch/blacklist/sslblacklist.csv).
 - `threatfox`: Collects threat indicators from ThreatFox via [ThreatFox API](https://threatfox.abuse.ch/api/#recent-iocs).
-- `url`: Collects malware URL-based threat indicators from URLhaus via [URLhaus API](https://urlhaus.abuse.ch/api/#csv).
+- `url`: Collects malware URL based threat indicators from URLhaus via [URLhaus API](https://urlhaus.abuse.ch/api/#csv).
 
 ### Supported use cases
 
-Integrating abuse.ch with Elastic enables the following use cases.
-
-- [Prebuilt threat intel detection rules](https://www.elastic.co/docs/reference/security/prebuilt-rules)
-- Real-time threat detection and hunting through [Elastic Security for Threat Intelligence](https://www.elastic.co/security/tip)
-- Real-time dashboards
+The abuse.ch integration brings threat intel into Elastic Security, enabling detection alerts when indicators of compromise (IoCs) like malicious [IPs](https://www.elastic.co/docs/reference/security/prebuilt-rules/rules/threat_intel/threat_intel_indicator_match_address), [domains](https://www.elastic.co/docs/reference/security/prebuilt-rules/rules/threat_intel/threat_intel_indicator_match_url), or [hashes](https://www.elastic.co/docs/reference/security/prebuilt-rules/rules/threat_intel/threat_intel_indicator_match_hash) match your event or alert data. This data can also support threat hunting, enrich alerts with threat context, and power dashboards to track known threats in your environment.
 
 ## What do I need to use this integration?
 
 ### From Elastic
 
-This integration supports both Elastic Agentless-based and Agent-based installations.
-
-#### Agentless-based installation
-
-Agentless integrations allow you to collect data without having to manage Elastic Agent in your cloud. They make manual agent deployment unnecessary, so you can focus on your data instead of the agent that collects it. For more information, refer to [Agentless integrations](https://www.elastic.co/guide/en/serverless/current/security-agentless-integrations.html) and the [Agentless integrations FAQ](https://www.elastic.co/guide/en/serverless/current/agentless-integration-troubleshooting.html).
-
-Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features.
-
-#### Agent-based installation
-
-Elastic Agent must be installed. For more details, check the Elastic Agent [installation instructions](docs-content://reference/fleet/install-elastic-agents.md). You can install only one Elastic Agent per host.
-
 #### Transform
 
 As this integration installs [Elastic latest transforms](https://www.elastic.co/docs/explore-analyze/transforms/transform-overview#latest-transform-overview), the requirements of transform must be met. For more details, check the [Transform Setup](https://www.elastic.co/docs/explore-analyze/transforms/transform-setup)
 
 ### From abuse.ch
 
-abuse.ch requires using an `Auth-Key` (API Key) in the requests for authentication. Requests without authentication will be denied by the abuse.ch APIs.
+abuse.ch requires using an `Auth Key` (API Key) in the requests for authentication. Requests without authentication will be denied by the abuse.ch APIs.
+
+#### Obtain `Auth Key`
 
-#### Obtain `Auth-Key`:
 1. Sign up for new account or login into [abuse.ch authentication portal](https://auth.abuse.ch).
 2. Connect with atleast one authentication provider, namely Google, Github, X, or LinkedIn.
 3. Select **Save profile**.
@@ -63,15 +50,28 @@ For more details, check the abuse.ch [Community First - New Authentication](http
 
 ## How do I deploy this integration?
 
+This integration supports both Elastic Agentless-based and Agent-based installations.
+
+#### Agentless-based installation
+
+Agentless integrations allow you to collect data without having to manage Elastic Agent in your cloud. They make manual agent deployment unnecessary, so you can focus on your data instead of the agent that collects it. For more information, refer to [Agentless integrations](https://www.elastic.co/guide/en/serverless/current/security-agentless-integrations.html) and the [Agentless integrations FAQ](https://www.elastic.co/guide/en/serverless/current/agentless-integration-troubleshooting.html).
+
+Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features.
+
+#### Agent-based installation
+
+Elastic Agent must be installed. For more details, check the Elastic Agent [installation instructions](docs-content://reference/fleet/install-elastic-agents.md). You can install only one Elastic Agent per host.
+
 ### Onboard / configure
 
-1. In Kibana navigate to **Management** > **Integrations**.
-2. In the search bar, type **AbuseCH**.
-3. Select the **AbuseCH** integration from the search results.
-4. Select **Add AbuseCH** to add the integration.
+1. In the top search bar in Kibana, search for **Integrations**.
+2. In the search bar, type **abuse.ch**.
+3. Select the **abuse.ch** integration from the search results.
+4. Select **Add abuse.ch** to add the integration.
 5. Enable and configure only the collection methods which you will use.
 
- * To **Collect AbuseCH logs via API**, you'll need to:
+ * To **Collect abuse.ch logs via API**, you'll need to:
+
  - Configure **Auth Key**.
  - Enable/Disable the required datasets.
  - For each dataset, adjust the integration configuration parameters if required, including the URL, Interval, etc. to enable data collection.
@@ -83,14 +83,14 @@ For more details, check the abuse.ch [Community First - New Authentication](http
 #### Dashboards populated
 
 1. In Kibana, navigate to **Dashboards**.
-2. In the search bar, type **AbuseCH**.
+2. In the search bar, type **abuse.ch**.
 3. Select a dashboard for the dataset you are collecting, and verify the dashboard information is populated.
 
 #### Transforms healthy
 
 1. In Kibana, navigate to **Management** > **Stack Management**.
 2. Under **Data**, select **Transforms**.
-3. In the search bar, type **AbuseCH**.
+3. In the search bar, type **abuse.ch**.
 4. All transforms from the search results should indicate **Healthy** under the **Health** column.
 
 ## Troubleshooting
@@ -113,22 +113,54 @@ For more information on architectures that can be used for scaling this integrat
 
 ### ECS field reference
 
+#### JA3 Fingerprint Blacklist
+
+{{fields "ja3_fingerprints"}}
+
+#### Malware
+
 {{fields "malware"}}
 
+#### MalwareBazaar
+
 {{fields "malwarebazaar"}}
 
+#### SSL Certificate Blacklist
+
+{{fields "sslblacklist"}}
+
+#### ThreatFox
+
 {{fields "threatfox"}}
 
+#### URL
+
 {{fields "url"}}
 
 ### Example event
 
+#### JA3 Fingerprint Blacklist
+
+{{event "ja3_fingerprints"}}
+
+#### Malware
+
 {{event "malware"}}
 
+#### MalwareBazaar
+
 {{event "malwarebazaar"}}
 
+#### SSL Certificate Blacklist
+
+{{event "sslblacklist"}}
+
+#### ThreatFox
+
 {{event "threatfox"}}
 
+#### URL
+
 {{event "url"}}
 
 ### Inputs used
@@ -141,14 +173,16 @@ These inputs can be used in this integration:
 
 This integration datasets uses the following APIs:
 
+- `ja3_fingerprints`: [SSLBL API](https://sslbl.abuse.ch/blacklist/ja3_fingerprints.csv).
 - `malware`: [URLhaus Bulk API](https://urlhaus-api.abuse.ch/#payloads-recent).
 - `malwarebazaar`: [MalwareBazaar API](https://bazaar.abuse.ch/api/#latest_additions).
+- `sslblacklist`: [SSLBL API](https://sslbl.abuse.ch/blacklist/sslblacklist.csv).
 - `threatfox`: [ThreatFox API](https://threatfox.abuse.ch/api/#recent-iocs).
 - `url`: [URLhaus API](https://urlhaus.abuse.ch/api/#csv).
 
 ### Expiration of Indicators of Compromise (IOCs)
 
-All AbuseCH datasets now support indicator expiration. For the `URL` dataset, a full list of active threat indicators are ingested every interval. For other datasets namely `Malware`, `MalwareBazaar`, and `ThreatFox`, the threat indicators are expired after duration `IOC Expiration Duration` configured in the integration setting. An [Elastic Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) is created for every source index to facilitate only active threat indicators be available to the end users. Each transform creates a destination index named `logs-ti_abusech_latest.dest_*` which only contains active and unexpired threat indicators. The indicator match rules and dashboards are updated to list only active threat indicators.
+All abuse.ch datasets now support indicator expiration. For the `URL` dataset, a full list of active threat indicators are ingested every interval. For other datasets namely `Malware`, `MalwareBazaar`, and `ThreatFox`, the threat indicators are expired after duration `IOC Expiration Duration` configured in the integration setting. An [Elastic Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) is created for every source index to facilitate only active threat indicators be available to the end users. Each transform creates a destination index named `logs-ti_abusech_latest.dest_*` which only contains active and unexpired threat indicators. The indicator match rules and dashboards are updated to list only active threat indicators.
 Destinations indices are aliased to `logs-ti_abusech_latest.<data_stream_name>`.
 
 | Source Data stream | Destination Index Pattern | Destination Alias |

packages/ti_abusech/data_stream/ja3_fingerprints/manifest.yml

@@ -1,14 +1,14 @@
 type: logs
-title: AbuseCH JA3 Fingerprint logs
+title: JA3 Fingerprints
 ilm_policy: logs-ti_abusech.ja3_fingerprints-default_policy
 streams:
  - input: cel
  enabled: true
  vars:
  - name: url
  type: text
- title: AbuseCH JA3 Fingerprint API
- description: Active JA3 Fingerprint API fetches malicious JA3 fingerprints identified by SSLBL.
+ title: URL
+ description: Base URL of the abuse.ch SSLBL API to collect active malicious JA3 fingerprints identified by SSLBL.
  multi: false
  required: true
  show_user: false
@@ -35,7 +35,7 @@ streams:
  required: true
  show_user: true
  default: 1h
- description: Interval for polling threat indicators from AbuseCH data dump. As data dump is generated every 5 minutes, it should be greater than 5 minutes. Default `1h`.
+ description: Duration between requests to the SSLBL API. Supported units for this parameter are h/m/s. Example `24h`. As data dump is generated every 5 minutes, it should be greater than 5 minutes.
  - name: ssl
  type: yaml
  title: SSL Configuration
@@ -70,5 +70,5 @@ streams:
  Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details.
 
  template_path: cel.yml.hbs
- title: AbuseCH JA3 Fingerprint logs using Elastic Agent
- description: Collect AbuseCH JA3 Fingerprint logs using Elastic Agent
+ title: JA3 Fingerprints
+ description: Collect malicious JA3 fingerprints from abuse.ch SSLBL.

packages/ti_abusech/data_stream/sslblacklist/manifest.yml

@@ -1,14 +1,14 @@
 type: logs
-title: AbuseCH SSL Blacklist logs
+title: SSL Blacklisted Certificates
 ilm_policy: logs-ti_abusech.sslblacklist-default_policy
 streams:
  - input: cel
  enabled: true
  vars:
  - name: url
  type: text
- title: AbuseCH SSL Blacklist API
- description: Active SSL Blacklist API fetches malicious SSL certificates identified by SSLBL.
+ title: URL
+ description: Base URL of the abuse.ch SSLBL API to collect malicious SSL blacklisted certificates identified by SSLBL.
  multi: false
  required: true
  show_user: false
@@ -35,7 +35,7 @@ streams:
  required: true
  show_user: true
  default: 1h
- description: Interval for polling threat indicators from AbuseCH data dump. As data dump is generated every 5 minutes, it should be greater than 5 minutes. Default `1h`.
+ description: Duration between requests to the SSLBL API. Supported units for this parameter are h/m/s. Example `24h`. As data dump is generated every 5 minutes, it should be greater than 5 minutes.
  - name: ssl
  type: yaml
  title: SSL Configuration
@@ -70,5 +70,5 @@ streams:
  Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details.
 
  template_path: cel.yml.hbs
- title: AbuseCH SSL Blacklist logs using Elastic Agent
- description: Collect AbuseCH SSL Blacklist logs using Elastic Agent
+ title: SSL Blacklisted Certificates
+ description: Collect malicious SSL blacklisted certificates from abuse.ch SSLBL.