[CrowdStrike] Add Support of Crowdstrike Event Stream

packages/crowdstrike/_dev/build/docs/README.md

@@ -1,24 +1,28 @@
 # CrowdStrike Integration
 
-The [CrowdStrike](https://www.crowdstrike.com/) Falcon integration allows you to easily connect your CrowdStrike Falcon platform to Elastic for seamless onboarding of alerts and telemetry from CrowdStrike Falcon and Falcon Data Replicator. Elastic Security can leverage this data for security analytics including correlation, visualization and incident response. It provides support using three different modes for integrating CrowdStrike to the Elastic:
+The [CrowdStrike](https://www.crowdstrike.com/) integration allows you to easily connect your CrowdStrike Falcon platform to Elastic for seamless onboarding of alerts and telemetry from CrowdStrike Falcon and Falcon Data Replicator. Elastic Security can leverage this data for security analytics including correlation, visualization and incident response. It provides support using four different modes for integrating CrowdStrike to the Elastic:
 
-1. Falcon SIEM Connector: This is a pre-built integration designed to connect CrowdStrike Falcon with Security Information and Event Management (SIEM) systems. It streamlines the flow of security data from CrowdStrike Falcon to the SIEM, providing a standardized and structured way of feeding information into the SIEM platform. It includes the following datasets for receiving logs:
+1. **Falcon SIEM Connector**: This is a pre-built integration designed to connect CrowdStrike Falcon with Security Information and Event Management (SIEM) systems. It streamlines the flow of security data from CrowdStrike Falcon to the SIEM, providing a standardized and structured way of feeding information into the SIEM platform. It includes the following datasets for receiving logs:
 
 - `falcon` dataset: consists of endpoint data and Falcon platform audit data forwarded from [Falcon SIEM Connector](https://www.crowdstrike.com/blog/tech-center/integrate-with-your-siem/).
 
-2. CrowdStrike REST API: This provides a programmatic interface to interact with the CrowdStrike Falcon platform. It allows users to perform various operations such as querying information about unified alerts and hosts/devices. It includes the following datasets for receiving logs:
+2. **CrowdStrike REST API**: This provides a programmatic interface to interact with the CrowdStrike Falcon platform. It allows users to perform various operations such as querying information about unified alerts and hosts/devices. It includes the following datasets for receiving logs:
 
 - `alert` dataset: It is typically used to retrieve detailed information about unified alerts generated by the CrowdStrike Falcon platform, via Falcon Intelligence Alert API - `/alerts/entities/alerts/v2`.
 
 - `host` dataset: It retrieves all the hosts/devices in your environment providing information such as device metadata, configuration, and status generated by the CrowdStrike Falcon platform, via Falcon Intelligence Host/Device API - `/devices/entities/devices/v2`. It is more focused to provide the management and monitoring information of devices such as login details, status, policies, configuration etc.
 
-3. Falcon Data Replicator: This Collect events in near real time from your endpoints and cloud workloads, identities and data. CrowdStrike Falcon Data Replicator (FDR) enables you with actionable insights to improve SOC performance. FDR contains near real-time data collected by the Falcon platform's single, lightweight agent. It includes the following datasets for receiving logs:
+3. **Falcon Data Replicator**: This Collect events in near real time from your endpoints and cloud workloads, identities and data. CrowdStrike Falcon Data Replicator (FDR) enables you with actionable insights to improve SOC performance. FDR contains near real-time data collected by the Falcon platform's single, lightweight agent. It includes the following datasets for receiving logs:
 
 - `fdr` dataset: consists of logs forwarded using the [Falcon Data Replicator](https://github.com/CrowdStrike/FDR).
 
+4. **CrowdStrike Event Stream**: This streams security logs from CrowdStrike Event Stream, including authentication activity, cloud security posture management (CSPM), firewall logs, user activity, and XDR data. It captures real-time security events like user logins, cloud environment changes, network traffic, and advanced threat detections. The streaming integration provides continuous monitoring and analysis for proactive threat detection. It enhances visibility into user behavior, network security, and overall system health. This setup enables faster response capabilities to emerging security incidents. It includes the following datasets for receiving logs:
+
+- `falcon` dataset: consists of streaming data forwarded from CrowdStrike Event Stream.
+
 ## Compatibility
 
-This integration is compatible with both CrowdStrike Falcon SIEM-Connector-v2.0 and REST API.
+This integration is compatible with CrowdStrike Falcon SIEM-Connector-v2.0, REST API, and CrowdStrike Event Streaming.
 For Rest API support, this module has been tested against the **CrowdStrike API Version v1/v2**.
 
 ## Setup
@@ -35,6 +39,19 @@ For Rest API support, this module has been tested against the **CrowdStrike API
     | Alert         | read:alert    |
     | Host          | read:host     |
 
+### To collect data from CrowdStrike Event Stream, the following parameters from your CrowdStrike instance are required:
+
+1. Client ID
+2. Client Secret
+3. Token URL
+4. API Endpoint URL
+5. CrowdStrike App ID
+6. Required scopes for event stream:
+
+    | Data Stream   | Scope               |
+    | ------------- | ------------------- |
+    | Event Stream  | read: Event streams |
+
 ## Logs
 
 ### Alert

packages/crowdstrike/changelog.yml

@@ -1,4 +1,12 @@
 # newer versions go on top
+- version: "1.46.0"
+  changes:
+    - description: Add Support of CrowdStrike Event Stream.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/11773
+    - description: Update the minimum kibana version to 8.16.0.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/11773
 - version: "1.45.0"
   changes:
     - description: Add support for FQL queries in `alert` and `host` data streams.

packages/crowdstrike/data_stream/falcon/agent/stream/streaming.yml.hbs

@@ -0,0 +1,29 @@
+stream_type: crowdstrike
+url: {{url}}/sensors/entities/datafeed/v2
+auth:
+  client_id: {{client_id}}
+  client_secret: {{client_secret}}
+  token_url: {{token_url}}
+crowdstrike_app_id: {{app_id}}
+redact:
+  fields: ~
+program: |
+  bytes(state.response).decode_json().as(body,{
+    "events": {
+          "message":  body.encode_json(),
+    }
+  })
+tags:
+{{#if preserve_original_event}}
+  - preserve_original_event
+{{/if}}
+{{#each tags as |tag|}}
+  - {{tag}}
+{{/each}}
+{{#contains "forwarded" tags}}
+publisher_pipeline.disable_host: true
+{{/contains}}
+processors:
+{{#if processors}}
+{{processors}}
+{{/if}}

packages/crowdstrike/data_stream/falcon/manifest.yml

@@ -41,3 +41,70 @@ streams:
     template_path: log.yml.hbs
     title: Crowdstrike falcon logs (log)
     description: Collect Crowdstrike falcon logs using log input
+  - input: streaming
+    template_path: streaming.yml.hbs
+    title: CrowdStrike Falcon Logs
+    description: Collect Falcon logs from CrowdStrike Event Stream.
+    enabled: false
+    vars:
+      - name: url
+        type: text
+        title: URL
+        description: Base URL of the CrowdStrike API. Defaults to https://api.crowdstrike.com.
+        default: https://api.crowdstrike.com
+        required: true
+        show_user: true
+      - name: token_url
+        type: text
+        title: Token URL
+        description: Token URL of CrowdStrike.
+        default: https://api.crowdstrike.com/oauth2/token
+        required: true
+        show_user: false
+      - name: client_id
+        type: text
+        title: Client ID
+        description: Client ID for the CrowdStrike.
+        multi: false
+        required: true
+        show_user: true
+      - name: client_secret
+        type: password
+        title: Client Secret
+        description: Client Secret for the CrowdStrike.
+        multi: false
+        required: true
+        show_user: true
+        secret: true
+      - name: app_id
+        type: text
+        title: App ID
+        description: App ID for the CrowdStrike.
+        multi: false
+        required: true
+        show_user: true
+      - name: tags
+        type: text
+        title: Tags
+        multi: true
+        required: true
+        show_user: false
+        default:
+          - forwarded
+          - crowdstrike-falcon
+      - name: preserve_original_event
+        required: true
+        show_user: true
+        title: Preserve original event
+        description: Preserves a raw copy of the original event, added to the field `event.original`.
+        type: bool
+        multi: false
+        default: false
+      - name: processors
+        type: yaml
+        title: Processors
+        multi: false
+        required: false
+        show_user: false
+        description: >
+          Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details.

packages/crowdstrike/docs/README.md

@@ -1,24 +1,28 @@
 # CrowdStrike Integration
 
-The [CrowdStrike](https://www.crowdstrike.com/) Falcon integration allows you to easily connect your CrowdStrike Falcon platform to Elastic for seamless onboarding of alerts and telemetry from CrowdStrike Falcon and Falcon Data Replicator. Elastic Security can leverage this data for security analytics including correlation, visualization and incident response. It provides support using three different modes for integrating CrowdStrike to the Elastic:
+The [CrowdStrike](https://www.crowdstrike.com/) integration allows you to easily connect your CrowdStrike Falcon platform to Elastic for seamless onboarding of alerts and telemetry from CrowdStrike Falcon and Falcon Data Replicator. Elastic Security can leverage this data for security analytics including correlation, visualization and incident response. It provides support using four different modes for integrating CrowdStrike to the Elastic:
 
-1. Falcon SIEM Connector: This is a pre-built integration designed to connect CrowdStrike Falcon with Security Information and Event Management (SIEM) systems. It streamlines the flow of security data from CrowdStrike Falcon to the SIEM, providing a standardized and structured way of feeding information into the SIEM platform. It includes the following datasets for receiving logs:
+1. **Falcon SIEM Connector**: This is a pre-built integration designed to connect CrowdStrike Falcon with Security Information and Event Management (SIEM) systems. It streamlines the flow of security data from CrowdStrike Falcon to the SIEM, providing a standardized and structured way of feeding information into the SIEM platform. It includes the following datasets for receiving logs:
 
 - `falcon` dataset: consists of endpoint data and Falcon platform audit data forwarded from [Falcon SIEM Connector](https://www.crowdstrike.com/blog/tech-center/integrate-with-your-siem/).
 
-2. CrowdStrike REST API: This provides a programmatic interface to interact with the CrowdStrike Falcon platform. It allows users to perform various operations such as querying information about unified alerts and hosts/devices. It includes the following datasets for receiving logs:
+2. **CrowdStrike REST API**: This provides a programmatic interface to interact with the CrowdStrike Falcon platform. It allows users to perform various operations such as querying information about unified alerts and hosts/devices. It includes the following datasets for receiving logs:
 
 - `alert` dataset: It is typically used to retrieve detailed information about unified alerts generated by the CrowdStrike Falcon platform, via Falcon Intelligence Alert API - `/alerts/entities/alerts/v2`.
 
 - `host` dataset: It retrieves all the hosts/devices in your environment providing information such as device metadata, configuration, and status generated by the CrowdStrike Falcon platform, via Falcon Intelligence Host/Device API - `/devices/entities/devices/v2`. It is more focused to provide the management and monitoring information of devices such as login details, status, policies, configuration etc.
 
-3. Falcon Data Replicator: This Collect events in near real time from your endpoints and cloud workloads, identities and data. CrowdStrike Falcon Data Replicator (FDR) enables you with actionable insights to improve SOC performance. FDR contains near real-time data collected by the Falcon platform's single, lightweight agent. It includes the following datasets for receiving logs:
+3. **Falcon Data Replicator**: This Collect events in near real time from your endpoints and cloud workloads, identities and data. CrowdStrike Falcon Data Replicator (FDR) enables you with actionable insights to improve SOC performance. FDR contains near real-time data collected by the Falcon platform's single, lightweight agent. It includes the following datasets for receiving logs:
 
 - `fdr` dataset: consists of logs forwarded using the [Falcon Data Replicator](https://github.com/CrowdStrike/FDR).
 
+4. **CrowdStrike Event Stream**: This streams security logs from CrowdStrike Event Stream, including authentication activity, cloud security posture management (CSPM), firewall logs, user activity, and XDR data. It captures real-time security events like user logins, cloud environment changes, network traffic, and advanced threat detections. The streaming integration provides continuous monitoring and analysis for proactive threat detection. It enhances visibility into user behavior, network security, and overall system health. This setup enables faster response capabilities to emerging security incidents. It includes the following datasets for receiving logs:
+
+- `falcon` dataset: consists of streaming data forwarded from CrowdStrike Event Stream.
+
 ## Compatibility
 
-This integration is compatible with both CrowdStrike Falcon SIEM-Connector-v2.0 and REST API.
+This integration is compatible with CrowdStrike Falcon SIEM-Connector-v2.0, REST API, and CrowdStrike Event Streaming.
 For Rest API support, this module has been tested against the **CrowdStrike API Version v1/v2**.
 
 ## Setup
@@ -35,6 +39,19 @@ For Rest API support, this module has been tested against the **CrowdStrike API
     | Alert         | read:alert    |
     | Host          | read:host     |
 
+### To collect data from CrowdStrike Event Stream, the following parameters from your CrowdStrike instance are required:
+
+1. Client ID
+2. Client Secret
+3. Token URL
+4. API Endpoint URL
+5. CrowdStrike App ID
+6. Required scopes for event stream:
+
+    | Data Stream   | Scope               |
+    | ------------- | ------------------- |
+    | Event Stream  | read: Event streams |
+
 ## Logs
 
 ### Alert

packages/crowdstrike/manifest.yml

@@ -1,13 +1,13 @@
 name: crowdstrike
 title: CrowdStrike
-version: "1.45.0"
+version: "1.46.0"
 description: Collect logs from Crowdstrike with Elastic Agent.
 type: integration
 format_version: "3.0.3"
 categories: [security, edr_xdr]
 conditions:
   kibana:
-    version: "^8.13.0"
+    version: ^8.16.0
 icons:
   - src: /img/logo-integrations-crowdstrike.svg
     title: CrowdStrike
@@ -53,6 +53,9 @@ policy_templates:
       - type: aws-s3
         title: "Collect CrowdStrike Falcon Data Replicator logs (input: aws-s3)"
         description: "Collecting logs from CrowdStrike Falcon Data Replicator (input: aws-s3)"
+      - type: streaming
+        title: Collect CrowdStrike Falcon Logs via Event Stream
+        description: Collecting CrowdStrike Falcon Logs via Event Stream.
       - type: cel
         title: Collect CrowdStrike logs via API
         description: Collecting CrowdStrike logs via API.