SOC - Alert Ingestion (Cron) - Template Playbook Overview
This document provides detailed information about the SOC - Alert Ingestion (Cron) - Template playbook, which leverages scheduled tasks to pull alerts from external systems, process them, and handle actions such as enrichment, correlation, and case creation. Users need to configure only the following components:
- Placeholder - Create TEDS Alert List
- Custom Alert Data Extension (Cron)
- Correlate (Cron)
Note: These playbooks can and should be duplicated and the Placeholder - Create TEDS Alert List component swapped out to match the technology stack in your organization. This ensures the playbook works seamlessly with your chosen vendor's tools.
Overview
- Objective: Automate scheduled alert ingestion using connectors, assets, and Turbine logic.
- Key Workflow Steps:
- Alerts are ingested on a cron schedule, fetching data from external systems.
- Fetched alerts are standardized into TEDS objects.
- Each alert undergoes deduplication, enrichment, and correlation.
- Enriched and correlated alerts are used to create cases for investigation.
Accessing the Playbook
To access the playbook:
- Navigate to Orchestration in the Swimlane platform.
- Click on Playbooks.
- Select SOC - Alert Ingestion (Cron) - Template.
Loop-Based Alert Processing in the Playbook
The playbook processes alerts in a loop, ensuring each alert is uniquely processed. Here's how it works:
- Alert Ingestion: A cron-based schedule triggers the fetching of alerts from external systems.
- Data Standardization: Raw alert data is converted into TEDS-compliant objects using the Placeholder - Create TEDS Alert List component.
- Deduplication: Alerts are evaluated for uniqueness using the Duplicate Alert Discovered (Cron) component to avoid redundant processing.
- Enrichment:
- The Link Knowledge Base Articles (Cron) component associates relevant KBAs with the alert.
- The Enrich Observables (Cron) component adds Threat Intelligence data to the alert's observables.
- Correlation: Enriched alerts are correlated with existing data using the Correlate (Cron) component.
- Case Creation: Based on predefined criteria, alerts are escalated into cases for further investigation.
Configuring the Key Components of SOC - Alert Ingestion (Cron) - Template
Placeholder - Create TEDS Alert List
Purpose: Queries alerts from external systems based on organization and time parameters, then converts the fetched alerts into TEDS (Turbine Extendable Data Schema) objects for further processing.
Configuration:
- Open the Placeholder - Create TEDS Alert List component.
- Configure the required inputs:
- organization (required): Specify the organization identifier for the alerts you want to fetch. This may be:
- An organization ID from your source system (e.g., "org-12345")
- An organization name (e.g., "Security Operations")
- A property reference from the playbook context (e.g., {{ playbook.organization }})
- A static value configured in your playbook
- start_time (required): Specify the start time for querying alerts. This determines which alerts are fetched based on their creation or ingestion time. The format depends on your source system and connector:
- ISO 8601 timestamp: "2024-01-01T00:00:00Z"
- Relative time expression: "1 hour ago" or "24 hours ago" (if supported by your connector)
- Property reference: {{ playbook.start_time }} or {{ playbook.last_run_time }}
- Note: The component uses this parameter to query alerts created or ingested after this time. For cron-based ingestion, you typically want to fetch alerts since the last run.
Inputs:
- organization (required): Organization identifier for filtering alerts from the source system
- start_time (required): Start time for querying alerts (determines the time range for alert retrieval)
Outputs:
- alerts: Array of standardized TEDS alert objects, where each alert contains fields such as:
- alert_category: Type of alert (e.g., phishing, malware, suspicious activity)
- alert_created_timestamp: Timestamp when the alert was created in the source system
- alert_provider: Source system or tool that generated the alert
- alert_uid: Unique identifier for the alert
- alert_title: Title or summary of the alert
- alert_description: Detailed description of the alert
- alert_severity: Severity level of the alert
- alert_risk_score: Risk score associated with the alert
- Additional TEDS-compliant fields as defined by the Alert schema
Important Notes:
- The component queries alerts from your configured source system (via connector/asset) using the organization and start_time parameters.
- The fetched alerts are automatically transformed into TEDS format according to the Alert Triage Ingestion interface.
- You may need to configure field mappings within the component if your source system's alert format differs from the standard TEDS schema.
- Ensure your connector/asset is properly configured to authenticate and access the source system.
- The start_time parameter is critical for incremental ingestion - it ensures you only fetch new alerts since the last run, avoiding duplicate processing.
Example Configuration:
organization: "org-12345"
start_time: "2024-01-15T00:00:00Z"Or using playbook context:
organization: {{ playbook.organization }}
start_time: {{ playbook.last_run_timestamp }}Custom Alert Data Extension (Cron)
Purpose: Takes the raw JSON payload of the incoming alert and adds new fields to a Custom_Alert_Data object.
Configuration:
- Open the Custom Alert Data Extension (Cron) component.
- Define custom fields required for your organization's workflows. For example:
- custom-alert-id: A unique identifier for alerts in your system.
- enriched-severity: A recalculated severity score based on internal logic.
- custom-sla: The unique SLA values for a specific alert.
- Map additional fields from the fetched alert data to these custom fields.
- Ensure the output object is updated to include the new fields for correlation.
Note: In order to automatically map custom fields to the CIM application, make sure the field names exactly match the field key values in the application definition. Otherwise, the playbook run results in an error.
Input:
- Raw Alert Data: JSON payload of the incoming alert.
Outputs:
- Enriched Alert TEDS: Object containing the enriched alert fields, including:
- custom-alert-id
- enriched-severity
- custom-sla
Correlate (Cron)
Purpose: Correlates the ingested and enriched alert data with existing data for better context and prioritization.
Configuration:
- Open the Correlate (Cron) component.
- Map fields from the enriched TEDS object to the correlation logic. Key fields include:
- MITRE Attack Tactic/Technique: Used for mapping to MITRE frameworks.
- alert_impacted_ip_addresses: IPs involved in the alert.
- alert_impacted_usernames: User accounts affected by the alert.
- Define correlation rules and logic.
- Ensure the output object includes CIM_Tracking_IDs and Correlation_Context.
Inputs:
- Enriched Alert TEDS: Object containing enriched alert fields.
- Existing CIM Records: Existing data for correlation.
Outputs:
- CIM_Tracking_IDs: Array of tracking IDs for correlation.
- Correlation_Context: Additional contextual data.
Testing and Validation
- Run the playbook manually to test the configuration of each component.
- Validate that:
- The organization and start_time inputs are correctly configured for the Placeholder - Create TEDS Alert List component.
- The fetched alerts are correctly converted into TEDS objects.
- Custom fields are populated as expected.
- Correlation rules are applied correctly, and relevant data is matched.
- Review the playbook's output to ensure the processed alerts are accurate and ready for case creation.
- Debug and refine mappings as needed.
Testing Tips:
- Start with a recent start_time to limit the number of alerts fetched during initial testing.
- Verify that the organization value matches your source system's organization identifier format.
- Check the component logs to ensure alerts are being fetched successfully.
- Validate that the output alerts contain all required TEDS fields.
Deployment
- Activate the playbook after testing.
- Monitor execution logs to ensure smooth operation.
- Adjust configurations in the Placeholder - Create TEDS Alert List, Custom Alert Data Extension (Cron), and Correlate (Cron) components as requirements evolve.
- Ensure the start_time parameter is configured to support incremental ingestion (e.g., using the last run timestamp).
Deployment Considerations:
- Configure the cron schedule appropriately for your alert volume and requirements.
- Set up proper error handling and alerting for failed playbook runs.
- Monitor the playbook execution to ensure alerts are being processed correctly.
- Consider implementing a mechanism to track and update the start_time parameter automatically based on the last successful run.
This documentation outlines the configuration and usage of the SOC - Alert Ingestion (Cron) playbook, focusing on the key components users need to configure.