---
title: SOC - Bulk Ingest Phishing Emails - Template Playbook Overview
slug: solutions/soc-bulk-ingest-phishing-emails-template-playbook-overview
docTags: 
createdAt: 2025-08-11T11:13:45.686Z
---

This document provides detailed information about the **SOC - Bulk Ingest Phishing Emails - Template** playbook. Configured to run on a cron schedule, this playbook retrieves emails, processes them by adding observables, and creates or updates records. Users need to configure the following components:

- **Get Phishing Emails**
- **Custom Email Data Extraction**
- **Correlate**

**Note:** These playbooks can and should be **duplicated** and the **Get Phishing Emails** component swapped out to match the technology stack in your organization. This ensures the playbook works seamlessly with your chosen vendor's tools.

## Overview

- **Objective:** Automate the bulk ingestion of phishing emails, extract relevant data, enrich observables, and update or create CIM records.
- **Key Workflow Steps:**
  1. Scheduled tasks retrieve reported phishing emails using the **Get Phishing Emails** component.
  2. Fetched emails are processed into standardized Email TEDS objects.
  3. Custom data fields are extracted and added using the **Custom Email Data Extraction** component.
  4. Observables are enriched, and correlation logic identifies related records.
  5. Results are used to create or update CIM records.

## Accessing the Playbook

To access the playbook:

1. Navigate to **Orchestration** in the Swimlane platform.
2. Click on **Playbooks**.
3. Select **SOC - Bulk Ingest Phishing Emails - Template**.

## Loop-Based Email Processing in the Playbook

The playbook processes emails iteratively, ensuring each email is uniquely handled. Here's how it works:

1. **Email Retrieval:** The **Get Phishing Emails** component fetches a list of reported phishing emails from the configured email vendor.
2. **Data Standardization:** Emails are transformed into standardized Email TEDS objects.
3. **Deduplication:** Emails are checked for duplicates using the **Duplicate Phishing Email Detected** component to avoid redundant processing.
4. **Enrichment:**
   - The **Enrich Observables** component adds Threat Intelligence (TI) data for associated observables.
   - The **Link Knowledge Base Articles** component associates relevant KBAs with the email.
5. **Correlation:** Correlates email data with existing records using the **Correlate** component.
6. **Case Creation:** Based on predefined criteria, phishing emails are escalated into cases for further investigation.

## Configuring the Key Components of SOC - Bulk Ingest Phishing Emails

### Get Phishing Emails

**Purpose:** Retrieves a list of reported phishing emails for processing.

**Details:** Placeholder component that can be replaced with vendor-specific integration components (VICs) to output TEDS data.

**Configuration:**

- Replace the **Get Phishing Emails** component with the VIC appropriate to your email vendor.
- Configure the email vendor settings to ensure proper retrieval of phishing emails.
- Set up the component inputs and any required authentication parameters.
- Configure any filtering or query parameters needed to retrieve phishing emails (e.g., folder location, date range, subject filters).

**Inputs:**

- **Email Configuration Settings:** Parameters to connect with the email vendor, which may include:
  - API credentials or authentication tokens
  - Email server connection details (host, port, protocol)
  - Folder or mailbox location for phishing emails
  - Filter criteria (date range, subject patterns, sender filters)
  - Rate limiting or pagination settings

**Note:** The specific inputs depend on your email vendor and the VIC you're using. Consult your vendor-specific VIC documentation for detailed input requirements.

**Outputs:**

- **Email TEDS Objects:** A list of standardized TEDS objects containing phishing email data, including:
  - `email_message_id`: Unique message identifier
  - `email_subject`: Email subject line
  - `email_from_address`: Sender email address
  - `email_to_addresses`: Array of recipient email addresses
  - `email_cc_addresses`: Array of CC recipient addresses
  - `email_bcc_addresses`: Array of BCC recipient addresses
  - `email_body`: Email body content
  - `email_text_body`: Plain text version of email body
  - `email_html_body`: HTML version of email body
  - `email_origination_timestamp`: When the email was sent
  - `email_delivery_timestamp`: When the email was delivered
  - `email_headers`: Array of email headers
  - `email_mime_parts`: Array of MIME parts including attachments
  - `observables`: Array of extracted observables (IPs, domains, URLs, etc.)
  - `raw_email`: Raw email content

**Important Notes:**

- Ensure your email vendor connector/asset is properly configured before using this component.
- The component queries emails from your configured email source based on the specified filters and parameters.
- The fetched emails are automatically transformed into TEDS Email format according to the Email to Email interface.
- You may need to configure field mappings if your email vendor's format differs from the standard TEDS Email schema.

### Custom Email Data Extraction

**Purpose:** Extracts or adds custom data fields to the email TEDS object, handling non-standardized data fields.

**Configuration:**

- Open the **Custom Email Data Extraction** component.
- Define the custom fields to be added:
  - The key of **Custom\_Email\_Data** should match the custom field added to the CIM application.
  - The value should correspond to the user-provided data.
- Ensure the output object includes the **Custom\_Email\_Data** field for downstream processing.

**Note:** In order to automatically map custom fields to the CIM application, make sure the field names exactly match the field key values in the application definition. Otherwise, the playbook run results in an error.

**Input:**

- **Email TEDS:** Object containing standardized email data.

**Output:**

- **Custom\_Email\_Data:** Object containing the extracted or added custom fields.

### Correlate

**Purpose:** Correlates the ingested email data with existing CIM records for contextual analysis.

**Configuration:**

- Open the **Correlate** component.
- Map fields from the Email TEDS object and custom data to the correlation logic.
- Ensure the output includes:
  - **CIM\_Tracking\_IDs:** An array of tracking IDs for correlation.
  - **Correlation\_Context:** Additional context generated during correlation.
- Add correlation logic to the parallel node, followed by **Append Variable** and **Update Variable** nodes.

**Input:**

- **Email TEDS:** Object containing standardized email data.
- **Custom Data:** Object containing additional custom fields.

**Output:**

- **CIM\_Tracking\_IDs:** Array of tracking IDs.
- **Correlation\_Context:** Contextual data from the correlation.

## Supporting Components in the Playbook

These components play a critical role in the workflow but do not require user configuration:

- **Enrich Observables:** Enriches observables like URLs, IPs, or domains and creates or finds mapped Threat Intelligence (TI) record IDs.
- **Duplicate Phishing Email Detected:** Checks for duplicate emails to prevent redundant processing.
- **Link Knowledge Base Articles:** Associates phishing emails with relevant Knowledge Base Articles for immediate context and resolution.

## Testing and Validation

1. Simulate the ingestion process by running the playbook manually.
2. Validate that:
   - Emails are correctly retrieved and transformed into TEDS objects.
   - Custom fields are populated as expected.
   - Correlation rules are applied correctly, and relevant records are matched.
3. Review the playbook's output to ensure accurate processing of phishing emails.
4. Debug and refine configurations as needed.

**Testing Tips:**

- Start with a small date range or limited number of emails during initial testing.
- Verify that your email vendor connector is properly authenticated and can access the email source.
- Check that email fields are correctly mapped to TEDS format.
- Validate that observables are being extracted correctly from email content.

## Deployment

1. Activate the playbook after testing.
2. Monitor execution logs to ensure smooth operation.
3. Adjust configurations in the **Get Phishing Emails**, **Custom Email Data Extraction**, and **Correlate** components as requirements evolve.

**Deployment Considerations:**

- Configure the cron schedule appropriately for your email volume and requirements.
- Set up proper error handling and alerting for failed playbook runs.
- Monitor the playbook execution to ensure emails are being processed correctly.
- Consider implementing rate limiting if processing large volumes of emails.

This documentation outlines the configuration and usage of the **SOC - Bulk Ingest Phishing Emails** playbook, focusing on the key components users need to configure.