Documentation
...
Working with Turbine Schema
Turbine Schema Reference (Classic SOC)
turbine schema was formerly known as turbine extensible data schema, or teds this document describes the turbine schema objects used by the classic soc solutions bundle , including alert, email, observable, enrichment, and supporting objects for general concepts, best practices, and troubleshooting, see docid\ tojmk0bspocoe9u12tt5m for the ai soc solution schema with additional alert, email, and search params fields, see docid 7efe4hvzv8uzngfkwhjtr for classic soc interface contracts, see docid\ lkj7c4qday5w52sv1sq 2 schema examples the following examples show real world turbine schema formatted data structures used in the soc solutions bundle alert object example here's a complete alert object following turbine schema conventions { "alert uid" "splunk alert 20250115 001234", "alert title" "suspicious powershell execution detected", "alert description" "powershell script executed with encoded commands on host workstation 01", "alert severity" "high", "alert created timestamp" "2025 01 15t10 30 00z", "alert start timestamp" "2025 01 15t10 28 15z", "alert end timestamp" "2025 01 15t10 30 00z", "alert ingested timestamp" "2025 01 15t10 30 05z", "alert provider" "splunk enterprise", "alert organization" "acme corporation", "alert categories" \["malware", "execution"], "alert impacted hostnames" \["workstation 01"], "alert impacted ip addresses" \["192 168 1 100"], "alert impacted usernames" \["jdoe"], "alert risk score" 85, "alert permalink" "https //splunk example com/alerts/001234", "alert rules" \[ { "rule id" "spl alert 001", "rule name" "suspicious powershell execution", "rule description" "detects powershell execution with base64 encoded commands", "rule type" "correlation" } ], "alert mitre attack tactic technique" \[ { "tactics" \[ { "uid" "ta0002", "name" "execution" } ], "technique" { "uid" "t1059 001", "name" "powershell" }, "version" "14 1" } ], "observables" \[ { "observable type" "sha256", "observable value" "a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5", "observable primary provider" "virustotal", "observable primary verdict" "malicious", "observable primary timestamp" "2025 01 15t10 31 00z", "observable enrichments" \[ { "enrichment type" "reputation", "enrichment provider" "virustotal", "enrichment verdict" "malicious", "enrichment timestamp" "2025 01 15t10 31 00z", "enrichment permalink" "https //www virustotal com/gui/file/a3b5c7d9 " } ] }, { "observable type" "ipv4 public", "observable value" "203 0 113 45", "observable primary provider" "recorded future", "observable primary verdict" "suspicious" } ], "alert originating files" \[ { "file name" "suspicious ps1", "file hashes" \[ { "hash algorithm" "sha256", "hash value" "a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5" } ], "mime type" "text/plain" } ], "raw alert" { "splunk sid" "1234567890 12345", "index" "security", "source" "powershell", "sourcetype" "wineventlog\ microsoft windows powershell/operational" } } email object example here's a complete email object following turbine schema conventions { "email message id" "\<cae1234567890\@mail example com>", "email from address" "attacker\@malicious domain com", "email to addresses" \["user\@example com"], "email cc addresses" \[], "email bcc addresses" \[], "email reply to addresses" \["noreply\@malicious domain com"], "email subject" "urgent action required on your account", "email body" "\<html>\<body>click here to verify your account \</body>\</html>", "email html body" "\<html>\<body>click here to verify your account \</body>\</html>", "email text body" "click here to verify your account http //phishing site com/verify", "email origination timestamp" "2025 01 15t09 00 00z", "email delivery timestamp" "2025 01 15t09 00 15z", "email organization" "example corp", "email headers" \[ { "header key" "received", "header value" "from mail malicious domain com (\[203 0 113 1])" }, { "header key" "x mailer", "header value" "phishing tool v1 0" } ], "email mime parts" \[ { "mime type" "text/html", "is attachment" false, "content" { "base64" "pgh0bww+ " } }, { "mime type" "application/pdf", "is attachment" true, "file name" "invoice pdf", "content" { "base64" "jvberi0xljqkjeljz9mk " } } ], "observables" \[ { "observable type" "url", "observable value" "http //phishing site com/verify", "observable primary provider" "urlhaus", "observable primary verdict" "malicious", "observable primary timestamp" "2025 01 15t09 05 00z" }, { "observable type" "ipv4 public", "observable value" "203 0 113 1", "observable primary provider" "recorded future", "observable primary verdict" "suspicious" } ], "raw email" "return path \<attacker\@malicious domain com> " } observable object example here's a complete observable object with enrichment data { "observable type" "sha256", "observable value" "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "observable metadata" { "file name" "malware exe", "file size" 524288, "mime type" "application/x msdownload" }, "observable primary provider" "virustotal", "observable primary verdict" "malicious", "observable primary timestamp" "2025 01 15t10 00 00z", "observable primary permalink" "https //www virustotal com/gui/file/e3b0c442 ", "observable primary context" "file hash analysis", "observable enrichments" \[ { "enrichment type" "reputation", "enrichment provider" "virustotal", "enrichment verdict" "malicious", "enrichment timestamp" "2025 01 15t10 00 00z", "enrichment permalink" "https //www virustotal com/gui/file/e3b0c442 ", "enrichment raw data" "{\\"detections\\" 45, \\"total\\" 70}" }, { "enrichment type" "reputation", "enrichment provider" "recorded future", "enrichment verdict" "malicious", "enrichment timestamp" "2025 01 15t10 01 00z", "enrichment context" "threat intelligence feed" } ] } transformation example raw siem alert to turbine schema alert this example shows how raw data from a siem system is transformed into turbine schema format raw splunk alert { " time" "2025 01 15t10 30 00z", "sid" "1234567890 12345", "search name" "suspicious powershell execution", "severity" "high", "host" "workstation 01", "user" "jdoe", "src ip" "192 168 1 100", "file hash" "a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5", "results link" "https //splunk example com/alerts/001234" } transformed to turbine schema alert { "alert uid" "splunk 1234567890 12345", "alert title" "suspicious powershell execution", "alert severity" "high", "alert created timestamp" "2025 01 15t10 30 00z", "alert start timestamp" "2025 01 15t10 30 00z", "alert provider" "splunk enterprise", "alert impacted hostnames" \["workstation 01"], "alert impacted usernames" \["jdoe"], "alert impacted ip addresses" \["192 168 1 100"], "alert permalink" "https //splunk example com/alerts/001234", "observables" \[ { "observable type" "sha256", "observable value" "a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5" } ], "raw alert" { " time" "2025 01 15t10 30 00z", "sid" "1234567890 12345", "search name" "suspicious powershell execution", "severity" "high", "host" "workstation 01", "user" "jdoe", "src ip" "192 168 1 100", "file hash" "a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5c7d9e1f2a3b5", "results link" "https //splunk example com/alerts/001234" } } key transformation steps map sid → alert uid (with prefix for uniqueness) map search name → alert title normalize severity → alert severity (lowercase to title case) map time → alert created timestamp and alert start timestamp extract file hash → create observable object in observables array map host , user , src ip → respective alert impacted arrays preserve original data in raw alert field field reference tables the following sections list the turbine schema objects and their fields for the classic soc solutions bundle inputs for different objects here is a list of objects and their corresponding inputs alert siem/xdr alert a siem (security information and event management) or xdr (extended detection and response) alert serves as an early warning system for potential security incidents within an organization's it environment by aggregating and analyzing log data, network activity, endpoint behaviors, and user actions, a siem or xdr platform can detect suspicious or malicious activities and generate alerts for security teams to investigate and respond name type key requirement description description string alert description recommended provides a brief summary of the alert's purpose, detailing the nature of the alert and the significance of the event or activity it highlights mitre attack tactic/technique array alert mitre attack tactic technique optional specifies mitre att\&ck tactics and techniques associated with the alert, giving context to the type of threat involved this can help analysts map the attack to known adversary behaviors title string alert title recommended name or title of the alert category string array alert categories optional defines the category or classification of the alert, such as “phishing,” “malware,” or “unauthorized access,” to group alerts for streamlined analysis created timestamp datetime alert created timestamp recommended indicates the date and time when the alert was first generated, providing the origin of the alert timeline end timestamp datetime alert end timestamp recommended specifies when the alert’s triggering activity ended, helping define the duration and scope of the security event impacted hostnames string array alert impacted hostnames optional lists hostnames of devices affected by the alert, which helps identify which systems need review or remediation impacted ip addresses ip address array alert impacted ip addresses optional lists ip addresses associated with impacted devices, aiding in network level investigation and isolation efforts impacted usernames string array alert impacted usernames optional provides usernames of users impacted by the alert, helping analysts understand who may be at risk or targeted ingested timestamp datetime alert ingested timestamp recommended indicates when the alert was ingested into the system, useful for tracking alert flow within security monitoring tools organization string alert organization optional identifies the organization impacted, important for multi tenant or mssp setups originating files file array alert originating files optional references files involved in the alert, helping analysts pinpoint malicious files or artifacts for review permalink url string alert permalink optional provides a direct link to the alert source for quick access to detailed information provider string alert provider optional identifies the tool or service that generated the alert, helping analysts understand the source’s reliability and capabilities risk score integer alert risk score optional reflects the alert’s risk score as determined by the alerting system, aiding in prioritization based on perceived threat level detection rules detection rule array alert rules recommended lists detection rules that triggered the alert, providing insight into the criteria met for the alert generation severity string alert severity recommended describes the alert’s severity, helping analysts prioritize alerts based on potential impact (example, high, medium, low) start timestamp datetime alert start timestamp recommended indicates when the activity that triggered the alert began, setting the start point of the incident timeline alert uid string alert uid required provides a unique identifier for the alert, essential for tracking, referencing, and correlating alerts observables observable array observables recommended contains indicators of compromise (iocs) such as hashes, urls, or ips linked to the alert, vital for further investigation raw alert json raw alert required raw json format of the alert alert triage ingestion this solution is designed to ingest alerts from various alerting tools, such as siem, edr, and av systems it enriches the alerts with data from third party services like virustotal, urlhaus, and recorded future, then creates cases for analysts to review, collaborate on, and resolve the goal is to automate the soc team's handling of the constant stream of alerts, driving incidents to remediation or closing invalid ones this results in faster remediation, reduced costs in human resources, and an overall more secure environment name type key requirement description organization string organization solutions optional the organization impacted by the alert start time datetime start time solutions optional the starting point for searching alerts (example, 4 hours ago, 30 minutes) attack the attack object defines the specific techniques and corresponding tactics involved in an attack it provides detailed information about the methods used and references the relevant version of the att\&ck matrix™ monitor the network data for uncommon data flows processes utilizing the network do not normally have network communication or have never been seen before are suspicious name type key requirement description tactics tactic array tactics recommended a list of tactic ids/names linked to the attack technique, as defined by the att\&ck matrix™ technique string array technique recommended the specific attack technique used version string version recommended the version of the att\&ck matrix™ framework used cloud storage query input cloud storage query input is a feature that enables users to perform customized searches for files within a cloud storage provider's system by inputting specific parameters or keywords, users can filter files based on attributes like file name, type, date, and other metadata this query input functionality is essential for quickly locating files within vast repositories, optimizing access times, and improving data management efficiency, especially in enterprise environments such as turbine schema name type key requirement description cloud storage query string cloud storage query recommended a query string for searching files within a cloud storage provider's system content file like content, attachment, or bytearray caption name requirement type description base64 base64 optional byte string the content encoded in base64 format turbine attachment turbine attachment optional turbine attachment the content formatted as a turbine attachment detection rule detection rule is a configurable rule designed to trigger alerts when specific conditions or anomalies are detected within a system it includes attributes such as a unique id, name, description, and type, allowing organizations to categorize and describe the rule’s purpose and scope, enabling more effective monitoring and response to potential threats caption name requirement type description rule description rule description optional string the description of the detection rule rule id rule id recommended string an unique id for the detection rule rule name rule name recommended string the name of the detection rule rule type rule type optional string the type of detection rule email the email object represents essential metadata related to an email, including details about the sender, recipients, and email's direction it contains fields for tracking various email components such as bcc, cc, subject, body, timestamps, headers, and observables, enabling comprehensive analysis and record keeping of email communications caption name requirement type description email bcc address email bcc addresses optional email address array lists the bcc recipients in the email email body email body recommended string contains the email body, showing the html version if available or the text version otherwise email cc address email cc addresses optional email address array lists the cc recipients in the email email delivery timestamp email delivery timestamp optional datetime records the delivery time of the email email from address email from address required email address shows the email address in the from header email headers email headers optional header array provides all email headers as key/value pairs email html body email html body optional string contains the html part of the email, as displayed in an email client email message id email message id required string shows the message id header email mime parts email mime parts optional mime part array lists the non multipart mime parts of the email, excluding any part identified as the body email organization email organization optional string identifies the recipient organization email origination timestamp email origination timestamp required datetime records the time from the date header when the email was sent email reply to address email reply to addresses optional email address array lists the reply to addresses email subject email subject recommended string displays the subject header email text body email text body optional string contains the plain text part of the email, as displayed in an email client email to address email to addresses required email address array lists the email recipients in the to header observables observables recommended observable array identifies any possible indicators of compromise within the email raw email raw email recommended string displays the raw email content as it was received by the server enrichment the enrichment object adds valuable context to event attributes, providing metadata such as the provider, type, timestamp, raw data, and a link to the original source it also includes a reputation verdict (example, benign, malicious, suspicious, or unknown) to quickly assess the nature of the enriched data and aid in detailed event analysis caption name requirement type description enrichment context enrichment context optional string enrichment context enrichment permalink enrichment permalink optional url string a link to the original enrichment source enrichment provider enrichment provider recommended string the enrichment data provider name enrichment raw data enrichment raw data optional string the raw enrichment as returned from the enrichment source enrichment timestamp enrichment timestamp recommended datetime datetime the enrichment was retrieved enrichment type enrichment type required string the enrichment type (only one used currently) reputation enrichment verdict enrichment verdict optional string the reputation verdict benign malicious suspicious unknown error the error object captures details about errors encountered during processing, including essential information to aid in troubleshooting it includes fields like the error provider, which specifies the tool, service, or software source of the error; error result, offering a description of the error; and error status, indicating the current status of the error, allowing for efficient identification and resolution of issues caption name requirement type description error provider error provider optional string the name of the tool, service, or software that generated the error error result error result optional string the description of the error error status error status optional string the error status file the file object provides comprehensive details about files, folders, links, and mounts, including relevant metadata and potential reputation information for security analysis key attributes include the file's content, access times (created, accessed, and modified), file name and encoding, unique file hashes for identification, mime type, file size, and any observable indicators of compromise, supporting robust file tracking and management across systems caption name requirement type description content content recommended content provides the actual content of the file file accessed file accessed optional datetime records when the file was last accessed file created file created optional datetime records when the file was created file hashes file hashes recommended file hash array identifies the file with unique hashes/fingerprints file modified file modified optional datetime records when the file was last modified file name file name recommended string displays the file name file name enc file name enc optional string indicates the text encoding of the file name (example, utf 8, cp 1252) file size file size optional integer shows the file size in bytes mime type mime type recommended string specifies the mime type of the file observables observables optional observable array lists any possible indicators of compromise in the file file hash the file hash object provides a unique digital fingerprint for a file, allowing verification of its integrity and authenticity it includes the hash algorithm name, specifying which hashing method (example, md5, sha 256) was used, and the hash value, which is the unique code generated by the hash function, ensuring files can be consistently identified and monitored for changes caption name requirement type description algorithm name hash algorithm required string specifies the name of the hashing algorithm hash value hash value required file hash provides the unique value returned from the hash function header the header object represents key value pairs within http or email headers, containing essential metadata for communication protocols it includes the header key, which specifies the name of the header (example, "content type" for http or "subject" for email), and the header value, which holds the corresponding information, such as the content type, sender, or other protocol specific data, facilitating structured data transmission and analysis caption name requirement type description smtp/http header value header value required string shows the value of the header smtp/http header key header key required string specifies the name of the header mime part the mime part object represents a single, non multipart section of a mime encoded email, containing data such as text, images, or attachments it includes fields like content, which stores the actual data of the mime part; file name, which is derived from the content disposition if available; is attachment, indicating whether the mime part is an attachment or inline content; and mime type, defining the media type (example, "text/plain" or "image/jpeg") for proper handling by email clients caption name requirement type description content content required content contains the actual contents of the mime part file name file name optional string extracts the file name from content disposition if present is attachment is attachment optional boolean indicates whether the mime part is an attachment or inline content based on content disposition mime type mime type required string defines the mime type of the part observable the observable object serves as a key reference element, capturing related data that appears across multiple points in an event, making it central for linking information and context key fields include observable enrichments, which is an array of enrichment data applied to the observable; observable metadata, used if the observable is composite (example, a file); primary context details from the main enrichment source, such as provider name, timestamp, and reputation verdict; observable type, defining the kind (example, url, file, email); and observable value, which contains either a simple identifier (example, ip address) or a unique string for complex observables (example, sha256 hash for files) caption name requirement type description observable enrichments observable enrichments optional enrichment array lists any enrichments that have been applied to the observable observable metadata observable metadata optional json populates information if the observable is a composite type, like a file observable primary context observable primary context optional string identifies the enrichment context from the primary enrichment source observable primary permalink observable primary permalink optional url string provides a link to the original enrichment source identified as primary observable primary provider observable primary provider optional string names the enrichment source identified as primary observable primary timestamp observable primary timestamp optional datetime records when the enrichment was retrieved from the primary enrichment source observable primary verdict observable primary verdict optional string states the verdict provided by the primary enrichment source observable type observable type required string specifies the type of observable domain email file ipv4 private ipv4 public ipv6 private ipv6 public md5 sha1 sha256 url observable value observable value required string contains the value of the observable for simple types or a unique identifier for composite types (eg the ip address) for a composite types, a unique string that identifies the observable (eg sha256 hash for file observables) phishing email report the phishing email report object captures comprehensive details about a suspected phishing email, providing critical information for investigation and security response it includes fields like report description for incident details, reporter information, and essential email components such as sender, recipients (to, cc, bcc), delivery timestamp, subject, and content (both html and text), along with observables for any compromise indicators, and raw email data as received by the server to support thorough analysis caption name requirement type description report description report description optional string describes the reported phishing incident reporter reporter optional user identifies the user who submitted the report email bcc address email bcc addresses optional email address array lists the recipients in the bcc header email body email body recommended string contains the email body (displays the html body if present, otherwise the text body) email cc address email cc addresses optional email address array lists the cc recipients email delivery timestamp email delivery timestamp optional datetime records the delivery time of the email email from address email from address required email address shows the email address in the from header email html body email html body optional string contains the html part of the email, as displayed in an email client email message id email message id required string shows the message id header email mime parts email mime parts optional mime part array lists the non multipart mime parts of the email, excluding the main body email organization email organization optional string identifies the recipient organization email origination timestamp email origination timestamp required datetime shows the time from the date header indicating when the email was sent email reply to address email reply to addresses optional email address array lists the reply to addresses email subject email subject recommended string shows the subject header email text body email text body optional string contains the plain text part of the email, as displayed in an email client email to address email to addresses required email address array lists the email recipients as shown in the to header observables observables recommended observable array identifies any possible indicators of compromise within the email raw email raw email recommended string displays the raw email as it was received by the server phishing triage email ingestion the phishing triage email ingestion process defines the parameters for processing and assessing incoming phishing emails, supporting proactive threat response key inputs include the organization field, specifying the affected organization, and start time, which determines how far back to search for relevant phishing reports, allowing for focused and time sensitive analysis of potential phishing threats caption name requirement type description organization solutions organization optional string specifies the organization affected by the alert start time solutions start time optional datetime sets how far back to search for phishing email reports (example, 4 hours ago, 4 hours, 30 minutes) simple observable the simple observable object acts as a key reference point within an event, containing data that is often repeated across multiple parts of the event it includes fields such as observable metadata, populated when the observable is a complex type (like a file), observable type, which defines the kind of data (example, domain, email, ip), and observable value, which provides a unique identifier or value, like an ip address for simple observables or a sha256 hash for composite ones caption name requirement type description observable metadata observable metadata optional json populates with data if the observable is a composite type, such as a file observable type observable type required string specifies the type of observable such as domain email file ipv4 private ipv4 public ipv6 private ipv6 public md5 sha1 sha256 url observable value observable value required string provides the value of the observable for simple types or a unique identifier for composite types status the status object provides detailed information about the current state of a tool, software, or service, capturing both the outcome and descriptive details for troubleshooting or logging key fields include status messages, an array of log like messages, status provider, identifying the source of the status, status raw, containing the unprocessed status in json format, status description, which summarizes the status, and status success, indicating whether the status reflects success or an error caption name requirement type description status messages status messages optional string array lists message strings, similar to log entries status provider status provider recommended string identifies the tool, software, or service returning the status status raw status raw optional json contains the raw status data as received from the provider status description status description recommended string summarizes the status status success status success required boolean indicates if the status reflects a successful return or an error tactic the tactic object identifies specific tactics linked to an attack technique, as outlined by the att\&ck matrix™, providing a standardized approach to describing adversarial behaviors it includes the name of the tactic, giving a descriptive label, and a unique id (uid), which serves as a standardized identifier for the tactic, enabling precise tracking and categorization of tactics used in cybersecurity threat analysis caption name requirement type description name name optional string https //attack mitre org/wiki/att\&ck matrix unique id uid required string https //attack mitre org/wiki/att\&ck matrix technique the technique object represents a specific att\&ck matrix™ technique used in an attack, detailing how adversaries may achieve certain objectives it includes fields like name, which gives the descriptive label of the technique (example, "drive by compromise"), and unique id (uid), a standardized identifier (example, "t1189"), which allows for consistent referencing and analysis of attack methods caption name requirement type description name name optional string https //attack mitre org/wiki/att\&ck matrix unique id uid required string https //attack mitre org/wiki/att\&ck matrix user the user object represents a user account with identifying information for secure and personalized interactions within a system it includes key attributes such as user email address, providing the user’s email for contact and identification, user id, a unique identifier for distinguishing each user account, and user name, the username associated with the account, supporting authentication and user specific activity tracking caption name requirement type description email address user email address recommended email address provides the user’s email address user id user id recommended string assigns a unique id to the user user name user name recommended string specifies the username associated with the account