--- title: Using the Big Data Native Action slug: turbine-user-guide/using-the-big-data-native-action docTags: createdAt: 2026-10-21T03:48:25.809Z --- :::hint{type="note"} **New Turbine User Guide topic for 26.5.0** — Publish under **Orchestration → Playbooks → Native Actions**, as a sibling of [Using the Script Native Action](https://docs.swimlane.com/using-the-script-native-action) (`C79iN0DV9X7FHwCK5OvS7`). Add a row on [Native Actions](docId\:rBn1ysy4kRUWYWmpwbyGa). Do not fold this into Script. ::: The **Big Data** native action runs PySpark against large file inputs in a playbook. Use [Using the Script Native Action](https://docs.swimlane.com/using-the-script-native-action) for sandboxed Python on a single action run. Use [Record Actions](docId\:j6eUvk5LaYMULE_3t1nOv) **Bulk mode** to create, update, or delete many records in a Performance Application. Big Data is supported for the Vulnerability Response Management (VRM) solution. :::hint{type="info"} The Big Data native action is behind a feature flag and must be enabled; to enable this feature, contact Swimlane Support. The Big Data native action is not supported on Turbine On-Prem. ::: ## When to Use the Big Data Native Action Use **Big Data** when the playbook must transform **large attachment files** with Spark, not one-record Python. | **Situation** | **Use this topic** | | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | You have CSV or JSON files and need to filter, join, or count rows at batch scale | Yes. Map **Attachment** inputs and write PySpark. See [Quick Path](docId\:BxHuwx2-qHFg1ZnIsZf4e). | | You need a Spark session and file helpers in the same action | Yes. The runtime injects `spark`, `helpers.csv`, `helpers.json`, and `output_attachment`. | | You only need sandboxed Python on a small payload | No. Use [Using the Script Native Action](https://docs.swimlane.com/using-the-script-native-action). | | You need to create, update, or delete many records in a Performance Application | No. Use **Bulk mode** on [Record Actions](Record%20Actions.md). | ## How Big Data Helps | **Benefit** | **How it helps** | | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Process file batches in the playbook | The action runs PySpark on attachments you map. You do not have to loop Script once per row. | | Reuse Script-style globals | You still set `action_inputs`, `action_outputs`, and `action_error`. Big Data also gives you `spark` and file helpers. | | Write files back to the playbook | `helpers.csv.write` and `helpers.json.write` return output attachments. Use `output_attachment` for other files. Do not put input attachments in `action_outputs`. | | Stay on a separate Canvas action | **Big Data** is not a mode on **Script**. The add-panel label is **Big Data**. The tile description is **Write a batch script to transform data**. | ## Quick Path | **Step** | **Action** | | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | Open a playbook in Canvas. On **Actions**, drag **Big Data** onto a flow. | | 2 | Set **Title** and **Key** on the **Action** panel. Select **Configure**. | | 3 | On **Inputs**, add an **Attachment** and map a file source. | | 4 | On **Script**, write PySpark. The starter example uses `helpers.csv.read` and sets `action_outputs`. Use `helpers.csv.load` or `helpers.json.load` only for small files in driver memory. | | 5 | Review **Outputs**. **Promote Outputs** is not available. In a later action, map a playbook property expression to this action. Select **Apply**. There is no **Test** tab. | ## Choose Your Path | **Goal** | **Where to go** | | ----------------------------------------- | ---------------------------------------------------------------------- | | Decide whether Big Data is the right path | [When to Use the Big Data Native Action](docId\:BxHuwx2-qHFg1ZnIsZf4e) | | Complete the first run | [Quick Path](docId\:BxHuwx2-qHFg1ZnIsZf4e) | | Add the action to a playbook | [Add the Big Data Action](docId\:BxHuwx2-qHFg1ZnIsZf4e) | | Set title, key, or enable the node | [Configure the Action Panel](docId\:BxHuwx2-qHFg1ZnIsZf4e) | | Define inputs and write PySpark | [Configure Inputs, Script, and Outputs](docId\:BxHuwx2-qHFg1ZnIsZf4e) | | Compare with Script | [How Big Data Differs from Script](docId\:BxHuwx2-qHFg1ZnIsZf4e) | ## Add the Big Data Action 1. Open a playbook in Canvas. 2. Open the add panel and select **Actions**. 3. Drag **Big Data** onto a **+** on the flow. The tile description is **Write a batch script to transform data**. 4. Select the node. ## Configure the Action Panel Select the **Big Data** node. The right **Action** panel opens. | **Field** | **Description** | | ----------------- | ------------------------------------------------------------------------------------------- | | **Action** | Type label **Big Data**. | | **Title** | Required display name. | | **Key** | Required. Locked by default. Hint: **Must be unique and only contain A-Z, a-z, 0-9 or \_**. | | **Description** | Optional. | | **Enable Action** | On by default. Turn off to keep the node on the canvas without running it. | The **Data** section has **Inputs** and **Outputs**. **Configure** opens the full dialog. Until you add inputs, **Inputs** reads **No inputs available**. ## Configure Inputs, Script, and Outputs 1. On the **Action** panel, select **Configure**. 2. Use the **Inputs**, **Script**, and **Outputs** tabs. 3. Select **Apply**. Footer actions are **Close** and **Apply**. There is no **Test** tab. ### Inputs The **Inputs** tab has two panes. The left pane lists input properties. The right pane is **Selection**. 1. Select **Add property**. The only type is **Attachment**. 2. Select the new input. The right pane shows **Choose input source**. 3. Map a value with **Add Playbook Property** or **Add Expression**. **Add Playbook Property** opens the playbook property drawer and lists file-type sources only. Until you add an input, the empty state is **Start by adding an input**. The Action panel **Inputs** list reads **No inputs available** until you apply at least one input. ### Script The **Script** tab is the PySpark editor. The editor title is **Code editor**. Starter comments begin with **# Big Data (PySpark)** and list `action_inputs`, `action_outputs`, `action_error`, `spark`, `output_attachment`, and `helpers.csv` / `helpers.json`. The product starter comments include this pattern: ```javascript # df = helpers.csv.read(action_inputs["attachments"]) # action_outputs["rows"] = df.count() # action_outputs["filtered"] = helpers.csv.write(df, "/tmp/filtered.csv") ``` Set `action_error` when the action must fail. After the script finishes, the runtime uploads output files and stops Spark. ### Outputs **Promote Outputs** is not available on Big Data. The **Outputs** tab lists **Action Outputs**: **Result** and **Error** (**Message**, **Traceback**, **Type**). To use a Big Data result in a later action, map a playbook property or expression that points at this action. Do not look for a Promote Outputs control. ## How Big Data Differs from Script | **Item** | **Big Data** | **Script** | | ---------------- | ------------------------------------------------------------------------------ | ------------------------------------------------------ | | Add-panel action | **Big Data** | **Script** | | Language | PySpark | Python 3 in a sandbox | | Configure tabs | **Inputs**, **Script**, **Outputs** | Combined inputs and code, plus **Test** when available | | Input types | **Attachment** only. Map with **Add Playbook Property** or **Add Expression**. | Schema types on the Script tab | | Test tab | None | Yes — see the Script topic | ## What You See After Setup | **Where** | **What you see** | | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Canvas | A **Big Data** node. The **Action** panel shows type **Big Data**. Until you apply inputs, **Inputs** reads **No inputs available**. | | Configure dialog | Tabs are **Inputs**, **Script**, and **Outputs**. Footer is **Close** and **Apply**. There is no **Test** tab. | | Playbook run | The action runs PySpark on the mapped attachments. Downstream actions read this result through a playbook property expression. Confirm success or `action_error` in the run result. | ## Limits | **Limit** | **What it means** | | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | | Attachment inputs only | **Add property** offers **Attachment** only. **Add Playbook Property** lists file-type sources only. | | No Test tab | You cannot dry-run the script in the dialog. Use a playbook run. | | Not a Script mode | Do not look for a Big Data switch on **Script**. | | Output files | Write files with `helpers.csv.write`, `helpers.json.write`, or `output_attachment`. Do not place input attachments in `action_outputs`. | ## Troubleshoot Big Data | **What you see** | **What to do** | | ---------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | | The **Big Data** tile is missing | The Big Data native action is behind a feature flag and must be enabled; to enable this feature, contact Swimlane Support. | | **Start by adding an input** or **No inputs available** | Add an **Attachment** and select **Apply**. | | No file sources in **Add Playbook Property** | Map a file-type playbook property or expression. Other field types do not appear. | | You expected a Test tab | That tab is on **Script** only. Run the playbook to verify Big Data. | | The action fails after you put an input file in `action_outputs` | Write a new file with the helpers or `output_attachment`. | ## See Also - [Native Actions](docId\:rBn1ysy4kRUWYWmpwbyGa) - [Using the Script Native Action](https://docs.swimlane.com/using-the-script-native-action) - [Record Actions](docId\:j6eUvk5LaYMULE_3t1nOv) - [Manage Applications and Applets](docId\:q3zurM9vFcJ5U51Vm3v4Y)