Using the Big Data Native Action

New Turbine User Guide topic for 26.5.0 — Publish under Orchestration → Playbooks → Native Actions, as a sibling of Using the Script Native Action (C79iN0DV9X7FHwCK5OvS7). Add a row on Native Actions. Do not fold this into Script.

The Big Data native action runs PySpark against large file inputs in a playbook. Use Using the Script Native Action for sandboxed Python on a single action run. Use Record Actions Bulk mode to create, update, or delete many records in a Performance Application. Big Data is supported for the Vulnerability Response Management (VRM) solution.

The Big Data native action is behind a feature flag and must be enabled; to enable this feature, contact Swimlane Support. The Big Data native action is not supported on Turbine On-Prem.

When to Use the Big Data Native Action

Use Big Data when the playbook must transform large attachment files with Spark, not one-record Python.

Situation	Use this topic
You have CSV or JSON files and need to filter, join, or count rows at batch scale	Yes. Map Attachment inputs and write PySpark. See Quick Path.
You need a Spark session and file helpers in the same action	Yes. The runtime injects spark, helpers.csv, helpers.json, and output_attachment.
You only need sandboxed Python on a small payload	No. Use Using the Script Native Action.
You need to create, update, or delete many records in a Performance Application	No. Use Bulk mode on Record Actions.

How Big Data Helps

Benefit	How it helps
Process file batches in the playbook	The action runs PySpark on attachments you map. You do not have to loop Script once per row.
Reuse Script-style globals	You still set action_inputs, action_outputs, and action_error. Big Data also gives you spark and file helpers.
Write files back to the playbook	helpers.csv.write and helpers.json.write return output attachments. Use output_attachment for other files. Do not put input attachments in action_outputs.
Stay on a separate Canvas action	Big Data is not a mode on Script. The add-panel label is Big Data. The tile description is Write a batch script to transform data.

Quick Path

Step	Action
1	Open a playbook in Canvas. On Actions, drag Big Data onto a flow.
2	Set Title and Key on the Action panel. Select Configure.
3	On Inputs, add an Attachment and map a file source.
4	On Script, write PySpark. The starter example uses helpers.csv.read and sets action_outputs. Use helpers.csv.load or helpers.json.load only for small files in driver memory.
5	Review Outputs. Promote Outputs is not available. In a later action, map a playbook property expression to this action. Select Apply. There is no Test tab.

Choose Your Path

Goal	Where to go
Decide whether Big Data is the right path	When to Use the Big Data Native Action
Complete the first run	Quick Path
Add the action to a playbook	Add the Big Data Action
Set title, key, or enable the node	Configure the Action Panel
Define inputs and write PySpark	Configure Inputs, Script, and Outputs
Compare with Script	How Big Data Differs from Script

Add the Big Data Action

Open a playbook in Canvas.

Open the add panel and select Actions.

Drag Big Data onto a + on the flow. The tile description is Write a batch script to transform data.

Select the node.

Configure the Action Panel

Select the Big Data node. The right Action panel opens.

Field	Description
Action	Type label Big Data.
Title	Required display name.
Key	Required. Locked by default. Hint: Must be unique and only contain A-Z, a-z, 0-9 or _.
Description	Optional.
Enable Action	On by default. Turn off to keep the node on the canvas without running it.

The Data section has Inputs and Outputs. Configure opens the full dialog. Until you add inputs, Inputs reads No inputs available.

Configure Inputs, Script, and Outputs

On the Action panel, select Configure.

Use the Inputs, Script, and Outputs tabs.

Select Apply.

Footer actions are Close and Apply. There is no Test tab.

Inputs

The Inputs tab has two panes. The left pane lists input properties. The right pane is Selection.

Select Add property. The only type is Attachment.

Select the new input. The right pane shows Choose input source.

Map a value with Add Playbook Property or Add Expression. Add Playbook Property opens the playbook property drawer and lists file-type sources only.

Until you add an input, the empty state is Start by adding an input. The Action panel Inputs list reads No inputs available until you apply at least one input.

Script

The Script tab is the PySpark editor. The editor title is Code editor. Starter comments begin with # Big Data (PySpark) and list action_inputs, action_outputs, action_error, spark, output_attachment, and helpers.csv / helpers.json.

The product starter comments include this pattern:

# df = helpers.csv.read(action_inputs["attachments"])
# action_outputs["rows"] = df.count()
# action_outputs["filtered"] = helpers.csv.write(df, "/tmp/filtered.csv")

Set action_error when the action must fail. After the script finishes, the runtime uploads output files and stops Spark.

Outputs

Promote Outputs is not available on Big Data. The Outputs tab lists Action Outputs: Result and Error (Message, Traceback, Type).

To use a Big Data result in a later action, map a playbook property or expression that points at this action. Do not look for a Promote Outputs control.

How Big Data Differs from Script

Item	Big Data	Script
Add-panel action	Big Data	Script
Language	PySpark	Python 3 in a sandbox
Configure tabs	Inputs, Script, Outputs	Combined inputs and code, plus Test when available
Input types	Attachment only. Map with Add Playbook Property or Add Expression.	Schema types on the Script tab
Test tab	None	Yes — see the Script topic

What You See After Setup

Where	What you see
Canvas	A Big Data node. The Action panel shows type Big Data. Until you apply inputs, Inputs reads No inputs available.
Configure dialog	Tabs are Inputs, Script, and Outputs. Footer is Close and Apply. There is no Test tab.
Playbook run	The action runs PySpark on the mapped attachments. Downstream actions read this result through a playbook property expression. Confirm success or action_error in the run result.

Limits

Limit	What it means
Attachment inputs only	Add property offers Attachment only. Add Playbook Property lists file-type sources only.
No Test tab	You cannot dry-run the script in the dialog. Use a playbook run.
Not a Script mode	Do not look for a Big Data switch on Script.
Output files	Write files with helpers.csv.write, helpers.json.write, or output_attachment. Do not place input attachments in action_outputs.

Troubleshoot Big Data

What you see	What to do
The Big Data tile is missing	The Big Data native action is behind a feature flag and must be enabled; to enable this feature, contact Swimlane Support.
Start by adding an input or No inputs available	Add an Attachment and select Apply.
No file sources in Add Playbook Property	Map a file-type playbook property or expression. Other field types do not appear.
You expected a Test tab	That tab is on Script only. Run the playbook to verify Big Data.
The action fails after you put an input file in action_outputs	Write a new file with the helpers or output_attachment.