Databricks

Databricks is a cloud-based data platform that provides a collaborative environment for big data analytics and machine learning.

Databricks is a unified data analytics platform, known for its ability to process large volumes of data and perform complex analytics. The Databricks Turbine Connector allows users to execute SQL queries directly on Databricks clusters or SQL warehouses, seamlessly integrating with Swimlane Turbine's security automation workflows. This integration empowers security teams to automate data-driven decision-making processes, enhancing their ability to respond to security incidents with data-backed insights and reducing manual intervention.

Limitations

None to date.

Supported Versions

This Databricks connector uses the latest Version.

Prerequisites

Before you can use the Databricks connector for Turbine, you'll need access to the Databricks API. This requires the following:

Custom authentication using the following parameters:

Server Hostname: The hostname of your Databricks server.

Client ID: The client identifier for authentication.

Client Secret: The secret key associated with the client ID.

HTTP Path: The HTTP path to connect to your Databricks instance.

Authentication Methods

This authentication with the following parameters:

Server Hostname: The address of your Databricks server.

Client ID: Your Databricks client identifier.

Client Secret: A secret key associated with your client ID for authentication.

HTTP Path: The HTTP path to the target server.

To check Server Hostname and HTTP Path Click Here.

To create Client ID and Client Secret Click Here.

Capabilities

This Databricks Connector provides the following capabilities:

Run Query

Run Query

This action call the Databricks SQL Connector for Python to run a basic SQL command on a cluster or SQL warehouse Click Here.

Additional Documentation

Databricks Connector Documentation

Databricks API Documentation

Configurations

Databricks Authentication

Authenticates using client ID and client secret.

Configuration Parameters

Parameter	Description	Type	Required
server_hostname	A hostname to the target server.	string	Required
client_id	The client ID to use for authentication.	string	Required
client_secret	The client secret to use for authentication.	string	Required
http_path	The HTTP path to the target server.	string	Required
verify_ssl	Verify SSL certificate.	boolean	Optional
http_proxy	A proxy to route requests through.	string	Optional

Actions

Run Query

Execute a specified SQL query on a Databricks cluster or SQL warehouse. Requires an input SQL query.

Endpoint

Method: GET

Input

Argument Name	Type	Required	Description
sql_query	string	Required	The SQL query to be executed. This can be any valid SQL command, such as SELECT, INSERT, UPDATE, DELETE, etc. The query should be a string and should not contain any special characters or formatting.

Input Example

{"sql_query":"SELECT * FROM samples.nyctaxi.trips LIMIT 2"}

Output

Parameter	Type	Description
result	array	Array of query result rows (one object per row). Omitted or empty when the query failed; see error instead.
result.tpep_pickup_datetime	string	Result of the operation
result.tpep_dropoff_datetime	string	Result of the operation
result.trip_distance	number	Result of the operation
result.fare_amount	number	Result of the operation
result.pickup_zip	number	Result of the operation
result.dropoff_zip	number	Result of the operation
error	object	Present when the query failed. Contains the SQL or connection error message for user-friendly display (e.g. "Your SQL syntax is incorrect").
error.message	string	Human-readable error message from Databricks or the connector.
error.type	string	Exception type (e.g. ServerOperationError, RequestError).
error.context	object	Optional extra context from the driver (e.g. diagnostic-info).

Output Example

{"result":[{"tpep_pickup_datetime":"2016-02-13T21:47:53Z","tpep_dropoff_datetime":"2016-02-13T21:57:15Z","trip_distance":1.4,"fare_amount":8,"pickup_zip":10103,"dropoff_zip":10110},{"tpep_pickup_datetime":"2016-02-13T18:29:09Z","tpep_dropoff_datetime":"2016-02-13T18:37:23Z","trip_distance":1.31,"fare_amount":7.5,"pickup_zip":10023,"dropoff_zip":10023}]}

Response Headers

Header	Description	Example
Content-Type	The media type of the resource	application/json
Date	The date and time at which the message was originated	Thu, 01 Jan 2024 00:00:00 GMT