Databricks

the databricks connector allows users to execute sql queries on their databricks clusters or sql warehouses, facilitating advanced data analytics within the swimlane ecosystem databricks is a powerful data analytics platform that enables users to execute complex sql queries on massive datasets within a collaborative environment the databricks turbine connector allows users to perform real time data analysis by executing sql queries directly from the swimlane turbine platform this integration empowers security teams to leverage databricks' robust data processing capabilities, enhancing their ability to make data driven decisions and respond to security incidents with greater speed and precision by automating data queries and analysis, users can streamline their workflows, reduce manual effort, and focus on strategic tasks limitations none to date supported versions this databricks connector uses the latest version additional docs databricks documentation https //docs databricks com/aws/en/dev tools/python sql connector prerequisites to utilize the databricks connector within swimlane turbine, ensure you have the following prerequisites custom authentication with the following parameters server hostname the address of your databricks server instance client id your databricks application client identifier client secret the secret key associated with your databricks application http path the http path used for api requests to your databricks instance authentication methods databricks custom authentication method this authentication with the following parameters server hostname the address of your databricks server client id your databricks client identifier client secret a secret key associated with your client id for authentication http path the http path to the target server to check server hostname and http path click here https //docs databricks com/aws/en/integrations/compute details to create client id and client secret click here https //docs databricks com/aws/en/dev tools/auth/oauth m2m capabilities this databricks connector provides the following capabilities run query run query this action call the databricks sql connector for python to run a basic sql command on a cluster or sql warehouse click here https //docs databricks com/aws/en/dev tools/python sql connector#query data configurations databricks authentication authenticates using client id and client secret configuration parameters parameter description type required server hostname a hostname to the target server string required client id the client id to use for authentication string required client secret the client secret to use for authentication string required http path the http path to the target server string required verify ssl verify ssl certificate boolean optional http proxy a proxy to route requests through string optional actions run query executes a specified sql query on a databricks cluster or sql warehouse using the databricks sql connector for python endpoint method get input argument name type required description sql query string required the sql query to be executed this can be any valid sql command, such as select, insert, update, delete, etc the query should be a string and should not contain any special characters or formatting output parameter type description result array result of the operation tpep pickup datetime string time value tpep dropoff datetime string time value trip distance number output field trip distance fare amount number output field fare amount pickup zip number output field pickup zip dropoff zip number output field dropoff zip example \[ { "result" \[ {}, {} ] } ]