Databricks

the databricks connector allows users to execute sql queries on their databricks clusters or sql warehouses, facilitating advanced data analytics within the swimlane ecosystem databricks is a powerful data analytics platform that enables users to execute complex sql queries on massive datasets within a collaborative environment the databricks turbine connector allows users to perform real time data analysis by executing sql queries directly from the swimlane turbine platform this integration empowers security teams to leverage databricks' robust data processing capabilities, enhancing their ability to make data driven decisions and respond to security incidents with greater speed and precision by automating data queries and analysis, users can streamline their workflows, reduce manual effort, and focus on strategic tasks limitations none to date supported versions this databricks connector uses the latest version additional docs https //docs databricks com/aws/en/dev tools/python sql connector prerequisites to utilize the databricks connector within swimlane turbine, ensure you have the following prerequisites custom authentication with the following parameters server hostname the address of your databricks server instance client id your databricks application client identifier client secret the secret key associated with your databricks application http path the http path used for api requests to your databricks instance authentication methods this authentication with the following parameters server hostname the address of your databricks server client id your databricks client identifier client secret a secret key associated with your client id for authentication http path the http path to the target server to check server hostname and http path https //docs databricks com/aws/en/integrations/compute details to create client id and client secret https //docs databricks com/aws/en/dev tools/auth/oauth m2m capabilities this databricks connector provides the following capabilities run query run query this action call the databricks sql connector for python to run a basic sql command on a cluster or sql warehouse https //docs databricks com/aws/en/dev tools/python sql connector#query data configurations databricks authentication authenticates using client id and client secret configuration parameters parameter description type required server hostname a hostname to the target server string required client id the client id to use for authentication string required client secret the client secret to use for authentication string required http path the http path to the target server string required verify ssl verify ssl certificate boolean optional http proxy a proxy to route requests through string optional actions run query executes a specified sql query on a databricks cluster or sql warehouse using the databricks sql connector for python endpoint method get input argument name type required description sql query string required the sql query to be executed this can be any valid sql command, such as select, insert, update, delete, etc the query should be a string and should not contain any special characters or formatting input example {"sql query" "select from samples nyctaxi trips limit 2"} output parameter type description result array result of the operation result tpep pickup datetime string result of the operation result tpep dropoff datetime string result of the operation result trip distance number result of the operation result fare amount number result of the operation result pickup zip number result of the operation result dropoff zip number result of the operation output example {"result" \[{"tpep pickup datetime" "2016 02 13t21 47 53z","tpep dropoff datetime" "2016 02 13t21 57 15z","trip distance" 1 4,"fare amount" 8,"pickup zip" 10103,"dropoff zip" 10110},{"tpep pickup datetime" "2016 02 13t18 29 09z","tpep dropoff datetime" "2016 02 13t18 37 23z","trip distance" 1 31,"fare amount" 7 5,"pickup zip" 10023,"dropoff zip" 10023}]} response headers header description example content type the media type of the resource application/json date the date and time at which the message was originated thu, 01 jan 2024 00 00 00 gmt