Databricks

the databricks connector allows users to perform data analytics by executing sql queries directly from the swimlane platform databricks is a unified data analytics platform that accelerates innovation by unifying data science, engineering, and business the databricks connector for swimlane turbine allows users to execute sql queries directly on their databricks clusters or sql warehouses, streamlining data analysis and integration within security workflows by leveraging this connector, end users can automate data retrieval and manipulation tasks, enhancing their ability to respond to security incidents with data driven insights this integration simplifies complex data operations, making it accessible for security teams to incorporate advanced analytics into their automated playbooks without the need for specialized coding skills limitations none to date supported versions this databricks connector uses the latest version additional docs https //docs databricks com/aws/en/dev tools/python sql connector prerequisites to effectively utilize the databricks connector within the swimlane platform, ensure you have the following prerequisites custom authentication configured with the necessary parameters server hostname the address of your databricks server client id your databricks service client identifier client secret the secret key associated with your databricks client id http path the specific path for http requests within your databricks environment authentication methods this authentication with the following parameters server hostname the address of your databricks server client id your databricks client identifier client secret a secret key associated with your client id for authentication http path the http path to the target server to check server hostname and http path https //docs databricks com/aws/en/integrations/compute details to create client id and client secret https //docs databricks com/aws/en/dev tools/auth/oauth m2m capabilities this databricks connector provides the following capabilities run query run query this action call the databricks sql connector for python to run a basic sql command on a cluster or sql warehouse https //docs databricks com/aws/en/dev tools/python sql connector#query data configurations databricks authentication authenticates using client id and client secret configuration parameters parameter description type required server hostname a hostname to the target server string required client id the client id to use for authentication string required client secret the client secret to use for authentication string required http path the http path to the target server string required verify ssl verify ssl certificate boolean optional http proxy a proxy to route requests through string optional actions run query executes a specified sql query on a databricks cluster or sql warehouse, with the requirement of an input sql query endpoint method get input argument name type required description sql query string required the sql query to be executed this can be any valid sql command, such as select, insert, update, delete, etc the query should be a string and should not contain any special characters or formatting input example {"sql query" "select from samples nyctaxi trips limit 2"} output parameter type description result array array of query result rows (one object per row) omitted or empty when the query failed; see error instead result tpep pickup datetime string result of the operation result tpep dropoff datetime string result of the operation result trip distance number result of the operation result fare amount number result of the operation result pickup zip number result of the operation result dropoff zip number result of the operation error object present when the query failed contains the sql or connection error message for user friendly display (e g "your sql syntax is incorrect") error message string human readable error message from databricks or the connector error type string exception type (e g serveroperationerror, requesterror) error context object optional extra context from the driver (e g diagnostic info) output example {"result" \[{"tpep pickup datetime" "2016 02 13t21 47 53z","tpep dropoff datetime" "2016 02 13t21 57 15z","trip distance" 1 4,"fare amount" 8,"pickup zip" 10103,"dropoff zip" 10110},{"tpep pickup datetime" "2016 02 13t18 29 09z","tpep dropoff datetime" "2016 02 13t18 37 23z","trip distance" 1 31,"fare amount" 7 5,"pickup zip" 10023,"dropoff zip" 10023}]} response headers header description example content type the media type of the resource application/json date the date and time at which the message was originated thu, 01 jan 2024 00 00 00 gmt