Elastic Search V8

elastic elasticsearch 8 the elastic search v8 connector enables seamless integration with swimlane turbine, allowing users to manage and interact with elastic search indices and documents directly within security automation workflows elastic search v8 is a powerful search and analytics engine that enables real time indexing, searching, and data analysis the elastic search v8 connector for swimlane turbine allows users to add, update, and delete documents within indices, as well as perform detailed searches to extract valuable insights by integrating with swimlane turbine, security teams can automate the ingestion and querying of data, enhancing threat detection, incident response, and security monitoring capabilities this connector streamlines complex data operations, making it easier for users to manage and analyze large volumes of security data without the need for extensive coding prerequisites to utilize the elastic search v8 connector with swimlane turbine, ensure you have the following prerequisites http basic authentication with the following parameters url the endpoint url for your elastic search v8 instance username your elastic search v8 username password your elastic search v8 password search action if the elasticsearch security features are enabled, you must have the read index privilege https //www elastic co/guide/en/elasticsearch/reference/current/security privileges html#privileges list indices for the target data stream, index, or alias for cross cluster search, see configure privileges for cross cluster search https //www elastic co/guide/en/elasticsearch/reference/current/remote clusters privileges html#remote clusters privileges ccs to search a point in time (pit) https //www elastic co/guide/en/elasticsearch/reference/current/point in time api html for an alias, you must have the read index privilege for the alias’s data streams or indices add document action if the elasticsearch security features are enabled, you must have the create doc , create , index , or write index privilege https //www elastic co/guide/en/elasticsearch/reference/current/security privileges html#privileges list indices capabilities this connector provides the following capabilities search data stored in elasticsearch indices and data streams add a json document to the specified data stream or index delete a json document by index and id update by query api actions setup add document please see the time units convention https //www elastic co/guide/en/elasticsearch/reference/current/api conventions html#time units if you want to use the timeout configuration configurations http basic authentication authenticates using username and password configuration parameters parameter description type required url a url to the target host string required username username string required password password string required verify ssl verify ssl certificate boolean optional http proxy a proxy to route requests through string optional actions add document adds or updates a document in an elastic search v8 index, making it searchable using specified 'data', 'index', and 'id' endpoint url /{{index}}/ doc/{{id}} method post input argument name type required description data object required data if seq no string optional only perform the operation if the document has this sequence number if primary term string optional only perform the operation if the document has this primary term op type string optional set to create to only index the document if it does not already exist pipeline string optional id of the pipeline to use to preprocess incoming documents refresh string optional if true, elasticsearch refreshes the affected shards to make this operation visible to search, if wait for then wait for a refresh to make this operation visible to search, if false do nothing with refreshes routing string optional custom value used to route operations to a specific shard timeout string optional defaults to 1m (one minute) this guarantees elasticsearch waits for at least the timeout before failing version string optional explicit version number for concurrency control the specified version must match the current version of the document for the request to succeed version type string optional specific version type such as external, external gte wait for active shards string optional the number of shard copies that must be active before proceeding with the operation set to all or any positive integer up to the total number of shards in the index (number of replicas+1) require alias string optional if true, the destination must be an index alias index string required parameter for add document id string required unique identifier output parameter type description status code number http status code of the response reason string response reason phrase index string output field index id string unique identifier version number output field version result string result of the operation shards object output field shards total number output field total successful number whether the operation was successful failed number output field failed seq no number output field seq no primary term number output field primary term example \[ { "status code" 201, "response headers" { "location" "/metrics endpoint metadata current default/ doc/ov7ioiqbipfzuowyqfzk", "x elastic product" "elasticsearch", "content type" "application/json", "content length" "195" }, "reason" "created", "json body" { " index" "metrics endpoint metadata current default", " id" "ov7ioiqbipfzuowyqfzk", " version" 1, "result" "created", " shards" {}, " seq no" 627, " primary term" 7 } } ] delete document removes a specified json document from an elastic search v8 index using the provided index and document id endpoint url /{{index}}/ doc/{{id}} method delete input argument name type required description if seq no string optional only perform the operation if the document has this sequence number if primary term string optional only perform the operation if the document has this primary term refresh string optional if true, elasticsearch refreshes the affected shards to make this operation visible to search, if wait for then wait for a refresh to make this operation visible to search, if false do nothing with refreshes routing string optional custom value used to route operations to a specific shard timeout string optional period to wait for active shards defaults to 1m (one minute) version string optional explicit version number for concurrency control the specified version must match the current version of the document for the request to succeed version type string optional specific version type wait for active shards string optional the number of shard copies that must be active before proceeding with the operation index string required name of the target index id string required unique identifier output parameter type description status code number http status code of the response reason string response reason phrase index string output field index id string unique identifier version number output field version result string result of the operation shards object output field shards total number output field total successful number whether the operation was successful failed number output field failed seq no number output field seq no primary term number output field primary term example \[ { "status code" 200, "response headers" { "x elastic product" "elasticsearch", "content type" "application/json", "content length" "195" }, "reason" "ok", "json body" { " index" "metrics endpoint metadata current default", " id" "ov7ioiqbipfzuowyqfzk", " version" 2, "result" "deleted", " shards" {}, " seq no" 628, " primary term" 7 } } ] search performs a search query on a specified index in elastic search v8 and returns matching hits requires path parameters and json body endpoint url /{{index}}/ search method get input argument name type required description query object optional query object defines the search definition using the query dsl size string optional defines the number of records to return index string required parameter for search output parameter type description status code number http status code of the response reason string response reason phrase took number output field took timed out boolean output field timed out shards object output field shards total number output field total successful number whether the operation was successful skipped number output field skipped failed number output field failed hits object output field hits total object output field total value number value for the parameter relation string output field relation max score number score value hits array output field hits index string output field index id string unique identifier score number score value source object output field source container object output field container id string unique identifier agent object output field agent name string name of the resource id string unique identifier ephemeral id string unique identifier example \[ { "status code" 200, "response headers" { "x elastic product" "elasticsearch", "content type" "application/json", "content length" "14790" }, "reason" "ok", "json body" { "took" 2262, "timed out" false, " shards" {}, "hits" {} } } ] update by query applies updates to elastic search v8 documents matching a specified query or all documents if no query is provided, ideal for mapping changes endpoint url /{{target}}/ update by query method post input argument name type required description target string required target specifies the comma separated list of data streams, indices, and aliases to search within this parameter supports wildcards ( ) if omitted, or if the value is set to or all, the search will include all data streams and indices allow no indices boolean optional if set to false, the request returns an error if any wildcard expression, index alias, or all value targets only missing or closed indices, even if other open indices are targeted for example, a request targeting foo ,bar returns an error if no index starts with bar, even if an index starts with foo analyzer string optional this parameter can only be used when the q query string parameter is specified analyze wildcard boolean optional this parameter can only be used when the q query string parameter is specified conflicts string optional what to do if update by query hits version conflicts are abort or proceed default operator string optional this parameter can only be used when the q query string parameter is specified df string optional this parameter can only be used when the q query string parameter is specified expand wildcards string optional type of index that wildcard patterns can match if the request can target data streams, this argument determines whether wildcard expressions match hidden data streams ignore unavailable boolean optional if false, the request returns an error if it targets a missing or closed index lenient boolean optional if true, format based query failures (such as providing text to a numeric field) in the query string will be ignored defaults to false max docs number optional maximum number of documents to process pipeline string optional id of the pipeline to use to preprocess incoming documents preference string optional specifies the node or shard the operation should be performed on q string optional query in the lucene query string syntax request cache boolean optional if true, the request cache is used for this request refresh boolean optional if true, elasticsearch refreshes affected shards to make the operation visible to search requests per second number optional the throttle for this request in sub requests per second routing string optional custom value used to route operations to a specific shard scroll string optional period to retain the search context for scrolling scroll size number optional size of the scroll request that powers the operation search type string optional the type of the search operation search timeout string optional explicit timeout for each search request slices number optional the number of slices this task should be divided into sort string optional a comma separated list of pairs stats string optional specific tag of the request for logging and statistical purposes output parameter type description status code number http status code of the response reason string response reason phrase took number output field took timed out boolean output field timed out total number output field total updated number output field updated deleted number output field deleted batches number output field batches version conflicts number output field version conflicts noops number output field noops retries object output field retries bulk number output field bulk search number output field search throttled millis number output field throttled millis requests per second number output field requests per second throttled until millis number output field throttled until millis failures array output field failures file name string name of the resource file string output field file example \[ { "status code" 200, "response headers" { "content encoding" "gzip", "content length" "182", "content type" "application/json", "x cloud request id" "tyt6p0gxqbijizqhv9 ea", "x elastic product" "elasticsearch", "x found handling cluster" "b4bc43b5ef0e44a4bc0181e4471683be", "x found handling instance" "instance 0000000001", "date" "mon, 03 jun 2024 10 35 31 gmt" }, "reason" "ok", "json body" { "took" 1588, "timed out" false, "total" 14207, "updated" 14207, "deleted" 0, "batches" 15, "version conflicts" 0, "noops" 0, "retries" {}, "throttled millis" 0, "requests per second" 1, "throttled until millis" 0, "failures" \[] } } ] response headers header description example content encoding http response header content encoding gzip content length the length of the response body in bytes 182 content type the media type of the resource application/json date the date and time at which the message was originated mon, 03 jun 2024 10 35 31 gmt location the url to redirect a page to /metrics endpoint metadata current default/ doc/ov7ioiqbipfzuowyqfzk x cloud request id http response header x cloud request id tyt6p0gxqbijizqhv9 ea x elastic product http response header x elastic product elasticsearch x found handling cluster http response header x found handling cluster b4bc43b5ef0e44a4bc0181e4471683be x found handling instance http response header x found handling instance instance 0000000001 notes index api https //www elastic co/guide/en/elasticsearch/reference/current/docs index htmlsearch api https //www elastic co/guide/en/elasticsearch/reference/current/search htmltime units https //www elastic co/guide/en/elasticsearch/reference/current/api conventions html#time unitsindex privileges https //www elastic co/guide/en/elasticsearch/reference/current/security privileges html#privileges list indicesconfigure privileges for cross cluster search https //www elastic co/guide/en/elasticsearch/reference/current/remote clusters privileges html#remote clusters privileges ccspoint in time (pit) https //www elastic co/guide/en/elasticsearch/reference/current/point in time api html this connector was last tested against product version 8 2 2