Elastic Search V8

elastic elasticsearch 8 the elastic search v8 connector enables seamless integration with swimlane turbine, allowing users to manage and interact with elastic search indices and documents directly within security automation workflows elastic search v8 is a powerful search and analytics engine that enables real time indexing, searching, and data analysis the elastic search v8 connector for swimlane turbine allows users to add, update, and delete documents within indices, as well as perform detailed searches to extract valuable insights by integrating with swimlane turbine, security teams can automate the ingestion and querying of data, enhancing threat detection, incident response, and security monitoring capabilities this connector streamlines complex data operations, making it easier for users to manage and analyze large volumes of security data without the need for extensive coding prerequisites to utilize the elastic search v8 connector with swimlane turbine, ensure you have the following prerequisites http basic authentication with the following parameters url the endpoint url for your elastic search v8 instance username your elastic search v8 username password your elastic search v8 password search action if the elasticsearch security features are enabled, you must have the read https //www elastic co/guide/en/elasticsearch/reference/current/security privileges html#privileges list indices for the target data stream, index, or alias for cross cluster search, see https //www elastic co/guide/en/elasticsearch/reference/current/remote clusters privileges html#remote clusters privileges ccs to search a https //www elastic co/guide/en/elasticsearch/reference/current/point in time api html for an alias, you must have the read index privilege for the alias’s data streams or indices add document action if the elasticsearch security features are enabled, you must have the create doc , create , index , or write https //www elastic co/guide/en/elasticsearch/reference/current/security privileges html#privileges list indices capabilities this connector provides the following capabilities search data stored in elasticsearch indices and data streams add a json document to the specified data stream or index delete a json document by index and id update by query api actions setup add document please see the https //www elastic co/guide/en/elasticsearch/reference/current/api conventions html#time units if you want to use the timeout configuration notes https //www elastic co/guide/en/elasticsearch/reference/current/docs index html https //www elastic co/guide/en/elasticsearch/reference/current/search html https //www elastic co/guide/en/elasticsearch/reference/current/api conventions html#time units https //www elastic co/guide/en/elasticsearch/reference/current/security privileges html#privileges list indices https //www elastic co/guide/en/elasticsearch/reference/current/remote clusters privileges html#remote clusters privileges ccs https //www elastic co/guide/en/elasticsearch/reference/current/point in time api html this connector was last tested against product version 8 2 2 configurations http basic authentication authenticates using username and password configuration parameters parameter description type required url a url to the target host string required username username string required password password string required verify ssl verify ssl certificate boolean optional http proxy a proxy to route requests through string optional actions add document adds or updates a document in an elastic search v8 index, making it searchable using specified 'data', 'index', and 'id' endpoint url /{{index}}/ doc/{{id}} method post input argument name type required description parameters if seq no string optional only perform the operation if the document has this sequence number parameters if primary term string optional only perform the operation if the document has this primary term parameters op type string optional set to create to only index the document if it does not already exist parameters pipeline string optional id of the pipeline to use to preprocess incoming documents parameters refresh string optional if true, elasticsearch refreshes the affected shards to make this operation visible to search, if wait for then wait for a refresh to make this operation visible to search, if false do nothing with refreshes parameters routing string optional custom value used to route operations to a specific shard parameters timeout string optional defaults to 1m (one minute) this guarantees elasticsearch waits for at least the timeout before failing parameters version string optional explicit version number for concurrency control the specified version must match the current version of the document for the request to succeed parameters version type string optional specific version type such as external, external gte parameters wait for active shards string optional the number of shard copies that must be active before proceeding with the operation set to all or any positive integer up to the total number of shards in the index (number of replicas+1) parameters require alias string optional if true, the destination must be an index alias path parameters index string required parameters for the add document action path parameters id string required parameters for the add document action data object optional data input example {"path parameters" {"index" "metrics endpoint metadata current default"}} output parameter type description status code number http status code of the response reason string response reason phrase index string output field index id string unique identifier version number output field version result string result of the operation shards object output field shards shards total number output field shards total shards successful number whether the operation was successful shards failed number output field shards failed seq no number output field seq no primary term number output field primary term output example {"status code" 201,"response headers" {"location" "/metrics endpoint metadata current default/ doc/ov7ioiqbipfzuowyqfzk","x elastic product" "elasticsearch","content type" "application/json","content length" "195"},"reason" "created","json body" {" index" "metrics endpoint metadata current default"," id" "ov7ioiqbipfzuowyqfzk"," version" 1,"result" "created"," shards" {"total" 1,"successful" 1,"failed" 0}," seq no" 627," primary term" 7}} delete document removes a specified json document from an elastic search v8 index using the provided index and document id endpoint url /{{index}}/ doc/{{id}} method delete input argument name type required description parameters if seq no string optional only perform the operation if the document has this sequence number parameters if primary term string optional only perform the operation if the document has this primary term parameters refresh string optional if true, elasticsearch refreshes the affected shards to make this operation visible to search, if wait for then wait for a refresh to make this operation visible to search, if false do nothing with refreshes parameters routing string optional custom value used to route operations to a specific shard parameters timeout string optional period to wait for active shards defaults to 1m (one minute) parameters version string optional explicit version number for concurrency control the specified version must match the current version of the document for the request to succeed parameters version type string optional specific version type parameters wait for active shards string optional the number of shard copies that must be active before proceeding with the operation path parameters index string required name of the target index path parameters id string required parameters for the delete document action input example {"path parameters" {"index" "metrics endpoint metadata current default"}} output parameter type description status code number http status code of the response reason string response reason phrase index string output field index id string unique identifier version number output field version result string result of the operation shards object output field shards shards total number output field shards total shards successful number whether the operation was successful shards failed number output field shards failed seq no number output field seq no primary term number output field primary term output example {"status code" 200,"response headers" {"x elastic product" "elasticsearch","content type" "application/json","content length" "195"},"reason" "ok","json body" {" index" "metrics endpoint metadata current default"," id" "ov7ioiqbipfzuowyqfzk"," version" 2,"result" "deleted"," shards" {"total" 1,"successful" 1,"failed" 0}," seq no" 628," primary term" 7}} search performs a search query on a specified index in elastic search v8 and returns matching hits requires path parameters and json body endpoint url /{{index}}/ search method get input argument name type required description parameters size string optional defines the number of records to return path parameters index string required parameters for the search action query object optional query object defines the search definition using the query dsl input example {"path parameters" {"index" "my index 000001"}} output parameter type description status code number http status code of the response reason string response reason phrase took number output field took timed out boolean output field timed out shards object output field shards shards total number output field shards total shards successful number whether the operation was successful shards skipped number output field shards skipped shards failed number output field shards failed hits object output field hits hits total object output field hits total hits total value number value for the parameter hits total relation string output field hits total relation hits max score number score value hits hits array output field hits hits hits hits index string output field hits hits index hits hits id string unique identifier hits hits score number score value hits hits source object output field hits hits source hits hits source container object output field hits hits source container hits hits source container id string unique identifier hits hits source agent object output field hits hits source agent hits hits source agent name string name of the resource hits hits source agent id string unique identifier hits hits source agent ephemeral id string unique identifier output example {"status code" 200,"response headers" {"x elastic product" "elasticsearch","content type" "application/json","content length" "14790"},"reason" "ok","json body" {"took" 2262,"timed out"\ false," shards" {"total" 228,"successful" 228,"skipped" 0,"failed" 0},"hits" {"total" {},"max score" 1,"hits" \[]}}} update by query applies updates to elastic search v8 documents matching a specified query or all documents if no query is provided, ideal for mapping changes endpoint url /{{target}}/ update by query method post input argument name type required description path parameters target string required target specifies the comma separated list of data streams, indices, and aliases to search within this parameter supports wildcards ( ) if omitted, or if the value is set to or all, the search will include all data streams and indices parameters allow no indices boolean optional if set to false, the request returns an error if any wildcard expression, index alias, or all value targets only missing or closed indices, even if other open indices are targeted for example, a request targeting foo ,bar returns an error if no index starts with bar, even if an index starts with foo parameters analyzer string optional this parameter can only be used when the q query string parameter is specified parameters analyze wildcard boolean optional this parameter can only be used when the q query string parameter is specified parameters conflicts string optional what to do if update by query hits version conflicts are abort or proceed parameters default operator string optional this parameter can only be used when the q query string parameter is specified parameters df string optional this parameter can only be used when the q query string parameter is specified parameters expand wildcards string optional type of index that wildcard patterns can match if the request can target data streams, this argument determines whether wildcard expressions match hidden data streams parameters ignore unavailable boolean optional if false, the request returns an error if it targets a missing or closed index parameters lenient boolean optional if true, format based query failures (such as providing text to a numeric field) in the query string will be ignored defaults to false parameters max docs number optional maximum number of documents to process parameters pipeline string optional id of the pipeline to use to preprocess incoming documents parameters preference string optional specifies the node or shard the operation should be performed on parameters q string optional query in the lucene query string syntax parameters request cache boolean optional if true, the request cache is used for this request parameters refresh boolean optional if true, elasticsearch refreshes affected shards to make the operation visible to search parameters requests per second number optional the throttle for this request in sub requests per second parameters routing string optional custom value used to route operations to a specific shard parameters scroll string optional period to retain the search context for scrolling parameters scroll size number optional size of the scroll request that powers the operation parameters search type string optional the type of the search operation parameters search timeout string optional explicit timeout for each search request parameters slices number optional the number of slices this task should be divided into parameters sort string optional a comma separated list of pairs parameters stats string optional specific tag of the request for logging and statistical purposes input example {"parameters" {"allow no indices"\ false,"analyzer" "standard","analyze wildcard"\ true,"conflicts" "abort","default operator" "or","df" "","expand wildcards" "open","ignore unavailable"\ false,"lenient"\ true,"max docs" 3,"pipeline" "","preference" "","q" "","request cache"\ true,"refresh"\ true,"requests per second" 1,"routing" "","scroll" "2d","scroll size" 1000,"search type" "query then fetch","search timeout" "2d","slices" 7,"sort" "","stats" "","terminate after" 3,"timeout" "2d","version"\ true,"wait for active shards" ""},"path parameters" {"target" "my index 000001"}} output parameter type description status code number http status code of the response reason string response reason phrase took number output field took timed out boolean output field timed out total number output field total updated number output field updated deleted number output field deleted batches number output field batches version conflicts number output field version conflicts noops number output field noops retries object output field retries retries bulk number output field retries bulk retries search number output field retries search throttled millis number output field throttled millis requests per second number output field requests per second throttled until millis number output field throttled until millis failures array output field failures failures file name string name of the resource failures file string output field failures file output example {"status code" 200,"response headers" {"content encoding" "gzip","content length" "182","content type" "application/json","x cloud request id" "tyt6p0gxqbijizqhv9 ea","x elastic product" "elasticsearch","x found handling cluster" "b4bc43b5ef0e44a4bc0181e4471683be","x found handling instance" "instance 0000000001","date" "mon, 03 jun 2024 10 35 31 gmt"},"reason" "ok","json body" {"took" 1588,"timed out"\ false,"total" 14207,"updated" 14207,"deleted" 0,"batches" 15,"version conflicts" 0,"noops" 0, response headers header description example content encoding http response header content encoding gzip content length the length of the response body in bytes 14790 content type the media type of the resource application/json date the date and time at which the message was originated mon, 03 jun 2024 10 35 31 gmt location the url to redirect a page to /metrics endpoint metadata current default/ doc/ov7ioiqbipfzuowyqfzk x cloud request id http response header x cloud request id tyt6p0gxqbijizqhv9 ea x elastic product http response header x elastic product elasticsearch x found handling cluster http response header x found handling cluster b4bc43b5ef0e44a4bc0181e4471683be x found handling instance http response header x found handling instance instance 0000000001