urlscan.io Analysis

Urlscan.io is a web service that analyzes websites and provides detailed reports on their behavior.

Urlscan.io is a web security analysis platform that allows users to scan and analyze websites for potential threats and vulnerabilities. The urlscan.io Analysis connector enables Swimlane Turbine users to automate the retrieval of detailed analysis results, DOM snapshots, and screenshots of web submissions. Additionally, users can monitor their account's quota usage and execute advanced searches on urlscan.io records. This integration enhances security operations by providing comprehensive web analysis capabilities, allowing for efficient threat detection and response within the Swimlane Turbine platform.

Prerequisites

Before you can use the urlscan.io Analysis connector for Turbine, you'll need access to the urlscan.io API. This requires the following:

an API key authentication using the following parameters:

URL: The endpoint for accessing urlscan.io services.

API Key: A unique key provided by urlscan.io to authenticate requests.

Connector Setup

Obtaining an API token:

Navigate to your user profile

Click Settings & API.

Click Create an API key

Capabilities

The urlscan.io connector has the following capabilities:

Submit URLs

Get information about URL

Search URLS for full info

Search URLS for minimal info

Get Quota Details

Notes

Quota Information Documentation

This connector was last tested against API version: 1

Configurations

API Key Authentication

Authenticates using an API Key

Configuration Parameters

Parameter	Description	Type	Required
url	A URL to the target host.	string	Required
API-Key	API key	string	Required
verify_ssl	Verify SSL certificate	boolean	Optional
http_proxy	A proxy to route requests through.	string	Optional

Actions

Get Result

Retrieve the analysis results for a specific submission using its unique identifier (UUID) from Urlscan.io.

Endpoint

URL: api/v1/result/{{uuid}}

Method: GET

Input

Argument Name	Type	Required	Description
path_parameters.uuid	string	Required	Parameters for the Get Result action

Input Example

{"path_parameters":{"uuid":"942a4e6d-6c30-421a-892b-30a5813e0ab0"}}

Output

Parameter	Type	Description
status_code	number	HTTP status code of the response
reason	string	Response reason phrase
data	object	Response data
data.requests	array	Response data
data.requests.request	object	Response data
data.requests.request.requestId	string	Response data
data.requests.request.loaderId	string	Response data
data.requests.request.documentURL	string	Response data
data.requests.request.request	object	Response data
data.requests.request.request.url	string	Response data
data.requests.request.request.method	string	Response data
data.requests.request.request.headers	object	Response data
data.requests.request.request.mixedContentType	string	Response data
data.requests.request.request.initialPriority	string	Response data
data.requests.request.request.referrerPolicy	string	Response data
data.requests.request.request.isSameSite	boolean	Response data
data.requests.request.timestamp	number	Response data
data.requests.request.wallTime	number	Response data
data.requests.request.initiator	object	Response data
data.requests.request.initiator.type	string	Response data
data.requests.request.redirectHasExtraInfo	boolean	Response data
data.requests.request.type	string	Response data
data.requests.request.frameId	string	Response data
data.requests.request.hasUserGesture	boolean	Response data
data.requests.requests	array	Response data

Output Example

{"status_code":200,"response_headers":{"Server":"nginx","Date":"Fri, 21 Oct 2022 17:17:38 GMT","Content-Type":"application/json; charset=utf-8","Transfer-Encoding":"chunked","Connection":"keep-alive","X-Rate-Limit-Scope":"user","X-Rate-Limit-Action":"retrieve","X-Rate-Limit-Window":"minute","X-Rate-Limit-Limit":"120","X-Rate-Limit-Remaining":"120","X-Rate-Limit-Reset":"2022-10-21T17:18:00.000Z","X-Rate-Limit-Reset-After":"21","Cache-Control":"public, max-age=60","ETag":"W/\"e0543-gWHtF0zjquwaWU7...

Get Result Dom Snapshot

Retrieve the Document Object Model (DOM) snapshot for a given submission in urlscan.io using the unique identifier (UUID).

Endpoint

URL: dom/{{uuid}}

Method: GET

Input

Argument Name	Type	Required	Description
path_parameters.uuid	string	Required	Parameters for the Get Result Dom Snapshot action

Input Example

{"path_parameters":{"uuid":"942a4e6d-6c30-421a-892b-30a5813e0ab0"}}

Output

Parameter	Type	Description
status_code	number	HTTP status code of the response
reason	string	Response reason phrase
response_text	string	Output field: response_text

Output Example

{"status_code":200,"response_headers":{"Server":"nginx","Date":"Fri, 21 Oct 2022 20:50:50 GMT","Transfer-Encoding":"chunked","Connection":"keep-alive","Cache-Control":"max-age=3600, public","Etag":"\"b562ca8316f97fa9c32eb5f914f12aef\"","Last-Modified":"Fri, 21 Oct 2022 17:11:41 GMT","Expires":"Fri, 21 Oct 2022 21:50:50 GMT","Content-Security-Policy":"default-src 'self' data: ; script-src 'self' data: developers.google.com www.goo...","Referrer-Policy":"unsafe-url","Strict-Transport-Security":"ma...

Get Result Screenshot

Retrieve a screenshot from a specific urlscan.io submission using the unique identifier (UUID).

Endpoint

URL: screenshots/{{uuid}}.png

Method: GET

Input

Argument Name	Type	Required	Description
path_parameters.uuid	string	Required	Parameters for the Get Result Screenshot action

Input Example

{"path_parameters":{"uuid":"942a4e6d-6c30-421a-892b-30a5813e0ab0"}}

Output

Parameter	Type	Description
file	object	Result Screenshot
file.file_name	string	Name of the resource
file.file	string	Output field: file.file

Output Example

{"file":{"file_name":"Example Name","file":"string"}}

Quotas and Rate Limiting

Retrieve the current quota usage and rate limit information for your Urlscan.io account.

Endpoint

URL: user/quotas

Method: GET

Output

Parameter	Type	Description
status_code	number	HTTP status code of the response
reason	string	Response reason phrase
source	string	Output field: source
limits	object	Output field: limits
limits.private	object	Output field: limits.private
limits.private.day	object	Output field: limits.private.day
limits.private.day.limit	number	Output field: limits.private.day.limit
limits.private.day.used	number	Output field: limits.private.day.used
limits.private.day.remaining	number	Output field: limits.private.day.remaining
limits.private.day.percent	number	Output field: limits.private.day.percent
limits.private.hour	object	Output field: limits.private.hour
limits.private.hour.limit	number	Output field: limits.private.hour.limit
limits.private.hour.used	number	Output field: limits.private.hour.used
limits.private.hour.remaining	number	Output field: limits.private.hour.remaining
limits.private.hour.percent	number	Output field: limits.private.hour.percent
limits.private.minute	object	Output field: limits.private.minute
limits.private.minute.limit	number	Output field: limits.private.minute.limit
limits.private.minute.used	number	Output field: limits.private.minute.used
limits.private.minute.remaining	number	Output field: limits.private.minute.remaining
limits.private.minute.percent	number	Output field: limits.private.minute.percent
limits.public	object	Output field: limits.public
limits.public.day	object	Output field: limits.public.day
limits.public.day.limit	number	Output field: limits.public.day.limit
limits.public.day.used	number	Output field: limits.public.day.used
limits.public.day.remaining	number	Output field: limits.public.day.remaining

Output Example

{"status_code":200,"response_headers":{"Server":"nginx","Date":"Fri, 21 Oct 2022 17:06:58 GMT","Content-Type":"application/json; charset=utf-8","Transfer-Encoding":"chunked","Connection":"keep-alive","Vary":"Accept","ETag":"W/\"ae0-aFYcnE0RNs8+abik6yuUcBAb9Lk\"","Content-Security-Policy":"default-src 'self' data: ; script-src 'self' data: developers.google.com www.goo...","Referrer-Policy":"unsafe-url","Strict-Transport-Security":"max-age=63072000; includeSubdomains; preload","X-Content-Type-Opt...

Search

Execute an ElasticSearch query on urlscan.io records to retrieve relevant analysis data. Requires a query parameter 'q'.

Endpoint

URL: api/v1/search

Method: GET

Input

Argument Name	Type	Required	Description
parameters.q	string	Required	Parameters for the Search action
parameters.size	number	Optional	Parameters for the Search action
parameters.search_after	string	Optional	Parameters for the Search action

Input Example

{"parameters":{"q":"*","size":100,"search_after":"1666385691272,7b659f9d-e509-4bb4-b07e-eda8ff6b0bc8"}}

Output

Parameter	Type	Description
status_code	number	HTTP status code of the response
reason	string	Response reason phrase
results	array	Result of the operation
results.task	object	Result of the operation
results.task.visibility	string	Result of the operation
results.task.method	string	HTTP method to use
results.task.domain	string	Result of the operation
results.task.apexDomain	string	Result of the operation
results.task.time	string	Result of the operation
results.task.uuid	string	Unique identifier
results.task.url	string	URL endpoint for the request
results.stats	object	Result of the operation
results.stats.uniqIPs	number	Result of the operation
results.stats.uniqCountries	number	Result of the operation
results.stats.dataLength	number	Response data
results.stats.encodedDataLength	number	Response data
results.stats.requests	number	Result of the operation
results.page	object	Result of the operation
results.page.country	string	Result of the operation
results.page.redirected	string	Result of the operation
results.page.ip	string	Result of the operation
results.page.mimeType	string	Type of the resource
results.page.title	string	Result of the operation
results.page.url	string	URL endpoint for the request
results.page.tlsValidDays	number	Unique identifier

Output Example

{"status_code":200,"response_headers":{"Server":"nginx","Date":"Fri, 21 Oct 2022 21:04:12 GMT","Content-Type":"application/json; charset=utf-8","Transfer-Encoding":"chunked","Connection":"keep-alive","X-Rate-Limit-Scope":"user","X-Rate-Limit-Action":"search","X-Rate-Limit-Window":"day","X-Rate-Limit-Limit":"1000","X-Rate-Limit-Remaining":"999","X-Rate-Limit-Reset":"2022-10-22T00:00:00.000Z","X-Rate-Limit-Reset-After":"10547","ETag":"W/\"243e9-rKQP/slcs/Yu8hGkeOCnO5+gtDw\"","Content-Security-Poli...

Submit

Submit a URL to urlscan.io for scanning and analysis, returning the scan's unique identifier. Requires 'url' and 'visibility' parameters.

Endpoint

URL: api/v1/scan

Method: POST

Input

Argument Name	Type	Required	Description
url	string	Optional	URL endpoint for the request
visibility	string	Optional	Parameter for Submit
tags	array	Optional	Parameter for Submit
customagent	string	Optional	Parameter for Submit
referer	string	Optional	Parameter for Submit
overrideSafety	string	Optional	Unique identifier
country	string	Optional	Parameter for Submit

Input Example

{"json_body":{"url":"swimlane.com","visibility":"public","tags":["tag1","tag2"],"customagent":"","referer":"","overrideSafety":"","country":"US"}}

Output

Parameter	Type	Description
status_code	number	HTTP status code of the response
reason	string	Response reason phrase
message	string	Response message
uuid	string	Unique identifier
result	string	Result of the operation
api	string	Output field: api
visibility	string	Output field: visibility
options	object	Output field: options
url	string	URL endpoint for the request

Output Example

{"status_code":200,"response_headers":{"Server":"nginx","Date":"Fri, 21 Oct 2022 17:11:33 GMT","Content-Type":"application/json; charset=utf-8","Transfer-Encoding":"chunked","Connection":"keep-alive","X-Rate-Limit-Scope":"user","X-Rate-Limit-Action":"public","X-Rate-Limit-Window":"minute","X-Rate-Limit-Limit":"60","X-Rate-Limit-Remaining":"60","X-Rate-Limit-Reset":"2022-10-21T17:12:00.000Z","X-Rate-Limit-Reset-After":"26","Vary":"Accept","ETag":"W/\"148-etytSyFmloAYNcETbYm/arVpacQ\"","Content-Se...

Response Headers

Header	Description	Example
Access-Control-Allow-Headers	HTTP response header: Access-Control-Allow-Headers	DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type
Access-Control-Allow-Methods	HTTP response header: Access-Control-Allow-Methods	GET, OPTIONS
Access-Control-Allow-Origin	HTTP response header: Access-Control-Allow-Origin	*
Access-Control-Max-Age	HTTP response header: Access-Control-Max-Age	1728000
Cache-Control	Directives for caching mechanisms	max-age=3600, public
Connection	HTTP response header: Connection	keep-alive
Content-Encoding	HTTP response header: Content-Encoding	gzip
Content-Security-Policy	HTTP response header: Content-Security-Policy	default-src 'self' data: ; script-src 'self' data: developers.google.com www.google.com www.gstatic.com https://js.hs-scripts.com https://js.hsleadflows.net https://js.hs-banner.com https://js.hsadspixel.net https://js.hubspotfeedback.com https://js.usemessages.com https://js.hs-analytics.net https://js.hscollectedforms.net https://js.hsforms.net https://js-na1.hs-scripts.com https://forms.hsforms.com; style-src 'self' 'unsafe-inline' fonts.googleapis.com www.google.com; img-src * data: .hubspot.com cdn2.hubspot.net forms.hsforms.com; font-src 'self' fonts.gstatic.com; child-src 'self' app.hubspot.com forms.hsforms.com js.hsadspixel.net js.hscollectedforms.net js.usemessages.com; frame-src https://www.google.com/recaptcha/ .hubspot.com forms.hsforms.com js.hsadspixel.net js.hscollectedforms.net js.usemessages.com; form-action 'self' forms.hsforms.com forms.hubspot.com; connect-src 'self' *.hubspot.com api.hubapi.com js.usemessages.com js.hsleadflows.net js.hs-banner.com js.hubspotfeedback.com js.hsadspixel.net js.hs-analytics.net js.hs-scripts.com forms.hsforms.com; upgrade-insecure-requests; frame-ancestors 'none'
Content-Type	The media type of the resource	application/json; charset=utf-8
Date	The date and time at which the message was originated	Fri, 21 Oct 2022 17:17:38 GMT
ETag	An identifier for a specific version of a resource	W/"148-etytSyFmloAYNcETbYm/arVpacQ"
Expires	The date/time after which the response is considered stale	Fri, 21 Oct 2022 21:50:50 GMT
Last-Modified	The date and time at which the origin server believes the resource was last modified	Fri, 21 Oct 2022 17:11:41 GMT
Referrer-Policy	HTTP response header: Referrer-Policy	unsafe-url
Server	Information about the software used by the origin server	nginx
Strict-Transport-Security	HTTP response header: Strict-Transport-Security	max-age=63072000; includeSubdomains; preload
Transfer-Encoding	HTTP response header: Transfer-Encoding	chunked
Vary	HTTP response header: Vary	Accept
X-Content-Type-Options	HTTP response header: X-Content-Type-Options	nosniff
X-Frame-Options	HTTP response header: X-Frame-Options	DENY
X-Proxy-Cache	HTTP response header: X-Proxy-Cache	MISS
X-Rate-Limit-Action	HTTP response header: X-Rate-Limit-Action	search
X-Rate-Limit-Limit	HTTP response header: X-Rate-Limit-Limit	60
X-Rate-Limit-Remaining	HTTP response header: X-Rate-Limit-Remaining	999
X-Rate-Limit-Reset	HTTP response header: X-Rate-Limit-Reset	2022-10-22T00:00:00.000Z
X-Rate-Limit-Reset-After	HTTP response header: X-Rate-Limit-Reset-After	26
X-Rate-Limit-Scope	HTTP response header: X-Rate-Limit-Scope	user
X-Rate-Limit-Window	HTTP response header: X-Rate-Limit-Window	minute
X-Robots-Tag	HTTP response header: X-Robots-Tag	all
X-XSS-Protection	HTTP response header: X-XSS-Protection	0