Understanding Swimlane Plugin collections in MongoDB

In this article, we’ll learn how a plugin’s data is stored in MongoDB and how this data is used in Swimlane. For illustrative purposes, we’ll use the sw_virus_total plugin for all examples given below.

Table of Contents:

Package Descriptors

Asset Descriptors

Available Action Descriptors

Assets

Files and Chunks

Tasks

Package Descriptors

This collection has one document related to the sw_virus_total plugin. The document includes basic information about the plugin including readme (overview) and changelog data.

MongoDB

Following is sample output from the 'PackageDescriptors' collection:

db.getCollection('PackageDescriptors').find({"_id":"sw_virus_total"})

Swimlane Application UI

The data is displayed in the “INTEGRATIONS” section in “PLUGINS → OVERVIEW” and “PLUGINS → CHANGE LOG”:

Asset Descriptors

This collection has one document related to the sw_virus_total plugin. The document includes information on asset definition.

MongoDB

Following is sample output from the 'AssetDescriptors' collection:

db.getCollection('AssetDescriptors').find({"PackageDescriptor.Name":"sw_virus_total"})

Swimlane Application UI

The data is displayed in the “INTEGRATIONS” section in “PLUGINS → ASSETS”:

Available Action Descriptors

This collection contains as many documents as the number of task actions available specifically for the sw_virus_total plugin.

MongoDB

Following is a sample output of AvailableActionDescriptors collection:

db.getCollection("AvailableActionDescriptors").find({"PackageDescriptor.Name":"sw_virus_total"})

Filter by the Name field:

db.getCollection("AvailableActionDescriptors").find({"PackageDescriptor.Name":"sw_virus_total"}, {Name: 1})

Swimlane Application UI

The data is displayed in the “INTEGRATIONS” section under “PLUGINS → ACTIONS” and “TASKS → Create a Task”:

Assets

This collection contains as many documents (s) as the number of assets created for the sw_virus_total plugin. A plugin can be associated with multiple assets.

MongoDB

Following is a sample output of 'Assets' collection:

db.getCollection('Assets').find({"Descriptor.PackageDescriptor.Name": "sw_virus_total"})

Filter by the Name field:

db.getCollection('Assets').find({"Descriptor.PackageDescriptor.Name": "sw_virus_total"}, {Name:1})

Swimlane Application UI

The data is displayed in the “INTEGRATIONS” section under “ASSETS”:

Files and Chunks

Per MongoDB, “GridFS is a specification for storing and retrieving files that exceed the BSON-document size limit of 16 MB. Instead of storing a file in a single document, GridFS divides the file into parts, or chunks [1], and stores each chunk as a separate document.”

It is a file system to store files, but its data is stored within MongoDB collections. The collection, ‘fs.chunks’, stores the document parts while the collection, ‘fs.files’ stores the file’s metadata.

Swimlane uses this system to store the actual plugin file (.swimbundle) when it is first uploaded in Swimlane. Upon restart of Swimlane, the plugins are re-installed into the containers using the file data available in these two collections.

MongoDB

Following is a sample output of ‘fs.files’ and ‘fs.chunks’ collection:

db.getCollection('fs.files').find({filename: /sw_virus_total/})

db.getCollection('fs.chunks').find({'files_id':'<_id>'})

Tasks

This collection contains all tasks of type Email Import, Powershell Core, Python 3.6, Python 2.7, Network File, API, plugin-based task action (non-forked), and plugin-based task action (forked).

MongoDB

The following query will list all tasks that use the sw_virus_total plugin task actions:

db.getCollection('Tasks').find({"Action.Descriptor.PackageDescriptor.Name" : "sw_virus_total"})

Filter by Name and Action Type:

db.getCollection('Tasks').find({"Action.Descriptor.PackageDescriptor.Name" : "sw_virus_total"}, {Name:1, "Action._t":1})

The following query will list all tasks forked based on the sw_virus_total plugin task actions:

db.getCollection('Tasks').find({"Action.ForkedFromPackage" : "sw_virus_total"})

Filter by Name and Action Type:

db.getCollection('Tasks').find({"Action.ForkedFromPackage" : "sw_virus_total"}, {Name:1, "Action._t":1})

Related Queries: Return a list of installed plugins:

db.getCollection('PackageDescriptors').distinct('Name')

Find all the tasks associated with a particular assetId:

db.getCollection('Tasks').find({'Action.AssetId': '<assetId>'})