Guide to Worker Side Scripting
This introduction covers:
What is Worker side scripting?
What can you do with a Worker Side Script?
What are examples of why and when you use a Worker Side Script?
What's the difference between source items and stored items?
Where do I find further information on Worker Side Scripts?
What is Worker side scripting?
Worker side scripting allows code to be executed whilst it is being processed. This script is placed in the processing settings through Nuix Workstation's user interface or inside the processing settings as a configuration titled workerItemCallback.
Worker side scripting is designed to run from Worker servers, both remote and local. However, the script itself is run independently. This means if any temporary resources need to be made available, they must also be available where the Worker is running. Each Worker runs in isolation and is unable to communicate its script 'orders' with other Workers. If you require shared communication, you must implement this by other means.
This guide details how to use a Worker Side Script (WSS) to fine tune how items are handled to perform specific actions as a Worker server processes them. An item processed by a Worker Side Script is called a 'WorkerItem'. See WorkerItem methods in Ways to add and configure Worker Side Scripts for details.
What can you do with a Worker Side Script?
The following diagram lists the most often used examples of scripting use in Nuix Workstation.
You can use a WSS, in any of these ways, to run against data when processing items in a case:
Filter the data for the following:
Date span
Type
Textual or entity matches
Property or exif data
Relationships
Adjust processing options like storing binary or processing text
Hydrate the data, (that is, import the data into an object) by doing the following:
Conditionally assign a mime type setting based on the logic in your WSS.
Add extra annotations like tags, entities or custom metadata.
Supply additional properties or remove distracting properties.
Map or morph the data, by doing the following:
Move properties to communication data.
Convert data from one type to another (for example, from csv to phone call).
Modify text (update, remove, append, and so on).
Adjust the family of an item.
Report on the data, through detailed analysis and filtering whilst the data is processing.
See Example Ruby WSSs for specific tasks for the list of example scripts included in this guide.
What are examples of why and when you use a Worker Side Script?
To process countless timestamped CSV call records wanted for communication searches in an investigation, reduce the original CSV record count from millions to a few thousands, as follows:
Use one Worker Side Script to morph the properties in each CSV into appropriate communication fields and 'hydrate the data' by providing the phone numbers to aliases through an external source.
Run another Worker Side Script on the list of known 'exempt' numbers for internally owned numbers to exclude those numbers or aliases to obtain only contacts of external numbers not owned by staff members.
Two main ways to leverage worker side scripting during case processing are when you:
Create a targeted Case index
Worker side scripting allows you to dictate, as the data processing is in progress, exactly what items you should add into your case index and how. This means you can reduce the amount of data that needs to be processed into an indexed case file and ensure the items for analysis are the specific ones selected and are based on the scope of your investigation. This saves you time when responding to a data breach incident.
Perform data enrichment
When analysis is conducted, as an investigator, you often have a preferred list of scripts to run that carry out a range of secondary operations on the case data, and often involve enriching the data in the case. This means taking the information available in the case and using it to compute, translate, or decode further metadata that adds to the intelligence available in the case. Leveraging worker side scripting allows users to carry out these secondary operations at the point of initial data processing and enrich the data as part of the case creation. This reduces and removes the need for secondary operations on the case data, providing improved efficiency and saving the investigator time. Often these data-enrichment operations allow you, as an investigator, to better utilize the range of visualization features in Nuix Workstation, so further enhancing your analysis options.
What's the difference between source items and stored items?
The Nuix Engine processes Nuix Workstation's "source items" and from them, creates "stored items". The source item is a representation of an item in its original location, for example, an image file on a disk. The stored item is a representation of the metadata and information about the source data that is created when Nuix Workstation ingests and processes an item. The stored data may contain the following:
Text that has been extracted from the item.
Possible relationships to other items in the case.
User applied metadata (for example: tags, custodians, notes, and printed images of the item).
When performing OCR any extracted text is saved with the stored item and does not modify the source item associated with any text. Therefore, when performing a reload of items in a case, if you want to view and use any text associated with an item, you have to use the stored item version of it.
You provide access to the stored item from a WSS, which also allows you to access other fields that the stored item has. The API for the worker_item parameter (a reference to the item being processed) allows the following:
Text to be added/appended to an item A custodian to be set
Tagging
All these things can be conditionally done to an item. The scripts in Example Ruby WSSs for specific tasks provide examples of what can be done.
Where do I find further information on Worker Side Scripts?
For extra information on Worker Side Scripts, consult the following: Nuix Workstation's Help menu, Script option.
These blogs on the Nuix website:
Getting Started with Scripting on the Nuix Engine – Part 1
Getting Started with Scripting on the Nuix Engine – Part-2
The Nuix SDK site is a source of scripts and examples created and maintained by Nuix.
This GitHub site does not require a login.