Automation Overview

To trigger scripts and third-party programs from the BaseSpace Clarity LIMS user interface, use command-line calls configured in step automations.

There are many ways to use automation in Clarity LIMS. Consider the following examples:

• Automate sample tracking and enhance the information recorded.
• Generate and attach specially formatted text files.
• Simplify data entry.
• Automate the population or updating of data fields and data files.

Before using automation, become familiar with the topics discussed in this article and understand how automation functionality interacts with users, Clarity LIMS, and REST.

As of BaseSpace Clarity LIMS v5.0, several terms have been deprecated:

• External Program Integration Plug-in (EPP) has been replaced with automation.
• Automated Informatics (AI) node has been replaced with Automation Worker (AW) node.
• Parameter has been replaced with token.

Automation scripts are often used to create and attach specially formatted text files automatically, such as the following examples:

• Files containing sample lists (sometimes called instrument driver files), which users can import into control software to save time and ensure accurate sample processing.
• Barcode or label files, which are specially formatted files that can be supplied to barcode software systems to allow users to print container labels.
• Summary analysis results, for example, from alignment or molecular identification and quantification algorithms.

The two common applications of automation are:

• To create files and attach them to process output placeholders.
• To update data fields with information created during data analysis.

Scripts triggered by automation also use REST to update information directly within the REST resources.

For details, see REST Web Services and the version-specific documentation in the following sections:

• Automation/EPP Parameters (LIMS v4.x & earlier)
• Automation Tokens

Record lab work in Clarity LIMS by running steps on samples. These steps may be configured in Clarity LIMS by an administrator.

Most steps can be configured with an automation trigger that invokes an external script. The script may include fixed and variable information parameters/tokens on the command line.

As of BaseSpace Clarity LIMS v5.0, the term command-line parameter has been replaced with token.

After an AI/AW node is installed, processes (in Clarity LIMS v4.2 and earlier) or steps (Clarity LIMS v5 and later) must be configured to call out to it.

This configuration is executed by the Clarity LIMS administrator:

• In Clarity LIMS v4.2 and earlier, execute configuration in the Operations Interface process configuration dialog on the External Programs tab.
• In Clarity LIMS v5 and later, execute configuration on the Automation configuration screen.

When configuring an automation, the following values must be defined:

• Name
• Channel
• Command-line call
• Trigger style and location (see also Automation Triggers and Command Line Calls)

Only a brief summary of automation configuration is provided here. This material should be familiar from the Clarity LIMS administration training.

Scripts or third-party programs are called using the operating system command line. They must meet the following requirements:

• Be callable on the command line and, preferably, be able to read and respond to command-line parameters.
• Be accessed by the user account running the automation, with appropriate permissions and disk locations.
• Be able to exit with appropriate exit codes, otherwise the automation may record the script completed with an error.
• Step 1: A User Runs a Step
• Step 2: Server Creates the New Process/Step
• Step 3: The Automation Invokes the Operating System Shell Command
• Steps 4 to 7: The External Program Executes (Optionally Calling REST Resources)
• Step 8: The Automation Returns (Optionally Attaching Created Files)
• Step 9: The User Accesses Updated Results

The following diagram illustrates what happens when a step is run in Clarity LIMS.

The lab scientist tracks activities by running a step in Clarity LIMS. The step is configured to display a button that invokes the configured automation script.

As of BaseSpace Clarity LIMS v5.0, the term parameter has been replaced with token.

The application server creates a new step, which is much like POSTing to the processes resource of the REST API. The server resolves any parameters/tokens found in the string and sends the resolved command-line string to the automation.

In this example, two of the most common parameters/tokens used when working with automation are discussed:

• {processURI:version:scheme}—This parameter/token passes the REST API URI of the step that issued the command-line string.
• {ouputFileN}—This parameter/token passes the LIMS ID of the specified expected output file of the step that issued the command-line string. You can use 0 (zero) for the first file, 1 (one) for the second file, etc.

For more information about the parameters/tokens available for use, refer to the articles in the following API documentation sections:

• Automation/EPP Parameters (LIMS v4.x & earlier)
• Automation Tokens

As of API v1 r12, the version and scheme values for {processURI:version:scheme} are automatically populated, based on the REST version and protocol of the deployed server.

The automation program receives the command-line string from the application server. It may also receive other process(step)-related information, such as temporary files. The command-line string is executed by the operating system of the host computer.

The automation can work with any third-party program that supports command-line parameters/tokens. The program may simply create files or it may manipulate information directly via the REST API (steps 5 and 6).

Simple automation operation does not require anything of the REST API. If a third-party program creates files that users would like brought back into Clarity LIMS, scripts should use the outputFileN parameter/token to specify that the program create file names that are expected by the client. The files are placed in a temporary local working directory and automatically imported into the client. With this method, the automation automatically handles many of the things that would need manually scripts using the REST API processes, artifacts, files, and glsstorage resources.

For more complicated scenarios, you may want to use automation with the REST API. This situation is where the processURI parameter/token is used. A GET request on the URI of the step that issues the command-line string provides all the information recorded by the user. This information includes links to the analytes (samples) used as inputs. The script can then use other REST API resources to create or update information.

On completion, the third-party program exits (step 7). Standard shell exit codes apply: zero (0) equals successful completion.

On exit of the third-party program, the automation software updates the application server. If the system finds files with names that match the file placeholders produced by a process/step, the files are uploaded to the file server and attached to the appropriate placeholders.

A nonzero exit code sets a flag on the step, indicating that there is an error.

With updates complete, the application server sends refresh events to Clarity LIMS. The user sees that files have been uploaded.

Verifying and testing scripts is an important part of working with automation. Remember that there are three software components:

• The server
• The automation instance (calling scripts)
• The script

The best way to debug scripts is to unit test each component separately. For example, a logical order to work on a script is as follows.

1. Define and test the REST calls required in a web browser.
2. Define the command-line parameters/tokens sent to the automation at step completion.
3. Test the script running from the command line.
4. Test automation calls to the script at step completion by running the step from the Clarity LIMS interface.

Before the system can use the automation, a system administrator installs one or more automation workers/AI nodes within the lab network. The installation typically occurs on the server that contains the script program, or third-party application, to be integrated.

The installer program is contained in the Automated Informatics/Automation Worker software package.