Managing Configurations with Config Slicer

Option

Description

-a,--apiuri <apiuri>

The Clarity LIMS REST API base URI (ends in /api) (or --server must be provided)

-k,--package <package file>

File to be imported from or exported to (Required if operation is import, importAndOverwrite*, export, or validate). If file is not local, a full path is required.

-f,--force <force> *

Force update without prompt when running in importAndOverwrite mode (Optional)

-m,--manifest <manifest>

Manifest file (Required if operation is export or example). If file is not local, a full path is required.

-o,--operation <operation>

The operation mode for the Config Slicer tool.

Options are import, export, validate, example, importAndOverwrite, and custom (Required)

-p,--password <password>

In Clarity LIMS REST API:

•

for v5.3 use ENC <encrypted-password>

•

for v5.4 use <password>

-pr,--protocols <protocols>

The protocols to include in the custom manifest (Optional)

-pt,--processTypes <processTypes>

The process types to include in the custom manifest (Optional)

-s,--server <server>

The Clarity LIMS REST API server (or --apiuri must be provided)

-S,--Strict

Strict mode for import (fail fast - default mode is best-effort) (Optional)

-u,--username <username>

The Clarity LIMS REST API username (Required)

-w,--workflows <workflows>

The workflows to include in the custom manifest (Optional)

* The importAndOverwrite option lets Config Slicer update existing configuration, rather than creating a configuration. This option is only available in Clarity LIMS 3.4.

Usage

EXPORT: Creating a Configuration Package File

Step 1: Create Configuration Manifest File

Create Simple manifest file: or Custom manifest file: .

Simple manifest file:

On the source server, copy and paste following code to the command line. Edit the variables (version, server IP address, username, password, and manifest file name) to match those items in the system.

java -jar config-slicer-<version>.jar -o example -s <server> -u <username> -p <password> -m <manifest-filename>

A command with the variables filled in might look like this:

java -jar config-slicer-3.0.13.jar -o example -s glsserver -u admin -p glspass -m goldenconfig.txt

This step produces a manifest file containing information about the entire system configuration.

For best practice, copy this file and rename the copy in a way that reflects the configuration (we'll use newconfiguration.txt for this example). Use the copied file for the next steps.

A manifest file is used as an intermediary step to produce an XML configuration package file.

The manifest file is only relevant to the data that exists in the system at the time it is created. Discard it after creating the configuration package file or save the manifest file for historical auditing purposes. It can provide a record of a known working configuration set on a particular system.

Custom manifest file:

To create a manifest file for only specific workflows, protocols, or process types, follow the steps outlined previously, using -o custom instead of -o example.

•

When using this operation provide additional parameters (-w, -pr, and/or -pt), specify the exact entities for which to create a manifest.

–

For example:

java -jar config-slicer-<version>.jar -o custom -s <server> -u <username> -p <password>

-m <manifest-filename> -w <workflow name> -pr <protocol name> -pt <process-type name>

•

Specifying every option is not required. It is also possible to specify more than one of each kind. For example, create a manifest file for a workflow with the following command:

java -jar config-slicer-3.0.13.jar -o custom -s glsserver -u admin -p glspass

-m workflowManifest.txt -w 'Nextera DNA for HiSeq'

•

Or for two protocols like so:

java -jar config-slicer-3.0.13.jar -o custom -s glsserver -u admin -p glspass

-m protocolsManifest.txt -pr 'DNA Initial QC' -pr 'Nextera DNA Library Prep'

•

Or for two process types and a protocol with this command:

java -jar config-slicer-3.0.13.jar -o custom -s glsserver -u admin -p glspass

-m processTypesManifest.txt -pt 'Blood Extraction' -pt 'Qubit QC (DNA) 4.0' -pr 'Illumina SBS (HiSeq GAIIx)'

IMPORT: Installing a Configuration Package File

Step 1: Import Configuration Package File

•

On the destination server, copy and paste the following code to the command line. Edit the variables (version, package file name, server IP address, username, and password) as required.

java -jar config-slicer-<version>.jar -k <configuration-package-file>.xml -o import -s <server> -u <username> -p <password>

- or -

java -jar config-slicer-<version>.jar -o importAndOverwrite -k <configuration-package-file>.xml -s <server> -u <username> -p <password>

•

A command with the variables filled in might look like this example:

java -jar config-slicer-3.0.13.jar -o import -k newconfiguration.xml -s glsserver

-u admin -p glspass

- or -

java -jar config-slicer-3.0.13.jar -o importAndOverwrite -k newconfiguration.xml -s glsserver -u admin -p glspass

About Duplicate Entities

If any of the configuration entities that are about to import, exist in the destination system, Config Slicer either logs a warning or attempts to update them. It depends on the mode being run (see Import/Export Modes).

If Clarity LIMS has maintained an internal record of deleted items, the previous information may also apply to deleted entities. This situation may occur if those entities have created outputs that currently still exist in the system.

•

When running in import mode, entities that exist and are different from the version in the package have a warning and full diff logged.

•

When running in importAndOverwrite mode, Config Slicer attempts to update entities that exist and are different from the version in the package.

–

In this scenario, back up configuration package containing copies of the updated entities before they were changed is saved to the directory where the configuration package is located. If that directory is not writable, the backup package is saved to the same directory as the log file.

•

If the version in the package is identical to the version on the server, no errors are logged and Config Slicer considers that entity successfully imported.

To avoid changing existing configuration (which could possibly break historical data), another option is manually renaming the old entities. Add an extension or a prefix and continue with importing the new configuration package.

Step 2: Validate the Import

Use the following methods to validate whether an import has completed successfully:

Check the Import Log

For each specific type of configuration that is being imported (eg, container types, process types, workflows), Config Slicer logs a set of messages. The messages look similar to the following examples:

2015-01-16 18:45:13,046 INFO - Found 4 Container Types in configuration package, importing now.

2015-01-16 18:45:13,614 INFO - Summary of Container Types:

Newly imported:

Container Type 'BioAnalyzer DNA 1000 Chip'

Container Type 'BioAnalyzer DNA High Sensitivity Chip'

Container Type 'BioAnalyzer RNA Nano Chip'

Container Type 'BioAnalyzer RNA Pico Chip'

Imported and updated (diffs in log): None

Already existing and identical (no update performed): None

Not imported (due to errors or existing entities): None

•

Before it begins to process a specific entity, the file logs how many entities were found. Any errors or warnings about this set of entities always appear between Found 4 $Entities and Summary of $Entities.

•

Every entity that is found in the configuration package always appears in the summary, in one category or another. If a scenario occurs where this isn't true, or where the initial count of entities does not match the number in the summary, something has gone wrong and a bug report should be filed.

Validate with Config Slicer

Running Config Slicer with the validate operation checks every entity in the package to see if it exists on the destination server. It reports results in a format similar to the log format shown previously.

Run the validate operation before or after importing:

•

Before importing—checks if there could be any problems when importing a configuration from a package. This feature is its primary use as it makes sure that during import, "configuration exists in package but not on server" is not considered an error case.

•

After importing—makes sure that the results are what was expected.

Example of validate output:

2015-01-16 15:42:31,174 INFO - Found 4 Container Types in configuration package, validating now.

2015-01-16 15:42:31,706 INFO - Summary of Container Types:

Do not exist: None

Exist and are identical:

Container Type 'BioAnalyzer DNA 1000 Chip'

Container Type 'BioAnalyzer DNA High Sensitivity Chip'

Container Type 'BioAnalyzer RNA Nano Chip'

Container Type 'BioAnalyzer RNA Pico Chip'

Exist and are different (diffs in log): None

Check Configuration on the Destination Server

The ultimate test of whether configuration has imported successfully is to check the configuration on the destination server itself. Make sure it looks and behaves as expected.

Configuration can be checked either via the Configuration screen in the Clarity LIMS user interface, or via the configuration endpoints in the API.

Guidelines and Semantics for Creating Manifest Files

Top-level Entities

•

To be included in a configuration package file, the top-level entities of the custom configuration set must be explicitly enumerated.

•

Some of the top-level entities are discrete self-contained units and do not include other units (for example, container types, reagent types, artifact groups, and nonartifact/nonprocess type UDFs).

•

Some top-level entities (process types, for example) automatically include other units (see Non-Top-Level Entities for more information).

•

For process types, only configured processes, vanilla Transfer processes, Pool Samples (since 7.5), and Add Multiple Reagents (since 7.6) processes are supported. All process type details are exportable/importable.

Non-Top-Level Entities

•

Some entities are only included as part of other entities.

For example, process templates, process type UDFs, and artifact UDFs are only included when included in a top-level process type. (The latter is a special case, given that the same artifact UDF can be used by multiple process types.)

Required Entities

•

Some entities may be required by other entities. In these cases, make sure that these entities are exported/imported in the correct order.

–

For example, because process types may require the existence of a container type, create the container type first.

•

Required entities are not automatically included. If they do not exist in the destination system, explicitly include them in the manifest file.

–

For example, suppose that a process type declares a particular container type as a default output plate. If that container type does not exist in the destination system, include that container type in the manifest file.

Import/Export Modes

Import modes affect the transactability of the tool, allowing it to make incremental changes if errors occur or provide an all-or-none option. For example, use the operation validate mode to determine if errors were encountered.

Best-Effort Mode

•

This mode is the default import mode.

•

This mode attempts to import as many units as possible. Any failures are logged, but the import operation is not interrupted.

–

For example, a failing container type import does not prevent other container types from being processed.

–

Similarly, if a process type fails import because it already exists, any UDFs and process templates for that process type are still processed.

•

There is no need to enter an option for this mode.

Strict Mode

•

If this mode is used, the import operation is aborted if it encounters a failure.

–

For example, if there is an API version mismatch, the operation aborts and no further imports are executed. Any changes already performed are not reverted.

•

Use the -Strict option to enable this import mode.

Validate Mode

Use the validate operation (instead of import) to enable this mode.

This mode produces a report listing showing the following items:

•

Entities that would be successfully imported because they do not exist on the target server.

•

Entities that already exist on the target server and are identical.

•

Entities that already exist on the target server but are different.

Validate mode can only detect a limited set of errors. For example, it can check if a particular piece of configuration already exists. If so, it checks if it is identical to the one included in the configuration package.

This information can help determine if the importAndOverwrite option is needed instead. For details, see Command-Line Options and Usage.

Example of console output:

2015-01-16 15:42:31,174 INFO - Found 4 Container Types in configuration package, validating now.

2015-01-16 15:42:31,706 INFO - Summary of Container Types:

Do not exist: None

Exist and are identical:

Container Type 'BioAnalyzer DNA 1000 Chip'

Container Type 'BioAnalyzer DNA High Sensitivity Chip'

Container Type 'BioAnalyzer RNA Nano Chip'

Container Type 'BioAnalyzer RNA Pico Chip'

Exist and are different (diffs in log): None

Example Mode

•

Use this mode to generate a manifest file if the configuration to export is not tied to a specific set of workflows, protocols, or process types.

•

Use the example operation (instead of import) to enable this mode.