Add Blank Lines

Compatibility: BaseSpace Clarity LIMS v3.0.0 or later; NGS Extensions Package v5.2.0 or later

Some instruments, such as Bioanalyzer and the ABI 7900HT, require input files that include a line for every well in a container.

Currently, files created with the file generator can only contain data lines for each well that is in use and cannot add lines for empty wells. This extension script allows you to process a file in a certain format to add customizable lines for the empty container wells.

The addBlankLines script allows for the creation of files that require a line entry for every well in a container, including those wells that are empty.

To accomplish this, the script takes in an existing file, created with file generator, and processes it to add the new lines.

1. The script reads the file and separates it into header, data, and footer sections.
2. Using the API,  the script obtains the container size, its unavailable wells, and the placement of each sample.
3. Using this information, the script runs through each possible placement and builds a full set of data. This includes a line for each empty well, in addition to all original lines of data. If desired, a line for each unavailable well may also be included using script parameters described.
4. The file is then rewritten with the header, full data, and footer.

The addBlankLines script overwrites the input file provided, replacing it with the new file that includes the additional container line entries.

Placement information is written using a numeric index that is converted from the LIMS well placement. The line information that is inserted for an empty well is provided to the script. This is controlled by the blankLineString parameter, described.

The logic that determines whether the script uses input samples and containers or output samples and containers is as follows:

• By default the script will use outputs.
• If there are no outputs or if the optional parameter -forceInputs true is configured, the script will use inputs.

The following table defines the parameters used by the addBlankLines script.

-i {URI}

LIMS step URI (Required)

-u {username}

The LIMS username (Required)

-p {password}

The LIMS password (Required)

-f {inputFileName}

The input file name (Required)

-l {logFileName}

The log file name (Required)

-h {headerRows}

The number of lines in the header (Required)

-sep {separator}

The separator to use for the file - COMMA and TAB supported (Required)

-b {blankLineString}

The string to be used for blank lines (Required)

-c {columnType}

The data type specified in the first column of the data - LIMSID and PLACEMENT supported (Required)

-pre {indexPrefix}

The prefix to be included before the index of each well in the file output (Optional)

-forceInputs {forceInputs}

Provide as "true" to force the script to use the input samples and container when processing, even if output samples and containers exist. (Optional)

-countUnavailable

Provide as "true" to include unavailable wells in the placement index count (Optional)

-addUnavailable

Provide as "true" to include unavailable wells as entries in the updated file (Optional)

The script uses this parameter to match each row to a sample in the LIMS. The parameter tells the script what data type to expect in the first column of any data row. Supported data types are PLACEMENT and LIMSID. For example:

-c LIMSID

• LIMSID - The script reads the LIMS ID in the first column of data, matches it directly to the sample, and uses this to get the placement of that sample through the API. It does not replace this entry in the data, however.
• PLACEMENT - The script reads the sample placement in the first column of data, then finds the sample in the container at that location. The script then replaces this placement entry with an entry of the form "{index-prefix}{index}". Supported placement formats are alphanumeric (eg A1), orcolon-separated (eg 1:1). Placements that are numeric only (eg 11) are not supported.

This parameter lets you force the script to use input samples and input container, even if output samples and output container are found for the step. If this parameter is not provided, the script uses output samples and output container if available; input samples and container if they are not.

This parameter provides a way to place a prefix directly before the index on blank lines. If the column type is set to PLACEMENT, the indexPrefix will also be printed before the index when replacing the well placement in the data.

This parameter defines what is placed after the index in a blank line for an empty well. A separator character is automatically placed between the index and the blankLineString.

Any ‘ \t ’ found in the blankLineString is replaced by a tab space in the output. Otherwise, the blankLineString is left unchanged.

The script supports tab and comma separators in the output file. These can be provided with this parameter in the form “TAB” and “COMMA”. The same separator is used to interpret the original file and produce the updated file.

If provided, a container is unavailable wells will be included when determining the placement index of a well.

For example, imagine a 2x2 container with wells A:1, A:2, B:1, and B:2, where wells B:1 and B:2 are marked as unavailable. By default, the script will calculate well A:2 as well index 2, excluding the unavailable wells from the index. When the countUnavailable parameter is provided, A:2 is instead calculated as well index 3.

If provided, blank lines will be added to the file for unavailable wells in addition to blank lines for unoccupied wells.

The index is the well number when the wells are counted from left to right, from top to bottom. By default, unavailable wells are not counted in this index. However, if the script is configured with the countUnavailable parameter, they will be included in the index count.

The format of any blank line printed for an empty well is as follows:

{indexPrefix}{index}{separator}{blankLineString}

The script operates with the following constraints:

• The input file and log file to update must exist locally.
• Only supports TAB and COMMA as separators in the file.
• Only supports processing one container at a time.
• Can only use the first column of the data and requires either well placement or LIMS ID as values in this column.
• Indexing always is done from left to right, top to bottom.
• The well placement format in the original file must be alphanumeric (eg A1), or separated by a colon (eg 1:1).
• As designed, the script supports processing one container only. As such, any step on which it is run must use either a single input container or a single output container, depending on which type of container you expect the script to use (see forceInputs).
• The script should be configured to run after the creation of a file occurs - either within the same automation call or as a separate automation triggered after the call that generated the file.
• It can also be configured standalone for files that are locally accessible.

This example uses the addBlankLines script within the same automation call as the file generation. It also includes the optional parameter '- pre ', adding the prefix 'Sample' before the index of each well in the file output.

• The first half of the call (the section preceding the 'and and' characters) uses the file generator to create the file.
• The second half of the call  (the section following the 'and and' characters) takes the generated file and processes it to add blank lines.

LIMS v3.5 or earlier:

bash -c "/usr/bin/java -cp /opt/gls/clarity/extensions/ngs-common/v5/EPP/DriverFileGenerator.jar driver_file_generator \

-i {processURI:v2:http} \

-u {username} \

-p {password} \

-t /opt/gls/clarity/extensions/ngs-common/v5/EPP/conf/readonly/bioA_driver_file_template.csv \

-o {compoundOutputFileLuid0}.csv \

-l {compoundOutputFile1} \

&& /usr/bin/java -cp /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar addBlankLines \

-i {stepURI:v2:http} \

-u {username} \

-p {password} \

-f {compoundOutputFileLuid0}.csv \

-l {compoundOutputFileLuid1} \

-sep COMMA \

-b ', False,' \

-h 1 \

-c LIMSID \

-pre 'Sample '"

LIMS v4.0 or later:

bash -c "/opt/gls/clarity/bin/java -cp /opt/gls/clarity/extensions/ngs-common/v5/EPP/DriverFileGenerator.jar driver_file_generator \

-i {processURI:v2:http} \

-u {username} \

-p {password} \ -t /opt/gls/clarity/extensions/ngs-common/v5/EPP/conf/readonly/bioA_driver_file_template.csv \

-o {compoundOutputFileLuid0}.csv \

-l {compoundOutputFile1} \

&& /opt/gls/clarity/bin/java

-cp /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar addBlankLines \ -i {stepURI:v2:http} \

-u {username} \

-p {password} \

-f {compoundOutputFileLuid0}.csv \

-l {compoundOutputFileLuid1} \

-sep COMMA \

-b ', False,' \

-h 1 \

-c LIMSID \

-pre 'Sample '"

These examples show the configuration of an automation trigger that runs the addBlankLines script by itself, on a local file:

LIMS v3.5 or earlier:

bash -c "/usr/bin/java -cp /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar addBlankLines \

-i {stepURI:v2:http} \

-u {username} \

-p {password} \

-f /opt/gls/clarity/customextensions/roboticsfiles/example.csv \

-l /opt/gls/clarity/customextensions/roboticsfiles/log.html \

-sep COMMA \

-b ', False,' \

-h 1 \

-c LIMSID"

LIMS v4.0 or later:

bash -c "/opt/gls/clarity/bin/java -cp /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar addBlankLines \

-i {stepURI:v2:http} \

-u {username} \

-p {password} \

-f /opt/gls/clarity/customextensions/roboticsfiles/example.csv \

-l /opt/gls/clarity/customextensions/roboticsfiles/log.html \

-sep COMMA \

-b ', False,' \

-h 1 \ -c LIMSID"