Troubleshooting

Failure Type

Actions

Hash Table Build

If you encounter the following message when running the build-hashtable-DRAGEN_TSO500-2.1.1.sh script:

ERROR: The maximum number of open file descriptors (ulimit -n) is currently set to a soft limit of 1024 -- required: 65535.

perform the following.

1. Edit the file, /etc/sysctl.conf, and add the following line.
fs.file-max = 65535

This line increases the ulimit of your server.

2. Log out of the server and then log back in.
3. Re-run the build hash table script.

Software

Open the log file ./<AnalysisFolder>/Logs_Intermediates/pipeline_trace.txt. This log file displays each pipeline step run by the Nextflow workflow manager software. If a step fails, it is marked as FAILED. Each step generates log files that are stored in step-specific subfolders in the Logs_Intermediates folder. Review the log files in the relevant Logs_Intermediates folder for the step to identify potential sources of error.
Open the errors folder ./<AnalysisFolder>/errors. The workflow creates an error file, error_<NameOfFailedStep>.json, for each step that failed during analysis. For steps that fail per sample, there is a separately labeled file for each sample that failed each step error_<NameOfFailedStep>_<SampleIDIfRelevant>.json. These files contain the command and stdout and stderr from the step.

Samples

Open the combined metrics output results file ./<AnalysisFolder>/Results/<PairId>/MetricsOutput.tsv. If a sample fails an analysis step, the Pair ID that contains the sample shows the failure under FAILED_STEPS in the Analysis Status section, and COMPLETED_ALL_STEPS show as False. If available, review the individual log files for the failed steps under ./<AnalysisFolder>/Logs_Intermediates to identify potential sources of error.

Multinode Gather

If the following error appears, check if the sample or pair ID was included multiple times during separate node analysis runs, before being gathered together. If the error exists, rerun one of the analyses without the duplicate and reattempt gathering.

ERROR:Gather:Destination file ... already exists - check if the same sample ID is in multiple input folders