CollectHiSeqXPfFailMetrics (Picard)

Classify PF-Failing reads in a HiSeqX Illumina Basecalling directory into various categories.

This tool categorizes the reads that did not pass filter (PF-Failing) into four groups. These groups are based on a heuristic that was derived by looking at a few titration experiments.

After examining the called bases from the first 24 cycles of each read, the PF-Failed reads are grouped into the following four categories:

MISALIGNED - The first 24 basecalls of a read are uncalled (numNs~24). These types of reads appear to be flow cell artifacts because reads were only found near tile boundaries and were concentration (library) independent
EMPTY - All 24 bases are called (numNs~0) but the number of bases with quality scores greater than two is less than or equal to eight (numQGtTwo<=8). These reads were location independent within the tiles and were inversely proportional to the library concentration
POLYCLONAL - All 24 bases were called and numQGtTwo>=12, were independent of their location with the tiles, and were directly proportional to the library concentration. These reads are likely the result of PCR artifacts
UNKNOWN - The remaining reads that are PF-Failing but did not fit into any of the groups listed above

The tool defaults to the SUMMARY output which indicates the number of PF-Failed reads per tile and groups them into the categories described above accordingly.

A DETAILED metrics option is also available that subdivides the SUMMARY outputs by the x- y- position of these reads within each tile. To obtain the DETAILED metric table, you must add the PROB_EXPLICIT_READS option to your command line and set the value between 0 and 1. This value represents the fractional probability of PF-Failed reads to send to output. For example, if PROB_EXPLICIT_READS=0, then no metrics will be output. If PROB_EXPLICIT_READS=1, then it will provide detailed metrics for all (100%) of the reads. It follows that setting the PROB_EXPLICIT_READS=0.5, will provide detailed metrics for half of the PF-Failed reads.

Note: Metrics labeled as percentages are actually expressed as fractions!

Usage example: (SUMMARY Metrics)

java -jar picard.jar CollectHiSeqXPfFailMetrics \
      BASECALLS_DIR=/BaseCalls/ \
      OUTPUT=/metrics/ \
      LANE=001

Usage example: (DETAILED Metrics)

java -jar picard.jar CollectHiSeqXPfFailMetrics \
      BASECALLS_DIR=/BaseCalls/ \
      OUTPUT=/Detail_metrics/ \
      LANE=001 \
      PROB_EXPLICIT_READS=1

Please see our documentation on the SUMMARY and DETAILED metrics for comprehensive explanations of the outputs produced by this tool.

Category Diagnostics and Quality Control

Overview

Collect metrics regarding the reason for reads (sequenced by HiSeqX) not passing the Illumina PF Filter. (BETA)

CollectHiSeqXPfFailMetrics (Picard) specific arguments

This table summarizes the command-line arguments that are specific to this tool. For more details on each argument, see the list further down below the table or click on an argument name to jump directly to that entry in the list.

Argument name(s)	Default value	Summary
Required Arguments
--BASECALLS_DIR -B	null	The Illumina basecalls directory.
--LANE -L	null	Lane number.
--OUTPUT -O	null	Basename for metrics file. Resulting file will be .pffail_summary_metrics
Optional Tool Arguments
--arguments_file	[]	read one or more arguments files and add them to the command line
--help -h	false	display the help message
--N_CYCLES	24	Number of cycles to look at. At time of writing PF status gets determined at cycle 24 so numbers greater than this will yield strange results. In addition, PF status is currently determined at cycle 24, so running this with any other value is neither tested nor recommended.
--NUM_PROCESSORS -NP	1	Run this many PerTileBarcodeExtractors in parallel. If NUM_PROCESSORS = 0, number of cores is automatically set to the number of cores available on the machine. If NUM_PROCESSORS < 0 then the number of cores used will be the number available on the machine less NUM_PROCESSORS.
--PROB_EXPLICIT_READS -P	0.0	The fraction of (non-PF) reads for which to output explicit classification. Output file will be .pffail_detailed_metrics (if PROB_EXPLICIT_READS != 0)
--version	false	display the version number for this tool
Optional Common Arguments
--COMPRESSION_LEVEL	5	Compression level for all compressed files created (e.g. BAM and VCF).
--CREATE_INDEX	false	Whether to create a BAM index when writing a coordinate-sorted BAM file.
--CREATE_MD5_FILE	false	Whether to create an MD5 digest for any BAM or FASTQ files created.
--GA4GH_CLIENT_SECRETS	client_secrets.json	Google Genomics API client_secrets.json file path.
--MAX_RECORDS_IN_RAM	500000	When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed.
--QUIET	false	Whether to suppress job-summary info on System.err.
--REFERENCE_SEQUENCE -R	null	Reference sequence file.
--TMP_DIR	[]	One or more directories with space available to be used by this program for temporary storage of working files
--USE_JDK_DEFLATER -use_jdk_deflater	false	Use the JDK Deflater instead of the Intel Deflater for writing compressed output
--USE_JDK_INFLATER -use_jdk_inflater	false	Use the JDK Inflater instead of the Intel Inflater for reading compressed input
--VALIDATION_STRINGENCY	STRICT	Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded.
--VERBOSITY	INFO	Control verbosity of logging.
Advanced Arguments
--showHidden	false	display hidden arguments

Argument details

Arguments in this list are specific to this tool. Keep in mind that other arguments are available that are shared with other tools (e.g. command-line GATK arguments); see Inherited arguments above.

--arguments_file / NA

read one or more arguments files and add them to the command line

List[File] []

--BASECALLS_DIR / -B

The Illumina basecalls directory.

R File null

--COMPRESSION_LEVEL / NA

Compression level for all compressed files created (e.g. BAM and VCF).

int 5 [ [ -∞ ∞ ] ]

--CREATE_INDEX / NA

Whether to create a BAM index when writing a coordinate-sorted BAM file.

Boolean false

--CREATE_MD5_FILE / NA

Whether to create an MD5 digest for any BAM or FASTQ files created.

boolean false

--GA4GH_CLIENT_SECRETS / NA

Google Genomics API client_secrets.json file path.

String client_secrets.json

--help / -h

display the help message

boolean false

--LANE / -L

Lane number.

R Integer null

--MAX_RECORDS_IN_RAM / NA

When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed.

Integer 500000 [ [ -∞ ∞ ] ]

--N_CYCLES / NA

Number of cycles to look at. At time of writing PF status gets determined at cycle 24 so numbers greater than this will yield strange results. In addition, PF status is currently determined at cycle 24, so running this with any other value is neither tested nor recommended.

int 24 [ [ -∞ ∞ ] ]

--NUM_PROCESSORS / -NP

Run this many PerTileBarcodeExtractors in parallel. If NUM_PROCESSORS = 0, number of cores is automatically set to the number of cores available on the machine. If NUM_PROCESSORS < 0 then the number of cores used will be the number available on the machine less NUM_PROCESSORS.

int 1 [ [ -∞ ∞ ] ]

--OUTPUT / -O

Basename for metrics file. Resulting file will be .pffail_summary_metrics

R File null

--PROB_EXPLICIT_READS / -P

The fraction of (non-PF) reads for which to output explicit classification. Output file will be .pffail_detailed_metrics (if PROB_EXPLICIT_READS != 0)

double 0.0 [ [ -∞ ∞ ] ]

--QUIET / NA

Whether to suppress job-summary info on System.err.

Boolean false

--REFERENCE_SEQUENCE / -R

Reference sequence file.

File null

--showHidden / -showHidden

display hidden arguments

boolean false

--TMP_DIR / NA

One or more directories with space available to be used by this program for temporary storage of working files

List[File] []

--USE_JDK_DEFLATER / -use_jdk_deflater

Use the JDK Deflater instead of the Intel Deflater for writing compressed output

Boolean false

--USE_JDK_INFLATER / -use_jdk_inflater

Use the JDK Inflater instead of the Intel Inflater for reading compressed input

Boolean false

--VALIDATION_STRINGENCY / NA

Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded.

The --VALIDATION_STRINGENCY argument is an enumerated type (ValidationStringency), which can have one of the following values:

STRICT
LENIENT
SILENT

ValidationStringency STRICT

--VERBOSITY / NA

Control verbosity of logging.

The --VERBOSITY argument is an enumerated type (LogLevel), which can have one of the following values:

ERROR
WARNING
INFO
DEBUG

LogLevel INFO

--version / NA

display the version number for this tool

boolean false

Return to top

GATK version 4.0.11.0 built at 23-11-2018 02:11:49.