DownsampleSam (Picard)

Downsample a SAM or BAM file.This tool applies a downsampling algorithm to a SAM or BAM file to retain only a (deterministically random) subset of the reads. Reads from the same template (e.g. read-pairs, secondary and supplementary reads) are all either kept or discarded as a unit, with the goal of retaining readsfrom PROBABILITY * input templates. The results will contain approximately PROBABILITY * input reads, however for very small PROBABILITIES this may not be the case. A number of different downsampling strategies are supported using the STRATEGY option: ConstantMemory: Downsamples a stream or file of SAMRecords using a hash-projection strategy such that it can run in constant memory. The downsampling is stochastic, and therefore the actual retained proportion will vary around the requested proportion. Due to working in fixed memory this strategy is good for large inputs, and due to the stochastic nature the accuracy of this strategy is highest with a high number of output records, and diminishes at low output volumes. HighAccuracy: Attempts (but does not guarantee) to provide accuracy up to a specified limit. Accuracy is defined as emitting a proportion of reads as close to the requested proportion as possible. In order to do so this strategy requires memory that is proportional to the number of template names in the incoming stream of reads, and will thus require large amounts of memory when running on large input files. Chained: Attempts to provide a compromise strategy that offers some of the advantages of both the ConstantMemory and HighAccuracy strategies. Uses a ConstantMemory strategy to downsample the incoming stream to approximately the desired proportion, and then a HighAccuracy strategy to finish. Works in a single pass, and will provide accuracy close to (but often not as good as) HighAccuracy while requiring memory proportional to the set of reads emitted from the ConstantMemory strategy to the HighAccuracy strategy. Works well when downsampling large inputs to small proportions (e.g. downsampling hundreds of millions of reads and retaining only 2%. Should be accurate 99.9% of the time when the input contains more than 50,000 templates (read names). For smaller inputs, HighAccuracy is recommended instead.

Usage examples:

Downsample file, keeping about 10% of the reads

java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ P=0.2

Downsample file, keeping about 2% of the reads

java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ STRATEGY=Chained \ P=0.02 \ ACCURACY=0.0001

Downsample file, keeping about 0.001% of the reads (may require more memory)

java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ STRATEGY=HighAccuracy \ P=0.00001 \ ACCURACY=0.0000001

Category Read Data Manipulation

---

Overview

Summary

This tool applies a downsampling algorithm to a SAM or BAM file to retain only a (deterministically random) subset of the reads. Reads from the same template (e.g. read-pairs, secondary and supplementary reads) are all either kept or discarded as a unit, with the goal of retaining reads from PROBABILITY * (input templates). The results will contain approximately PROBABILITY * (input reads), however for very small probabilities this may not be the case. A number of different downsampling strategies are supported using the #STRATEGY option:

ConstantMemory

Downsamples a stream or file of SAMRecords using a hash-projection strategy such that it can run in constant memory. The downsampling is stochastic, and therefore the actual retained proportion will vary around the requested proportion. Due to working in fixed memory this strategy is good for large inputs, and due to the stochastic nature the accuracy of this strategy is highest with a high number of output records, and diminishes at low output volumes.

HighAccuracy

Attempts (but does not guarantee) to provide accuracy up to a specified limit. Accuracy is defined as emitting a proportion of reads as close to the requested proportion as possible. In order to do so this strategy requires memory that is proportional to the number of template names in the incoming stream of reads, and will thus require large amounts of memory when running on large input files.

Chained

Attempts to provide a compromise strategy that offers some of the advantages of both the ConstantMemory and HighAccuracy strategies. Uses a ConstantMemory strategy to downsample the incoming stream to approximately the desired proportion, and then a HighAccuracy strategy to finish. Works in a single pass, and will provide accuracy close to (but often not as good as) HighAccuracy while requiring memory proportional to the set of reads emitted from the ConstantMemory strategy to the HighAccuracy strategy. Works well when downsampling large inputs to small proportions (e.g. downsampling hundreds of millions of reads and retaining only 2%. Should be accurate 99.9% of the time when the input contains more than 50,000 templates (read names). For smaller inputs, HighAccuracy is recommended instead.

The number of records written can be output to a QualityYieldMetrics metrics file via the #METRICS_FILE.

Usage examples:

Downsample file, keeping about 10% of the reads

 java -jar picard.jar DownsampleSam \
       I=input.bam \
       O=downsampled.bam \
       P=0.1
 

Downsample file, keeping 2% of the reads

 java -jar picard.jar DownsampleSam \
       I=input.bam \
       O=downsampled.bam \
       STRATEGY=Chained \
       P=0.02 \
       ACCURACY=0.0001
 

Downsample file, keeping 0.001% of the reads (may require more memory)

 java -jar picard.jar DownsampleSam \
       I=input.bam \
       O=downsampled.bam \
       STRATEGY=HighAccuracy \
       P=0.00001 \
       ACCURACY=0.0000001
 

DownsampleSam (Picard) specific arguments

This table summarizes the command-line arguments that are specific to this tool. For more details on each argument, see the list further down below the table or click on an argument name to jump directly to that entry in the list.

Argument name(s)	Default value	Summary
Required Arguments
--INPUT  -I	null	The input SAM or BAM file to downsample.
--OUTPUT  -O	null	The output, downsampled, SAM or BAM file to write.
Optional Tool Arguments
--ACCURACY  -A	1.0E-4	The accuracy that the downsampler should try to achieve if the selected strategy supports it. Note that accuracy is never guaranteed, but some strategies will attempt to provide accuracy within the requested bounds.Higher accuracy will generally require more memory.
--arguments_file	[]	read one or more arguments files and add them to the command line
--help  -h	false	display the help message
--METRICS_FILE  -M	null	The metrics file (of type QualityYieldMetrics) which will contain information about the downsampled file.
--PROBABILITY  -P	1.0	The probability of keeping any individual read, between 0 and 1.
--RANDOM_SEED  -R	1	Random seed used for deterministic results. Setting to null will cause multiple invocations to produce different results.
--REFERENCE_SEQUENCE	null	The reference sequence file.
--STRATEGY  -S	ConstantMemory	The downsampling strategy to use. See usage for discussion.
--version	false	display the version number for this tool
Optional Common Arguments
--COMPRESSION_LEVEL	5	Compression level for all compressed files created (e.g. BAM and VCF).
--CREATE_INDEX	false	Whether to create a BAM index when writing a coordinate-sorted BAM file.
--CREATE_MD5_FILE	false	Whether to create an MD5 digest for any BAM or FASTQ files created.
--GA4GH_CLIENT_SECRETS	client_secrets.json	Google Genomics API client_secrets.json file path.
--MAX_RECORDS_IN_RAM	500000	When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed.
--QUIET	false	Whether to suppress job-summary info on System.err.
--TMP_DIR	[]	One or more directories with space available to be used by this program for temporary storage of working files
--USE_JDK_DEFLATER  -use_jdk_deflater	false	Use the JDK Deflater instead of the Intel Deflater for writing compressed output
--USE_JDK_INFLATER  -use_jdk_inflater	false	Use the JDK Inflater instead of the Intel Inflater for reading compressed input
--VALIDATION_STRINGENCY	STRICT	Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded.
--VERBOSITY	INFO	Control verbosity of logging.
Advanced Arguments
--showHidden	false	display hidden arguments

Argument details

Arguments in this list are specific to this tool. Keep in mind that other arguments are available that are shared with other tools (e.g. command-line GATK arguments); see Inherited arguments above.

---

--ACCURACY / -A

The accuracy that the downsampler should try to achieve if the selected strategy supports it. Note that accuracy is never guaranteed, but some strategies will attempt to provide accuracy within the requested bounds.Higher accuracy will generally require more memory.

double  1.0E-4  [ [ -∞  ∞ ] ]

---

--arguments_file / NA

read one or more arguments files and add them to the command line

List[File]  []

---

--COMPRESSION_LEVEL / NA

Compression level for all compressed files created (e.g. BAM and VCF).

int  5  [ [ -∞  ∞ ] ]

---

--CREATE_INDEX / NA

Whether to create a BAM index when writing a coordinate-sorted BAM file.

Boolean  false

---

--CREATE_MD5_FILE / NA

Whether to create an MD5 digest for any BAM or FASTQ files created.

boolean  false

---

--GA4GH_CLIENT_SECRETS / NA

Google Genomics API client_secrets.json file path.

String  client_secrets.json

---

--help / -h

display the help message

boolean  false

---

--INPUT / -I

The input SAM or BAM file to downsample.

R File  null

---

--MAX_RECORDS_IN_RAM / NA

When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed.

Integer  500000  [ [ -∞  ∞ ] ]

---

--METRICS_FILE / -M

The metrics file (of type QualityYieldMetrics) which will contain information about the downsampled file.

File  null

---

--OUTPUT / -O

The output, downsampled, SAM or BAM file to write.

R File  null

---

--PROBABILITY / -P

The probability of keeping any individual read, between 0 and 1.

double  1.0  [ [ -∞  ∞ ] ]

---

--QUIET / NA

Whether to suppress job-summary info on System.err.

Boolean  false

---

--RANDOM_SEED / -R

Random seed used for deterministic results. Setting to null will cause multiple invocations to produce different results.

Integer  1  [ [ -∞  ∞ ] ]

---

--REFERENCE_SEQUENCE / NA

The reference sequence file.

File  null

---

--showHidden / -showHidden

display hidden arguments

boolean  false

---

--STRATEGY / -S

The downsampling strategy to use. See usage for discussion.

The --STRATEGY argument is an enumerated type (Strategy), which can have one of the following values:

HighAccuracy

ConstantMemory

Chained

Strategy  ConstantMemory

---

--TMP_DIR / NA

One or more directories with space available to be used by this program for temporary storage of working files

List[File]  []

---

--USE_JDK_DEFLATER / -use_jdk_deflater

Use the JDK Deflater instead of the Intel Deflater for writing compressed output

Boolean  false

---

--USE_JDK_INFLATER / -use_jdk_inflater

Use the JDK Inflater instead of the Intel Inflater for reading compressed input

Boolean  false

---

--VALIDATION_STRINGENCY / NA

Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded.

The --VALIDATION_STRINGENCY argument is an enumerated type (ValidationStringency), which can have one of the following values:

STRICT

LENIENT

SILENT

ValidationStringency  STRICT

---

--VERBOSITY / NA

Control verbosity of logging.

The --VERBOSITY argument is an enumerated type (LogLevel), which can have one of the following values:

ERROR

WARNING

INFO

DEBUG

LogLevel  INFO

---

--version / NA

display the version number for this tool

boolean  false

---

Return to top

---

See also General Documentation | Tool Docs Index Tool Documentation Index | Support Forum

GATK version 4.0.11.0 built at 23-11-2018 02:11:49.