pastForward

Config Designer · aDNA Pipeline · Snakemake

ℹ️
Default behaviour: any stage or step not present in the config defaults to execute: true. If an enabled step requires data from a disabled upstream step, the disabled step is executed automatically to satisfy the dependency. The species folder name must match the key used here under Species.
🧬

Project Settings

Top-level project identifier written into the config

▶
🦠

Species

Each key must match a species folder in the pipeline root directory

▶
Key Display Name

Key = folder name and YAML map key (alphanumeric / underscore, no spaces). Must match the directory under the pipeline root.
Display Name = human-readable species name used in reports.
Click ▶ on a row to configure BUSCO lineage, reference path, individual/reference/feature-library filters, and optional data locations outside the project folder for that species.

⚙️

Global Pipeline Settings

Applies to every stage and step

▶

Default: OFF Before each run the pipeline checks which output files already exist and excludes them from the target list. Warning: because those files are never requested, changed input files do not propagate to existing downstream outputs, which can leave results inconsistent. Keep off so Snakemake decides itself what needs re-running. Turn on only to resume an interrupted run without reprocessing completed steps, and only when you know the inputs have not changed.

📚

Raw Reads Processing

Runs per sample — produces clean, merged reads ready for mapping

▶

Default: ONTurn off only if you already have clean, merged per-individual reads from a previous run and want to skip directly to reference mapping. Leave on for all new datasets.

Quality Analysis ▶
FastQCMultiQC Runs FastQC at each processing stage and aggregates results into a per-stage MultiQC HTML report. Individual stages can be toggled below.
Default: ONTurn off individual stages to save time when reprocessing data or when a specific stage has already been assessed.
Adapter Removal ▶
fastp Removes adapter sequences and enforces per-base quality and length thresholds. Single-end vs. paired-end is detected automatically from the presence of an R2 file. For paired-end data, overlapping read pairs are merged into single reads — critical for ancient and historical DNA where fragment lengths are often shorter than the combined read length. Unmerged trimmed R1/R2 and any unpaired reads are concatenated into one FASTQ so no data is lost. Poly-X tail trimming is always active. If no adapter sequences are provided, fastp performs automatic detection.
Default: ONShould almost always remain on. Turn off only if reads have already been adapter-trimmed upstream. Running mapping on untrimmed reads produces artificially low alignment rates and corrupted read merging.

Default: 0 (off). Increase to trim low-quality bases from read ends during adapter removal. Keep at 0 and use the Quality Filtering step below as the quality gate instead — two separate thresholds give finer control.

Default: 0 (off). Keep at 0 to retain all fragments here; use the Quality Filtering min length (default 30 bp) as the single length gate to avoid discarding reads twice.

optional

Default: 5. Minimum length to trigger poly-X tail trimming.

Default: 40. Max % of unqualified bases allowed in a read.

Default: 5. Max number of N bases allowed in a read.

optional

If left empty, fastp auto-detects adapters. For single-end data only R1 is used.

optional
Quality Filtering ▶
fastp A dedicated second fastp pass acting purely as a quality gate — adapter trimming is explicitly disabled here to avoid double-processing. Thresholds are configured independently from the adapter removal step, allowing fine-grained control of each stage separately.
Default: ONTurn off only if you have very few reads and cannot afford further losses, or to benchmark unfiltered mapping. Recommended for all standard aDNA workflows — low-quality bases inflate error rates and reduce mapping sensitivity.

Default: 15. Q15 is a reasonable baseline for aDNA. Increase to Q20 for stricter filtering on high-coverage data; decrease only if read depth is critically low and you need to retain more reads.

Default: 30 bp — the standard minimum for reliable mapping. Increase for longer-fragment aDNA; decrease only if fragments are genuinely shorter (e.g. highly degraded specimens).

optional

Default: 40. Max % of unqualified bases allowed in a read.

Default: 5. Max number of N bases allowed in a read.

Taxonomic Screening ▶
ECMSDCentrifuge Both tools operate on quality-filtered reads. Each tool can be toggled independently. Results from multiple samples belonging to the same individual are merged into a combined summary.
Default: ONRecommended for all aDNA work — the share of reads assigned to non-target taxa is a key quality metric for any ancient specimen. Turn off only if taxonomic screening has already been done externally, or if you need mapping results quickly and plan to run this separately.

Maps reads against a curated mitochondrial reference database and reports the proportional contribution of different taxa. Configurable coverage threshold, top-N plot count, mapping quality cutoff, and taxonomic level (default: species). Per-sample results are merged per individual.

Default: ONTurn off if ECMSD is not installed or configured on your system. Well-suited for vertebrate aDNA where mitochondrial contamination is the primary concern.

optional

Default: conda. Only takes effect when the ecmsd conda environment is (re)created — see FAQ.md.

optional

Path to the ECMSD database folder. If not set, the pipeline auto-creates a database at resources/ecmsd_database via ECMSD --create-db.

optional

Default: 25. Minimum % of reference covered by reads to retain it.

Default: 25. Number of top references to generate alignment plots for.

Default: 20. Minimum mapping quality score to include a read.

Default: species. Taxonomic level at which to aggregate and report results.

k-mer-based taxonomic classification against a user-provided database. Reports top 10 taxa ranked by both total and unique read assignments. An index path is required (see optional settings below).

Default: ONTurn off if no Centrifuge index is available — the index path must be set for this tool to run. Provides broad taxonomic screening across all domains of life, complementing the mitochondria-focused ECMSD.

optional
optional

Full path to the Centrifuge database index prefix.

🗺️

Reference Processing

Runs per individual per reference — mapping, deduplication, damage analysis, coverage

▶

Default: ONTurn off if you only need raw-reads QC and taxonomic screening without genome mapping. Must be explicitly set to true in the config to run.

Mapping ▶
BWAsamtools Merged per-individual reads are mapped to the reference genome. Output is immediately coordinate-sorted and indexed. The unsorted BAM is discarded to save disk space. The pre-dedup sorted BAM is used for Preseq library complexity estimation — running Preseq after deduplication would remove the duplication information it requires, producing invalid estimates.
Default: ONMapping always runs when Reference Processing is enabled — it is the core step that all downstream analyses depend on. Choose your mapper based on your data: bwa-aln for typical short aDNA reads (<70 bp), bwa-mem2 for longer reads or speed, minimap2 for highly divergent references.
Classic seed-and-extend, well-suited for short aDNA reads.
optional

For bwa-aln, defaults to -n 0.01 -k 2 -l 1024 -o 2 (Oliva et al. 2021).

Deduplication ▶
DeDup Removes PCR and sequencing duplicates using DeDup, which is specifically designed for ancient and historical DNA and correctly handles merged single-stranded reads. If disabled, the sorted BAM from mapping is passed directly to subsequent steps.

To manage memory on references with many contigs, contigs are grouped into clusters and each cluster is deduplicated independently, then merged. The cluster size is configurable below. Each deduplication run produces a histogram and JSON statistics file that feed into MultiQC.
Default: ONLeave on for most aDNA analyses — PCR duplicates inflate apparent read depth and bias variant calling. Turn off only if library complexity is very low (duplicate removal would leave too few reads) or if you specifically need the pre-dedup BAM for a downstream tool.

Default: 10. Minimum number of contigs grouped into a cluster. Small contigs below this count are merged together before deduplication.

Default: 500. Lower values use less memory per cluster but increase runtime due to more merge operations. Reduce (e.g. to 100) only if deduplication is failing on a large, fragmented reference genome.

Default: 20000. DeDup's JVM heap (-Xms/-Xmx), also requested from the cluster scheduler as resources.mem_mb. Increase for large reference genomes/BAM files; decrease for small ones to free up cluster resources.

Damage Rescaling ▶
mapDamage2 Profiles cytosine deamination — the hallmark of ancient and historical DNA — appearing as C→T substitutions at the 5′ end and G→A substitutions at the 3′ end of reads. mapDamage2 estimates these damage patterns and rescales base quality scores accordingly, reducing the influence of damage-induced errors in downstream variant calling.

Input BAM is selected dynamically: deduplicated BAM if deduplication is enabled, otherwise the sorted BAM. The rescaled BAM is sorted and indexed; the unsorted rescaled BAM is discarded.
Default: ONRecommended for any analysis involving variant calling or genotyping — rescaling reduces the influence of damage-induced errors. Turn off if you want to inspect raw damage patterns before deciding, or if your downstream tools model damage explicitly and do not benefit from rescaled base qualities.
Filter Unmapped Reads ▶
Optionally dispose of reads that did not map to the reference. Choose between writing a mapped-reads-only BAM, or extracting the unmapped reads to FASTQ.gz or FASTA.gz for downstream analysis (e.g. further taxonomic screening).
Default: OFFTurn on to process unmapped reads. The default action is keep (no-op) — you must explicitly change it to remove (mapped-only BAM), extract_fastq, or extract_fasta. Not needed for standard aDNA workflows.
BAM Analysis ▶
samtoolsQualimapPreseq samtools depth reports per-position depth across the entire reference including zero-coverage sites. A custom script computes coverage breadth at multiple depth thresholds and mean depth per individual, combined into species-level summary tables formatted for MultiQC.

Additionally: Qualimap bamqc produces an HTML QC report (mapping rate, coverage uniformity, GC content, insert size). Preseq estimates library complexity and extrapolates sequencing yield.
Default: ONKey QC metrics for aDNA — coverage breadth and depth determine whether a sample has enough data for downstream genotyping or population analysis. Turn off only if you need mapping results quickly and will assess coverage separately.

Default: 4096. Memory requested from the cluster scheduler for Qualimap. Increase for large reference genomes/BAM files; decrease for small ones to free up cluster resources.

📊

REVEAL

TE & genomic feature abundance — maps to combined SCG + feature library for depth-normalised comparisons

▶

Default: ONTurn off if you are not performing TE or genomic feature abundance analysis, or if the feature library and SCG library are not yet prepared. Requires both libraries placed in {species}/input/reveal_module/.

optional

Default: conda. REVEAL is not yet on bioconda and is always side-loaded, so conda currently installs the newest tagged release; this only takes effect when the reveal conda environment is (re)created — see FAQ.md.

SCG Selector ▶
BUSCO Automatically identifies single-copy genes (SCGs) from the reference genome via BUSCO when no user-provided SCG FASTA is placed in {species}/input/reveal_module/scg/. SCGs act as coverage normalisers for the REVEAL pipeline. Skipped automatically when a user-provided file is found or when no BUSCO lineage is configured for the species (set per species in the Species section above).
Default: ONCan also be used standalone (without any feature library) to produce an SCG ranking table as the sole output. Set the BUSCO Lineage per species in the Species section; if exactly one reference exists in {species}/input/reference_module/ it is auto-detected, otherwise specify it explicitly.
Uses the same mapper as the REVEAL feature-library mapping by default.
optional

SCG selection parameters

Number of highest-ranked SCGs retained as coverage normalisers (Default: 20; overridable per species)
Minimum sequence length for a BUSCO SCG to be included (Default: 4000; overridable per species)
Maximum sequence length for a BUSCO SCG to be included (Default: 8000; overridable per species)

BAM filtering

Reads with MAPQ below this value are removed from the SCG BAM. Also applied when computing coverage stats. Set to 0 (default) to disable.
Mapping ▶
BWA-MEM2 Merged per-individual reads are mapped to a combined reference consisting of TE/feature sequences (suffixed _fle) and single-copy gene sequences (suffixed _scg). Unmapped reads are discarded immediately after conversion from SAM to BAM to reduce file sizes. Multiple feature libraries per species are supported — each produces an independent set of results.
Default: ONMapping always runs when REVEAL is enabled. Use bwa-aln for typical short aDNA fragments; switch to bwa-mem2 for longer reads or higher throughput. The combined SCG + TE library is indexed automatically.
Classic seed-and-extend, well-suited for short aDNA reads.
optional
Reads mapping to SCG (_scg) references with MAPQ below this value are removed after mapping. Feature library reads are not affected. (Default: 0)
Reads mapping to feature library (_fle) references with MAPQ below this value are removed after mapping. SCG reads are not affected. Set to 0 (default) to disable.
REVEAL Visualization ▶
REVEAL Generates sequence overview (SO) profiles — per-position coverage, SNP, and indel information for each reference sequence. Profiles are normalised and converted into a REVEAL-compatible directory structure, then used to produce per-individual TE occupancy plots and a faceted species-level comparison plot showing TE dynamics across all individuals simultaneously.
Default: ONTurn off if you only need normalised abundance counts and do not require per-position TE occupancy profiles. REVEAL is more computationally intensive than PF Normalization alone.

Output selection

Plot settings

Y-axis value above which individual plots switch to log scale (Default: 25)
Y-axis value above which the species comparison plot switches to log scale (Default: 25)
Bin size for per-position coverage plotables (Default: "target:5000"). Fixed integer (e.g. "100"), "target:N" to auto-compute bin_size = max(1, seq_len // N) per sequence, or length-threshold rules (e.g. "10000:1,100000:10,default:500")

Coverage analysis (bam2so)

Reads below this mapping quality are treated as ambiguous and excluded (Default: 5)
Minimum number of supporting reads to call a SNP (Default: 5)
Minimum allele frequency (0–1) for a SNP call (Default: 0.1)
Minimum number of supporting reads to report an indel (Default: 3)
Minimum allele frequency (0–1) for an indel call (Default: 0.01)

Normalisation (normalize-so)

Positions from each sequence end excluded when computing the normalisation factor (Default: 100)
Percentile used to exclude extreme coverage values from normalisation (Default: 25)
📋

Summary Processing

Consolidates all QC outputs into MultiQC reports per individual and per species

▶

Default: ONGenerates the final MultiQC HTML reports. There is rarely a reason to turn this off — it is the primary deliverable of the pipeline. Only disable if you are running a partial test and do not need consolidated reports yet.

Produces two levels of MultiQC HTML reports:

  • Per-individual — all QC and analytics for one individual across all references
  • Per-species — all individuals aggregated for cross-sample comparison
YAML Preview
config.yaml