[Dorado] New Dorado Basecalling Workflow Terra

.dockstore.yml

@@ -281,5 +281,10 @@ workflows:
  - name: Snippy_Streamline_FASTA_PHB
    subclass: WDL
    primaryDescriptorPath: /workflows/phylogenetics/wf_snippy_streamline_fasta.wdl
+   testParameterFiles:
+    - /tests/inputs/empty.json
+ - name: Dorado_Basecalling_PHB
+   subclass: WDL
+   primaryDescriptorPath: /workflows/utilities/wf_dorado_basecalling.wdl
    testParameterFiles:
     - /tests/inputs/empty.json

docs/workflows/standalone/dorado_basecalling.md

@@ -0,0 +1,114 @@
+# Dorado Basecalling Workflow
+
+## Quick Facts
+
+| **Workflow Type** | **Applicable Kingdom** | **Last Known Changes** | **Command-line Compatibility** | **Workflow Level** |
+|---|---|---|---|---|
+| [Standalone](../../workflows_overview/workflows_type.md/#standalone) | [Any Taxa](../../workflows_overview/workflows_kingdom.md/#any-taxa) | v2.3.0 | Yes | Sample-level |
+
+## Dorado_Basecalling_PHB
+
+The Dorado Basecalling workflow is used to convert Oxford Nanopore `POD5` sequencing files into `FASTQ` format by utilizing a GPU-accelerated environment. This workflow is ideal for high-throughput applications where fast and accurate basecalling is essential. The workflow will upload fastq files to a user designated terra table for downstream analysis.
+
+### Model Type Selection
+
+Users can configure the basecalling model by setting the `dorado_model` input parameter:
+
+Default Model: "sup" (super accuracy) is used unless overridden by the user.
+Manual Model Input: Users can specify the full path or name of a specific model (e.g., [email protected]).
+Automatic Detection: When set to sup, hac, or fast, Dorado will automatically select the appropriate model version if available.
+
+- **Model Type (sup):** (Super Accuracy) The most accurate model, recommended for applications requiring the highest basecall accuracy. It is the slowest of the three model types.
+- **Model Type (hac):** (High Accuracy) A balance between speed and accuracy, recommended for most users. Provides accurate results faster than `sup` but less accurate than `sup`.
+- **Model Type (fast):** (Fast Model) The fastest model, recommended when speed is prioritized over accuracy, such as for initial analyses or non-critical applications.
+
+### Example Manual Models:
+
+- `[email protected]`
+- `[email protected]`
+- `[email protected]`
+
+??? Info "Supported Kit Names"
+
+    Ensure you use an accepted kit name in the `kit_name` parameter. The following kit names are supported in Dorado:
+
+    ```
+    EXP-NBD103, EXP-NBD104, EXP-NBD114, EXP-NBD114-24, EXP-NBD196, EXP-PBC001, EXP-PBC096, 
+    SQK-16S024, SQK-16S114-24, SQK-LWB001, SQK-MLK111-96-XL, SQK-MLK114-96-XL, SQK-NBD111-24, 
+    SQK-NBD111-96, SQK-NBD114-24, SQK-NBD114-96, SQK-PBK004, SQK-PCB109, SQK-PCB110, SQK-PCB111-24, 
+    SQK-PCB114-24, SQK-RAB201, SQK-RAB204, SQK-RBK001, SQK-RBK004, SQK-RBK110-96, SQK-RBK111-24, 
+    SQK-RBK111-96, SQK-RBK114-24, SQK-RBK114-96, SQK-RLB001, SQK-RPB004, SQK-RPB114-24, 
+    TWIST-16-UDI, TWIST-96A-UDI, VSK-PTC001, VSK-VMK001, VSK-VMK004, VSK-VPS001
+    ```
+
+    Select from these options to avoid input errors.
+
+### Inputs
+
+| **Terra Task Name** | **Variable** | **Type** | **Description** | **Default Value** | **Terra Status** |
+|---|---|---|---|---|---|
+| dorado_basecalling_workflow | **input_files** | Array[File] | Array of `POD5` files for basecalling | None | Required |
+| dorado_basecalling_workflow | **fastq_file_name** | String | Prefix for naming output FASTQ files | None | Required |
+| dorado_basecalling_workflow | **kit_name** | String | Sequencing kit name used (e.g., `SQK-RPB114-24`) | None | Required |
+| dorado_basecalling_workflow | **fastq_upload_path** | String | Terra folder path for uploading FASTQ files | None | Required |
+| dorado_basecalling_workflow | **terra_project** | String | Terra project ID for FASTQ file upload | None | Required |
+| dorado_basecalling_workflow | **terra_workspace** | String | Terra workspace for final FASTQ file upload | None | Required |
+| dorado_basecalling_workflow | **dorado_model** | String | Model speed for basecalling ('sup' for super accuracy, 'hac' for high accuracy, or 'fast' for high speed). Users may also specify a full model name. | "sup" | Optional |
+| basecall_task.basecall | **cpu** | Int | Number of CPUs allocated | 8 | Optional |
+| basecall_task.basecall | **gpuCount** | Int | Number of GPUs to use | 1 | Optional |
+| basecall_task.basecall | **gpuType** | String | Type of GPU (e.g., `nvidia-tesla-t4`) | "nvidia-tesla-t4" | Optional |
+| basecall_task.basecall | **memory** | Int | Amount of memory to allocate (GB) | 32 | Optional |
+| dorado_basecalling_workflow | **assembly_data** | Boolean | Indicates if the data is for assembly | false | Optional |
+| dorado_basecalling_workflow | **file_ending** | String? | File extension pattern for identifying files (e.g., ".fastq.gz") | None | Optional |
+| dorado_basecalling_workflow | **paired_end** | Boolean | Indicates if data is paired-end | false | Optional |
+
+!!! info "Detailed Input Information"
+    - **dorado_model**: If set to 'sup', 'hac', or 'fast', the workflow will run with automatic model selection. If a full model name is provided, Dorado will use that model directly.
+    - **fastq_file_name**: This will serve as a prefix for the output FASTQ files. For example, if you provide `project001-`, the resulting files will be named `project001-barcodeXX.fastq.gz`.
+    - **kit_name**: Ensure the correct kit name is provided, as it determines the barcoding and adapter trimming behavior.
+    - **fastq_upload_path**: This is the folder path in Terra where the final FASTQ files will be transferred for further analysis. Ensure the path matches your Terra workspace configuration.
+
+!!! tip "File Naming Guidelines"
+    - **Avoid special characters**: Do not include special characters such as underscores (`_`), periods (`.`), or any non-alphanumeric symbols in the `fastq_file_name` prefix, as these can interfere with proper file parsing and sample name recognition.
+    - **Use a clear, simple prefix**: The prefix `projectname` will automatically append identifiers like `-barcodeXX.fastq.gz` or `-unclassified.fastq.gz` to name each output file, ensuring each one is distinct.
+
+    ### Examples:
+
+    - **Accepted Prefix**: `projectname-barcode01.fastq.gz`
+    - **Not Recommended**: `projectname_2024_test-barcode01.fastq.gz` (would recognize only `projectname` as the sample name, leading to ambiguity with multiple files).
+
+### Workflow Tasks
+
+This workflow is composed of several tasks to process, basecall, and analyze rabies genome data:
+
+??? task "`Dorado Basecalling`: Converts `POD5` files to 'SAM' files"
+    The basecalling task takes `POD5` files as input and converts them into 'SAM' format using the specified model. This step leverages GPU acceleration for efficient processing.
+
+??? task "`Samtools Convert`: Converts SAM to BAM"
+    Once the SAM files are generated, this task converts them into BAM format, optimizing them for downstream applications and saving storage space.
+
+??? task "`Dorado Demultiplexing`: Produces barcode-specific FASTQ files"
+    This task demultiplexes the BAM files based on barcodes, generating individual FASTQ files for each barcode to support further analyses.
+
+??? task "`FASTQ File Transfer`: Transfers files to Terra"
+    After demultiplexing, the FASTQ files are uploaded to Terra for storage and potential use in other workflows.
+
+??? task "`Terra Table Creation`: Creates a Terra table with FASTQ files"
+    A Terra table is created to index the uploaded FASTQ files, enabling easy access and integration with other workflows for downstream analyses.
+
+
+### Outputs
+
+| **Variable** | **Type** | **Description** |
+|---|---|---|
+| **fastq_files** | Array[File] | FASTQ files produced from basecalling and demultiplexing |
+| **terra_table_tsv** | File | TSV file for Terra table upload |
+| **dorado_version** | String | Version of Dorado used in the workflow |
+| **dorado_model** | String | Model used for basecalling |
+|  **samtools_version** | String | Version of Samtools used in the workflow |
+|  **dorado_analysis_date** | String | Date of Dorado analysis |
+|  **dorado_phb_version** | String | Version of Dorado PHB workflow |
+
+## References
+<!-- -->
+><https://github.com/nanoporetech/dorado/>

docs/workflows_overview/workflows_alphabetically.md

@@ -18,6 +18,7 @@ title: Alphabetical Workflows
 | [**Core_Gene_SNP**](../workflows/phylogenetic_construction/core_gene_snp.md) | Pangenome analysis | Bacteria | Set-level | Some optional features incompatible, Yes | v2.1.0 | [Core_Gene_SNP_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Core_Gene_SNP_PHB:main?tab=info) |
 | [**Create_Terra_Table**](../workflows/data_import/create_terra_table.md)| Upload data to Terra and then run this workflow to have the table automatically created | Any taxa | | Yes | v2.2.0 | [Create_Terra_Table_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Create_Terra_Table_PHB:main?tab=info) |
 | [**CZGenEpi_Prep**](../workflows/phylogenetic_construction/czgenepi_prep.md)| Prepare metadata and fasta files for easy upload to the CZ GEN EPI platform. | Monkeypox virus, SARS-CoV-2, Viral | Set-level | No | v1.3.0 | [CZGenEpi_Prep_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/CZGenEpi_Prep_PHB:main?tab=info) |
+| [**Dorado_Basecalling**](../workflows/standalone/dorado_basecalling.md)| GPU-accelerated basecalling of Oxford Nanopore sequencing data | Any taxa | Sample-level | Yes | v2.3.0 | [Dorado_Basecalling_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Dorado_Basecalling_PHB:main?tab=info) |
 | [**Find_Shared_Variants**](../workflows/phylogenetic_construction/find_shared_variants.md)| Combines and reshapes variant data from Snippy_Variants to illustrate variants shared across multiple samples | Bacteria, Mycotics | Set-level | Yes | v2.0.0 | [Find_Shared_Variants_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Find_Shared_Variants_PHB:main?tab=info) |
 | [**Freyja Workflow Series**](../workflows/genomic_characterization/freyja.md)| Recovers relative lineage abundances from mixed sample data and generates visualizations | SARS-CoV-2, Viral | Sample-level, Set-level | Yes | v2.2.0 | [Freyja_FASTQ_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Freyja_FASTQ_PHB:main?tab=info), [Freyja_Plot_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Freyja_Plot_PHB:main?tab=info), [Freyja_Dashboard_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Freyja_Dashboard_PHB:main?tab=info), [Freyja_Update_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Freyja_Update_PHB:main?tab=info) |
 | [**GAMBIT_Query**](../workflows/standalone/gambit_query.md)| Taxon identification of genome assembly using GAMBIT | Bacteria, Mycotics | Sample-level | Yes | v2.0.0 | [Gambit_Query_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Gambit_Query_PHB:main?tab=info) |

docs/workflows_overview/workflows_kingdom.md

@@ -16,6 +16,7 @@ title: Workflows by Kingdom
 | [**BaseSpace_Fetch**](../workflows/data_import/basespace_fetch.md)| Import data from BaseSpace into Terra | Any taxa | Sample-level | Yes | v2.0.0 | [BaseSpace_Fetch_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/BaseSpace_Fetch_PHB:main?tab=info) |
 | [**Concatenate_Column_Content**](../workflows/data_export/concatenate_column_content.md) | Concatenate contents of a specified Terra data table column for many samples ("entities") | Any taxa | Set-level | Yes | v2.1.0 | [Concatenate_Column_Content_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Concatenate_Column_Content_PHB:main?tab=info) |
 | [**Create_Terra_Table**](../workflows/data_import/create_terra_table.md)| Upload data to Terra and then run this workflow to have the table automatically created | Any taxa | | Yes | v2.2.0 | [Create_Terra_Table_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Create_Terra_Table_PHB:main?tab=info) |
+| [**Dorado_Basecalling**](../workflows/standalone/dorado_basecalling.md)| GPU-accelerated basecalling of Oxford Nanopore sequencing data | Any taxa | Sample-level | Yes | v2.3.0 | [Dorado_Basecalling_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Dorado_Basecalling_PHB:main?tab=info) | 
 | [**Kraken2**](../workflows/standalone/kraken2.md) | Taxa identification from reads | Any taxa | Sample-level | Yes | v2.0.0 | [Kraken2_PE_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Kraken2_PE_PHB:main?tab=info), [Kraken2_SE_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Kraken2_SE_PHB:main?tab=info) |
 | [**NCBI_Scrub**](../workflows/standalone/ncbi_scrub.md)| Runs NCBI's HRRT on Illumina FASTQs | Any taxa | Sample-level | Yes | v2.2.1 | [NCBI_Scrub_PE_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/NCBI_Scrub_PE_PHB:main?tab=info), [NCBI_Scrub_SE_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/NCBI_Scrub_SE_PHB:main?tab=info) |
 | [**RASUSA**](../workflows/standalone/rasusa.md)| Randomly subsample sequencing reads to a specified coverage | Any taxa | Sample-level | Yes | v2.0.0 | [RASUSA_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/RASUSA_PHB:main?tab=info) |

docs/workflows_overview/workflows_type.md

@@ -97,6 +97,7 @@ title: Workflows by Type
 | **Name** | **Description** | **Applicable Kingdom** | **Workflow Level** | **Command-line Compatibility**[^1] | **Last Known Changes** | **Dockstore** |
 |---|---|---|---|---|---|---|
 | [**Cauris_CladeTyper**](../workflows/standalone/cauris_cladetyper.md)| C. auris clade assignment | Mycotics | Sample-level | Yes | v1.0.0 | [Cauris_CladeTyper_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Cauris_CladeTyper_PHB:main?tab=info) |
+| [**Dorado_Basecalling**](../workflows/standalone/dorado_basecalling.md)| GPU-accelerated basecalling of Oxford Nanopore sequencing data | Any taxa | Sample-level | Yes | v2.3.0 | [Dorado_Basecalling_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Dorado_Basecalling_PHB:main?tab=info) |
 | [**GAMBIT_Query**](../workflows/standalone/gambit_query.md)| Taxon identification of genome assembly using GAMBIT | Bacteria, Mycotics | Sample-level | Yes | v2.0.0 | [Gambit_Query_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Gambit_Query_PHB:main?tab=info) |
 | [**Kraken2**](../workflows/standalone/kraken2.md) | Taxa identification from reads | Any taxa | Sample-level | Yes | v2.0.0 | [Kraken2_PE_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Kraken2_PE_PHB:main?tab=info), [Kraken2_SE_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/Kraken2_SE_PHB:main?tab=info) |
 | [**NCBI-AMRFinderPlus**](../workflows/standalone/ncbi_amrfinderplus.md)| Runs NCBI's AMRFinderPlus on genome assemblies (bacterial and fungal) | Bacteria, Mycotics | Sample-level | Yes | v2.0.0 | [NCBI-AMRFinderPlus_PHB](https://dockstore.org/workflows/github.com/theiagen/public_health_bioinformatics/NCBI-AMRFinderPlus_PHB:main?tab=info) |

mkdocs.yml

@@ -52,6 +52,7 @@ nav:
                - Zip_Column_Content: workflows/data_export/zip_column_content.md
             - Standalone:
                - Cauris_CladeTyper: workflows/standalone/cauris_cladetyper.md
+               - Dorado_Basecalling: workflows/standalone/dorado_basecalling.md
                - GAMBIT_Query: workflows/standalone/gambit_query.md
                - Kraken2: workflows/standalone/kraken2.md
                - NCBI-AMRFinderPlus: workflows/standalone/ncbi_amrfinderplus.md
@@ -67,6 +68,7 @@ nav:
                - BaseSpace_Fetch: workflows/data_import/basespace_fetch.md
                - Concatenate_Column_Content: workflows/data_export/concatenate_column_content.md
                - Create_Terra_Table: workflows/data_import/create_terra_table.md
+               - Dorado_Basecalling: workflows/standalone/dorado_basecalling.md
                - Kraken2: workflows/standalone/kraken2.md
                - NCBI-Scrub: workflows/standalone/ncbi_scrub.md
                - RASUSA: workflows/standalone/rasusa.md
@@ -126,6 +128,7 @@ nav:
             - Core_Gene_SNP: workflows/phylogenetic_construction/core_gene_snp.md
             - Create_Terra_Table: workflows/data_import/create_terra_table.md
             - CZGenEpi_Prep: workflows/phylogenetic_construction/czgenepi_prep.md
+            - Dorado_Basecalling: workflows/standalone/dorado_basecalling.md
             - Find_Shared_Variants: workflows/phylogenetic_construction/find_shared_variants.md
             - Freyja Workflow Series: workflows/genomic_characterization/freyja.md
             - GAMBIT_Query: workflows/standalone/gambit_query.md