[TheiaCoV] Split database from Kraken2_TheiaCoV task

[cimendes]

This PR closes #563, closes #520

🗑️ This dev branch should be deleted after merging to main.

🧠 Summary

This PR addresses several issues that came up when the RSV target organism was set differently for RSV-A and RSV-B, i.e. rsv_a_kraken_target_organism = "Respiratory syncytial virus" and rsv_b_kraken_target_organism = "Human orthopneumovirus" due to the use of an old database. As the existing docker image had an embedded database, it meant that each time the Kraken database would need to be updated, it would have to be done inside the docker container, which would not be efficient.

This PR simply updates the existing kraken2_theiacov task to split the database from the container. The new database is custom built with human + refseq viral sequences as of 2024-08-28.

No inputs or outputs have been altered other than what is strictly required to make the new code changes functional.

Behaviour change:

• the percentage of sc2 is now only reported for TheiaCoV is the target organism is indeed SC2. To account for this the output type has been changed from Float to String.
• Freyja now has a kraken2_target_organism input that by default is SC2, but is user modifyable

⚡ Impacted Workflows/Tasks

Impacted tasks

• kraken2_theiacov
• kraken2_parse_classified

Impacted workflows

• TheiaCoV
  • Illumina_PE
  • Illumina_SE
  • Clearlabs
  • ONT
• Freyja_FASTQ
• NCBI_Scrub
  • SE
  • PE

This PR may lead to different results in pre-existing outputs: Yes

• Use of an updated database may reflect changes in how taxa are identified.

This PR uses an element that could cause duplicate runs to have different results: No

🛠️ Changes

⚙️ Algorithm

As explained above, the only change in algorithm is that the percentage of sc2 is now only reported if the target organism is SC2, otherwise this column is blank. To accomplish this the data type has been changed from Float to String. Impacts column sorting as it is now alpha-numeric.

➡️ Inputs

freyja_fastq new input:

• kraken2_target_organism

⬅️ Outputs

Percent sc2 is now a string output, rather than a float output

🧪 Testing

• Freyja_FASTQ (SC2 PE): https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Mendes_Sandbox/job_history/3c8f46c1-a0b7-44cb-b949-10f6323b9c9b ⚠️ 2 failures but unrelated, they are weird tho!
• TheiaCoV
  • ClearLabs (SC2 only): https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Mendes_Sandbox/job_history/e917f233-897d-43b9-b6ff-1d6e2d594c61 ✅
  • SE (multiple orgs, including RSV-A and RSV-B): https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Mendes_Sandbox/job_history/ddc6bb13-c026-46ca-a8f6-4f09b0f97722 ⚠️ unrelated failures in 3 wnv samples, maybe worth investigating?
  • ONT (multiple orgs): https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Mendes_Sandbox/job_history/f30871bd-ebda-4133-8699-9b120d4023f7 ⚠️ failures in all HIV samples because HIV hates me, but unrelated to the PR
  • PE (multiple orgs, including RSV-A and RSV-B): https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Mendes_Sandbox/job_history/b5fae176-0f07-4f8e-9266-e66aaec7e7cd ⚠️ 1 failure in wnv unrelated to the PR, same error as observed in SE runs
• NCBI_Scrub
  • PE (SC2): https://app.terra.bio/#workspaces/theiagen-validations/ines_sandbox/job_history/c4360ec0-9779-440e-9982-ac10e2fc5361 ✅
  • SE (SC2): https://app.terra.bio/#workspaces/theiagen-validations/ines_sandbox/job_history/9c222e8c-4291-49c7-8453-011353e393f2 ✅

Suggested Scenarios for Reviewer to Test

🔬 Final Developer Checklist

• The workflow/task has been tested and results, including file contents, are as anticipated
• The CI/CD has been adjusted and tests are passing (Theiagen developers)
• Code changes follow the style guide
• Documentation and/or workflow diagrams have been updated if applicable (Theiagen developers only)

🎯 Reviewer Checklist

• All changed results have been confirmed
• You have tested the PR appropriately (see the testing guide for more information)
• All code adheres to the style guide
• MD5 sums have been updated
• The PR author has addressed all comments
• The documentation has been updated

[Michal-Babins]

For NCBI scrub workflows can we set this to be a workflow input and not hard coded on the task level?

[Michal-Babins]

For ncbi scrubber, please pass in workflow level variable and do not hard code on task level.

[Michal-Babins]

Same here for workflow level input

[Michal-Babins]

Testing illumina PE here for WNV, MPox, Flu, Sars-Cov-2.

[Michal-Babins]

Everything looks good in the test and the output is properly identified by kraken from what we would expect. One of the flu samples returned nothing from kraken, but I ran it against PHBv2.2.1 and it was the same thing so something with that sample is just slightly off.

Run with isolated sample here