SAM/VCF Strict draft #283

d-cameron · 2018-02-05T04:42:30Z

PR for discussing proposed SAM Strict rule changes. Work in progress continuing from #281 discussion.

d-cameron · 2018-02-05T05:01:55Z

@jkbonfield I'd rather not expand this particular PR to include changes to the SAM specs themselves. My PR does appear to be highlighting some issues with SAMv1.5 that have avoided scrutiny, but I think the best course of action is to get a SAM v1.5 strict document and validator implementation based on the v1.5 specifications before updating the specs.

jkbonfield · 2018-02-05T09:51:26Z

That's fine - I can accept that as an order in which things are done. Yes I think there are problems with the spec which haven't been spotted as no one has really gone over it with such a fine tooth-comb before. Thank you for doing this.

yfarjoun · 2018-02-08T13:35:40Z

Could you do me a favor and include the pdf in these commits? when we get to the end you can remove the pdf and compile it "correctly" for the sake of having the PDF versioned.

yfarjoun · 2018-02-08T13:37:53Z

I was wondering if we want the error "code" to not have spaces in it. This will help when we want to run a validation tool on the commandline and specify which codes to ignore. Spaces will mean that every code needs to be surrounded by quotes. COULD_WE_USE_THIS_FORMAT_FOR_CODES? (not shouting, just suggesting!)

jkbonfield · 2018-02-08T14:23:37Z

The TeX is invalid at the moment, meaning a PDF cannot currently be produced.

I'm not a TeX expert, but it looks like you need eg \vfifteen instead of \v15 commands and braces around the {\tt text} sections. There are some other minor things like \bam not being defined too. I tried correcting this and have a pdf that builds now, although it's formatted wrong and becomes pure fixed-width after a while.

jkbonfield · 2018-02-08T14:58:44Z

The version in https://github.com/jkbonfield/hts-specs/tree/strict builds now, but needs a lot of formatting still. I'm not planning on doing this myself - this was just a test to see how to get it in a readable format.

yfarjoun

Great work!! there seems to be a lo of work left at the bottom part of the document, but I like the rigor so far.

I put some comments in the discussion...

yfarjoun · 2018-02-08T13:17:07Z

SAMstrict.tex

+
+\section{Headers}
+
+{paragraph}


is there a \begin or a \section missing here? looks like copy-paste error

yfarjoun · 2018-02-08T13:17:17Z

SAMstrict.tex

+
+{paragraph}
+
+The first segment is is considered to be the "next" segment of the final segment in a template as per the SAM specifications.


this seems out of place

yfarjoun · 2018-02-08T13:18:42Z

SAMstrict.tex

+
+
+\section{Headers}
+\samstrictrule{Undefined reserved header present}{Upper-case header record type codes are not defined in the SAM specifications must not be used.}


missing "that" between codes and are (or remove are if that's the style you're going for)

yfarjoun · 2018-02-08T13:23:15Z

SAMstrict.tex

+\section{Headers}
+\samstrictrule{Undefined reserved header present}{Upper-case header record type codes are not defined in the SAM specifications must not be used.}
+\samstrictrule{Undefined header tag present}{Upper-case header tags not defined in the SAM specifications must not be used.}
+\samstrictrule{Tag present as both lowercase and uppercase}{A file should not contain the same tag in both upper-case and lowercase format. See the SAM specifications header tags best practice footnote.}


tag is ambiguous, should be header tag (both in code and text)

I don't interpret that foot-note the same way.....I interpret it as "lower case is a free for all, for upper case tags open an issue in hts-spec"

yfarjoun · 2018-02-08T13:25:18Z

SAMstrict.tex

+\samstrictrule{Undefined reserved header present}{Upper-case header record type codes are not defined in the SAM specifications must not be used.}
+\samstrictrule{Undefined header tag present}{Upper-case header tags not defined in the SAM specifications must not be used.}
+\samstrictrule{Tag present as both lowercase and uppercase}{A file should not contain the same tag in both upper-case and lowercase format. See the SAM specifications header tags best practice footnote.}
+\samstrictrule{Malformed header}{Header lines with conform to either the {\tt


"malformed header line"
s/with/must/
either is generally used with two options...could you reword without either?

yfarjoun · 2018-02-08T15:13:55Z

SAMstrict.tex

+\samstrictrule{SEQ QUAL length mismatch.}{The length of a non-* QUAL must match the length of SEQ.}{\samrule}
+\samstrictrule{Invalid QUAL}{The ASCII value of all QUAL bases must be at least 33.}
+\samstrictrule{QUAL of secondary alignments specified.}{QUAL of secondary alignments (0x100 FLAG set) should be set to * to reduce the file size.}{\v15bestpractice}
+\samstrictrule{TODO: QUAL edge case}{What should we do when a read is length 1 and the QUAL encodes to "*" ?}{\samrule}


I have to say that spending any time on this edge case seems to be a waste...

I think I've actually hit that one before! Hah.

yfarjoun · 2018-02-08T15:17:43Z

SAMstrict.tex

+\samstrictrule{Unknown reserved tag}{
+	No record can include any reserved tags not defined in the
+	{\sl Sequence Alignment/Map Optional Fields Specification}.
+	Non-standard tags must start X, Y, Z or a lowercase letter as per the SAM specifications.


...must start with...

yfarjoun · 2018-02-08T15:18:28Z

SAMstrict.tex

+}
+\samstrictrule{Incorrect tag type}{The \textit{type} of all \textit{standard tags} must match the type
+defined in the {\sl Sequence Alignment/Map Optional Fields Specification}.}
+\samstrictrule{Duplicate tag}{}{\samrule}


no text for "duplicate tag"?

yfarjoun · 2018-02-08T15:19:47Z

SAMstrict.tex

+
+
+\subsection{RG}
+3 When a RG tag appears anywhere in the alignment section, there should be a single corresponding


remove 3 and 4 here and below

should->must

also, need error codes

yfarjoun · 2018-02-08T15:20:42Z

SAMstrict.tex

+3 When a RG tag appears anywhere in the alignment section, there should be a single corresponding
+@RG line with matching ID tag in the header.{\v15bestpractice}
+\subsection{RG}
+4 When a PG tag appears anywhere in the alignment section, there should be a single corresponding


is this duplicated?

jmarshall · 2020-07-14T13:23:05Z

Any discussion or review comments on SAM and VCF strictness proposals are naturally separate. Would it not be more practical to open a VCF strict proposal as a new separate PR?

tskir · 2020-11-09T15:18:49Z

I agree with a comment from @jmarshall above about splitting this into two. Since in its current form it concerns both VCF and SAM, I've added the appropriate labels for the time being

d-cameron · 2023-06-20T15:39:31Z

Happy to progress these specs if someone has funding/budget/time to actually write a validator

yfarjoun reviewed Feb 8, 2018

View reviewed changes

d-cameron mentioned this pull request Sep 26, 2019

Should we clarify the intent of the duplicate SAM flag 0x400 ? #445

Open

brainstorm mentioned this pull request Dec 10, 2019

Signedness for fields that should only represent positive values such as lengths #460

Closed

d-cameron mentioned this pull request May 28, 2020

We got a grant to improve Htsjdk! samtools/htsjdk#1482

Open

d-cameron changed the title ~~DO NOT MERGE SAM Strict draft~~ SAM/VCF Strict draft Jul 13, 2020

Rebased SAM and VCF strict specifications drafts

feaeb69

d-cameron force-pushed the strict branch from 14b4ca0 to feaeb69 Compare July 14, 2020 04:33

tskir added sam vcf labels Nov 9, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SAM/VCF Strict draft #283

SAM/VCF Strict draft #283

d-cameron commented Feb 5, 2018

d-cameron commented Feb 5, 2018

jkbonfield commented Feb 5, 2018

yfarjoun commented Feb 8, 2018

yfarjoun commented Feb 8, 2018

jkbonfield commented Feb 8, 2018

jkbonfield commented Feb 8, 2018

yfarjoun left a comment

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

jkbonfield Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

yfarjoun Feb 8, 2018

jmarshall commented Jul 14, 2020 •

edited

Loading

tskir commented Nov 9, 2020

d-cameron commented Jun 20, 2023


		{paragraph}

		The first segment is is considered to be the "next" segment of the final segment in a template as per the SAM specifications.



		\section{Headers}
		\samstrictrule{Undefined reserved header present}{Upper-case header record type codes are not defined in the SAM specifications must not be used.}



		\subsection{RG}
		3 When a RG tag appears anywhere in the alignment section, there should be a single corresponding

SAM/VCF Strict draft #283

Are you sure you want to change the base?

SAM/VCF Strict draft #283

Conversation

d-cameron commented Feb 5, 2018

d-cameron commented Feb 5, 2018

jkbonfield commented Feb 5, 2018

yfarjoun commented Feb 8, 2018

yfarjoun commented Feb 8, 2018

jkbonfield commented Feb 8, 2018

jkbonfield commented Feb 8, 2018

yfarjoun left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmarshall commented Jul 14, 2020 • edited Loading

tskir commented Nov 9, 2020

d-cameron commented Jun 20, 2023

jmarshall commented Jul 14, 2020 •

edited

Loading