GrammarElementField-TypeValue and External-Refs

[GGBeer]

In my projects I often have the need to add references to external documents in the Requirement items, which set technical and/or regulatory boundaries for a project, e.g. must use crypto XYZ as defined by NIST, should implement TLS 1.3 as defined by RFC-8446, only ISO-15408 Common-Criteria certified products should be used, etc.
The requirement's statement and rationale should detail which specific item/section of such an ext. reference is to be fulfilled.
Currently this is handled by adding a simple custom grammar multistring field "EXT_REFS".
As usual, these external references should be collected in a list detailing the exact referenced items and its version to be applied.
Having different versions of a referenced item in this list should be avoided.

I am already working on a feature to extend the "REFS" field with an "External" variant.
While doing this, I noticed that the Custom GRAMMAR does not support a definition type for a "REFS" field matching the standard Parent-Reference and File-Reference fields.
i.e. the Custom GRAMMAR and grammar.py/type_system.py parts defining a GrammarElementFieldTypeValue TYPE and its subelements, which would allow to add the following to the GRAMMAR section:

  - TITLE: REFS
    TYPE: TypeValue(ParentReqReference, FileReference, ExternalReference)
    REQUIRED: False

which then would allow to do something like this in a REQUIREMENT:

[REQUIREMENT]
UID: SDOC.RQ.PKI.TLS.VERSION.4009d86a
UUID: 4009d86a-2482-4329-a4ce-b6f3e6586221
REQ_LEVEL: SHOULD
REFS:
- TYPE: Parent
  VALUE: SDOC.RQ.PKI.PROFILES.3161927f
- TYPE: External
  VALUE: RFC 5246 TLS Protocol Version 1.2, https://www.rfc-editor.org/rfc/rfc5246
- TYPE: External
  VALUE: RFC 8446 TLS Protocol Version 1.3, https://www.rfc-editor.org/rfc/rfc8446
- TYPE: File
  VALUE: myTls.cc
TITLE: TLS protocol version
STATEMENT: Only TLS protocol version 1.2 and/or 1.3 should be supported.
RATIONALE: For security reasons, only the newer TLS protocol versions should be used (currently TLS Version 1.2 and 1.3).

My suggestion for rendering the external refs:

• DOC => Just an additional "EXTERNAL REFS:" row in the Requirements table, having a list item for each external reference.
• TBL => An additional subsection ""External:" in the REFS column, having a list item for each external reference.

Would this feature be of interest to you?
Just have a look on my GrammarElementField-TypeValue branch

Thx,
Guus

[stanislaw]

Hi @GGBeer,

The external REFS feature looks like a nice addition! Let's integrate your changes but do so in a staged way: I noticed that your branch contains several changes which seem to be not necessarily related:

1. Replacing strings with RequirementFieldType.STRING and similar trivial changes (everywhere where you are introducing an enum instead of raw strings)
2. The trivial refactoring of has_children_requirements method
3. All other changes with the "meat" of your work.

I suggest you to open the changes 1, 2, and 3 as separate Pull Requests and I will trivially merge the 1 and 2. The changeset 3 should become more focused on your actual change.

I guess you have figured this out but just a reminder that there is a small guide: https://strictdoc.readthedocs.io/en/latest/strictdoc-6-contributing.html. You definitely want to add at least an integration test for the new grammar aspect around here: tests/integration/per_document_grammar/type_system. We will also need to add corresponding documentation.

Your change looks almost mergeable but please get ready to address some minor code review changes after you open the distilled PR after the point 3.

Thank you.

[GGBeer]

Some additional thoughts about the ExternalReference (type-value) items.

Defining SDoc encodings for (Externa)l References.
Add an addtional optional field "FORMAT" to define how the VALUE entry should be used/interpreted.
We do not want to define multiple ExternalXYZ ReferenceTypes or convert Encoding-Variants into specific SDoc grammar entries.

• Simple Text string per reference

- TYPE: External
  FORMAT: STRING  <-- Optional, defaults to STRING
  VALUE: Just some informational text string

• Full BibTex entry. Starting with @bibtexEntryName{citekey, tag=value, ... ,tag=value} e.g.

- TYPE: External
  FORMAT: BIBTEX
  VALUE: @techreport{ENX-093, title="Certificate CRL Profiles for ENX Version 3", author="G. Gerrits", institution="ENX Association", number="ENX-093", edition="3.2", date="2010-03-25", classification="restricted", note="Draft"}

• Reference to a BibTex' citekey entry, optionally followed by a string text with additional info.
  Note: The full BibTex entry should be included somewhere in the Document before the reference is made, i.e.
  • defined by a prior full (BibTex) Entry,
  • part of a new to be defined [BIBLIOGRAPHY] SDoc Node or
  • a new to be defined [INCLUDE_BIBLIOGRAPHY_FROM_FILE] SDoc directive

- TYPE: External
  FORMAT: CITATION
  VALUE: ENX-093, Section 3.2.5   <-- <CitationKey>, [AdditionalInfo]  <-- Optional STRING e.g. specific Section, Page, Table, Figure, etc.>

[BIBLIOGRAPHY]
ENTRIES:
  - FORMAT: BIBTEX
    VALUE: @techreport{ENX-093, title="Certificate CRL Profiles for ENX Version 3", author="G. Gerrits", institution="ENX Association", number="ENX-093", edition="3.2", date="2010-03-25", classification="restricted", note="Draft"}
  - FORMAT: BIBTEX
    VALUE: @misc{CitekeyMisc, title="Pluto: The 'Other' Red Planet", author= "{NASA}", url= "https://www.nasa.gov/nh/pluto-the-other-red-planet", howpublished = "\url{https://www.nasa.gov/nh/pluto-the-other-red-planet}", year=2015, note="Accessed: 2018-12-06"}

The other SDoc Reference variants could also be extended using the FORMAT field:

REFS:
- TYPE: Parent
  VALUE: <ParentRequirement.UID>
  FORMAT: REQUIREMENT_UID  <-- Optional, as UID is currently the only available default. 

REFS:
- TYPE: File
  VALUE: <Filepathname/URI/URN/...>
  FORMAT: PATH_NAME  <-- Optional, Default is the full/absolute Filepathname
        : SOURCE_CODE_FILE:[C, CPP, PY, JAVA, TEX, TAR, ...]
        : FILE_URI       <-- URL to retrieve the file from a (remote) server
        : GIT_OBJECT_URN <-- Reference to a specific Git-Commmit or Object

The rendering of Bibliography/Reference entries in SDoc is another subject.

OK, lets start with the simple STRING variant ;-)

Thx,
Guus