Fix generated search URLs for project statistics

[haxtibal]

Search URLs generated for the project statistic page were caught by Jinja2 autoescaping, which makes them invalid.

A good place to mark them as safe is the view object: It's the immediate layer below Jinja templates and specifically made to provide strings included by a Jinja2 template.

Relates to #1920.

[haxtibal]

Unrelated to this MR: The new e2e test show two results that I wouldn't have expected

Root-level requirements not connected to by any requirement 0

This always remains 0, no matter where I place unconnected requirements. What is it meant to count?

Non-root-level requirements not connected to any parent requirement 4

Should IMO be 5. It seems that node["STATUS"] != "Backlog" evaluates to False if STATUS is not defined. I would expect logic like undefined STATUS != "Backlog" == True. Just like NaN ≠ x is always True.

[haxtibal]

render_url() and friends should internally do URL escaping with something like urllib.parse.urlencode, then mark as safe. This is currently not considered, but we could add it to the TODO list in #1920.

[stanislaw]

Unrelated to this MR: The new e2e test show two results that I wouldn't have expected

Root-level requirements not connected to by any requirement 0

This always remains 0, no matter where I place unconnected requirements. What is it meant to count?

A root-level requirement becomes 'connected' if there is another requirement that has a RELATION to it.

Non-root-level requirements not connected to any parent requirement 4

Should IMO be 5. It seems that node["STATUS"] != "Backlog" evaluates to False if STATUS is not defined. I would expect logic like undefined STATUS != "Backlog" == True. Just like NaN ≠ x is always True.

This logic was written with the assumption that the Backlog requirements do not contribute to these project statistics. If a requirement has no status, it can then be found with another metric and its status should be set to a meaningful vallue. I think it is pretty consistent or?

[stanislaw]

render_url() and friends should internally do URL escaping with something like urllib.parse.urlencode, then mark as safe. This is currently not considered, but we could add it to the TODO list in #1920.

Added!

[haxtibal]

A root-level requirement becomes 'connected' if there is another requirement that has a RELATION to it.

Yes but the query says "not connected to". Nothing in the test input.sdoc is connected (there are no relations), so this should be 5, not 0. The question comes down to: How would node.is_root as standalone query become True? For me it evaluates to False even for requirements directly written under [DOCUMENT].

This logic was written with the assumption that the Backlog requirements do not contribute to these project statistics. If a requirement has no status, it can then be found with another metric and its status should be set to a meaningful vallue. I think it is pretty consistent or?

Not sure. All other queries in project statistics are written so that backlog requirements do contribute. That's a bit inconsistent.

Anyway, I made a wrong assumption in my previous comment:

It seems that node["STATUS"] != "Backlog" evaluates to False

It actually evaluates to True and that's good.

[stanislaw]

A root-level requirement becomes 'connected' if there is another requirement that has a RELATION to it.

Yes but the query says "not connected to". Nothing in the test input.sdoc is connected (there are no relations), so this should be 5, not 0. The question comes down to: How would node.is_root as standalone query become True? For me it evaluates to False even for requirements directly written under [DOCUMENT].

Now I understand what was causing the confusion. The document needs to be marked as ROOT: True, see this for example:

[DOCUMENT]
MID: b7bcca934cd148c3855beb4c85723bc5
TITLE: Requirements Tool Specification (L1)
REQ_PREFIX: SDOC-SSS-
ROOT: True
OPTIONS:
  ENABLE_MID: True

I found no better way of assigning a root because StrictDoc cannot know what is a root of a graph.

This logic was written with the assumption that the Backlog requirements do not contribute to these project statistics. If a requirement has no status, it can then be found with another metric and its status should be set to a meaningful vallue. I think it is pretty consistent or?

Not sure. All other queries in project statistics are written so that backlog requirements do contribute. That's a bit inconsistent.

Yes, I can see a bit of consistency. The idea was to exclude Backlog reqs clearly from the not connected to-stats because otherwise you could end up with a non-zero number for these stats even if the non-backlog requirements are all traced.

Which / are there any other metrics you would want to see with != Backlog too?

Anyway, I made a wrong assumption in my previous comment:

It seems that node["STATUS"] != "Backlog" evaluates to False

It actually evaluates to True and that's good.

Good.

[haxtibal]

Now I understand what was causing the confusion. The document needs to be marked as ROOT: True

Now I understand too :) Thanks for explaining. Just learnt one can mark even multiple documents with ROOT: True. This makes sense in our use case. We have multiple documents that can be considered a possible source (= root?) of requirements

• external requirements from IEC, connected via a compliance matrix (as not all external requirements apply, it could make sense to consider the compliance matrix document as root)
• requirements to the operating system
• requirements to the application

Which / are there any other metrics you would want to see with != Backlog too?

Up to now we don't use the Backlog state. But we have things like a "Proposed" state for REQUIREMENT and for custom grammar and have not defined metrics yet. Guess it's hard to find a "fits everywhere" statistics page. Maybe it would make sense to be able to define a custom statistics page (with custom queries)?