Unhandled Exception when Importing Doors Project in ReqIF

[stellarpower]

Requirements software is new to me, so am limited in how much useful information I can provide. I'm simply looking to visualise the data in an export I've been provided.

2 ❯ strictdoc import reqif --reqif-enable-mid sdoc '/ProjectFromDoors/ReqIFv10.reqif' output.sdoc
Step 'Import' start.
Traceback (most recent call last):
  File "/...//bin/strictdoc", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/...//lib/python3.12/site-packages/strictdoc/cli/main.py", line 200, in main
    raise exc
  File "/...//lib/python3.12/site-packages/strictdoc/cli/main.py", line 195, in main
    _main(parallelizer)
  File "/...//lib/python3.12/site-packages/strictdoc/cli/main.py", line 117, in _main
    import_action.do_import(import_reqif_config, project_config)
  File "/...//lib/python3.12/site-packages/strictdoc/helpers/timing.py", line 18, in wrap
    result = func(*args, **kw)
             ^^^^^^^^^^^^^^^^^
  File "/...//lib/python3.12/site-packages/strictdoc/core/actions/import_action.py", line 31, in do_import
    reqif_documents: List[SDocDocument] = ReqIFImport.import_from_file(
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/...//lib/python3.12/site-packages/strictdoc/backend/reqif/reqif_import.py", line 45, in import_from_file
    documents: List[SDocDocument] = converter.convert_reqif_bundle(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/...//lib/python3.12/site-packages/strictdoc/backend/reqif/p01_sdoc/reqif_to_sdoc_converter.py", line 76, in convert_reqif_bundle
    document = P01_ReqIFToSDocConverter._create_document_from_reqif_specification(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/...//lib/python3.12/site-packages/strictdoc/backend/reqif/p01_sdoc/reqif_to_sdoc_converter.py", line 121, in _create_document_from_reqif_specification
    grammar_element: GrammarElement = P01_ReqIFToSDocConverter.create_grammar_element_from_spec_object_type(
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/...//lib/python3.12/site-packages/strictdoc/backend/reqif/p01_sdoc/reqif_to_sdoc_converter.py", line 283, in create_grammar_element_from_spec_object_type
    raise NotImplementedError(attribute) from None
NotImplementedError: SpecAttributeDefinition(attribute_type = SpecObjectAttributeType(...), identifier = "_d8bc2b92-7a0f-455b-aea7-8aefe398aa5f_ABSOLUTENUMBER", datatype_definition = "_b5e1292d-cde6-473e-be3d-9b138e1add56", xml_node = _Element(...), description = "System Attribute", last_change = "2024-08-07T19:17:17", long_name = "ReqIF.ForeignID", editable = True, default_value_definition_ref = NoneType(...), default_value = NoneType(...), multi_valued = NoneType(...))

Don't know if this belongs in the ReqIF package or as part of strictdoc proper; feel free to move if appropriate. I guess if this is an unsupported feature, at the least it'd be nice to catch it and then print a user-readable message explaining what feature isn't supported.

Thanks

[stanislaw]

Hi @stellarpower,

The ReqIF conversion is not an easy task, and from time to time, there is always a user who has something in their ReqIF that we have previously not seen. From looking at the error location, it is most likely that your file has numeric spec attribute definitions (for example, of type INTEGER or REAL), and this is not handled by StrictDoc yet.

The error message could be improved indeed. In order to fix this issue for your case, I would need a good MRE ReqIF example that you are using with StrictDoc. Is there a chance you could send me an anonymized ReqIF example without any sensitive data but still the one that reproduces this issue?

[stellarpower]

Thanks, and totally understood. Standards are great except when they aren't standardised and just a free-for-all.

Interestingly I'm not seeing the same problems when reading in the file programatically using the python package. This is where the anonymisation script is helpful ;)

ReqIFv10 - Anonymised.reqif.txt

This may not be entirely minimal, but it's a small dummy project export from Doors anyway and you know what you're looking for. I did see those numeric definitions before so expect that's probably it. Don't know why they exist TBH, needing to specify what a boolean or a string is within the requirements and assign it a UUID seems a bit excessive to me. But ours (well, mine) is not to question why!

Cheers

[stanislaw]

Hey @stellarpower,

I have tried generating SDoc from your ReqIF file and found a few issues. Some of them are minor limitations of StrictDoc that can be fixed, and some are related to the ReqIF properties that I didn't encounter before in the ReqIF files provided by the users.

Here is the branch that handles your file with an integration test: https://github.com/strictdoc-project/strictdoc/compare/stanislaw/reqif_numeric_attributes?expand=1

Your ReqIF file has two unusual details that I haven't seen before. This is not to say that something is wrong with this ReqIF, it is just that it makes things a little more complicated for detecting things correctly and mapping them to SDoc.

1. This ReqIF has two <SPEC-OBJECT-TYPE.... declarations and both of them don't have a LONG-NAME. Usually a LONG-NAME has a human-readable value such as Requirement, HIGH-LEVEL REQUIREMENT or something similar to help a user understand which object type it is. StrictDoc uses this LONG-NAME to map it directly to a StrictDoc grammar tag such as [REQUIREMENT] or similar. Without this information, StrictDoc can only generate something like: NODE_ZCFAELGJ where the right part of this string is a random string to avoid collisions.

2. Inside your ReqIF file there are two documents (ReqIF tag is <SPECIFICATION...>). Both of these documents have their elements (ReqIF term SPEC-OBJECTs) with ReqIF.ForeignID starting from 1, 2, ... This results in duplicate identifiers in the documentation tree which is something that StrictDoc prohibits by design but also something that we could change.

Previously, the users could integrate SDoc with Doors, so with some tweaks in your document structure we could make it work. Otherwise, I could think of extending StrictDoc to make it more flexible when it comes to supporting the ReqIF files like yours.

How eager are you to set up an end-to-end ReqIF-SDoc export/import, and can you control the structure/settings of this ReqIF? We can certainly make it work, the question is how soon, depending on what you want to do with this document structure.

[stellarpower]

Thanks a lot, appreciate the time!

So, this was originally for a work project - an export from IBM Doors (afraid I don't have the version to hand right now).

I think we're largely wrapped up on that stage of the project - but quite probably will come back to it at some point in the future. It was sort of a PoC-level thing to start with. So afraid I won't have time in the near future to take a look again, but if I end up cycling back to parsing data on this project later will try to check out your branch and see what I get, as well as try to get you more details about the exact Doors version (it's on a clients' infrastructure, so we don't use it ourselves but are working with them on the data. I don't entirely have direct access). To answer your question, I think longer-term, potentially integrating with Doors using their query language thing (as I believe the previous export did as you're mentioning) is something that's been discussed. For now we are just getting the data out of doors and into our own code, but round-trip is definitely something we may look at doing if this project continues to be expanded. I expect strictdoc as well as some of the other FOSS packages out there could be very helpful with working out how we do this.

If we get more files in the medium-term, I might also be able to anonymise those and get them to you. Not knowing what I'm looking for, it was a bit hard to get a handle on what's going on in these files when each viewer I tried was struggling with them and without having a local copy of doors I have full access to. So unfortunately not of much help as to what's really in them in terms of the XML structure or why they're a bit exotic. Eventually after several java versions I did manage to get ReqIFStudio working on them, I don't know if you've used it but might help. The somewhat older tool that's more like a full version of Eclipse was next to useless.

Anyway sorry not to be more help but glad at least you can reproduce it and it's given some insights. I'll be back in touch when my todo list can accommodate it. Greatly appreciate the help and best of luck in the meantime!