Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import simple NodeSet2 file is incomplete #510

Open
eoursel opened this issue Feb 6, 2021 · 30 comments
Open

Import simple NodeSet2 file is incomplete #510

eoursel opened this issue Feb 6, 2021 · 30 comments
Assignees
Labels
Feature request Consider to apply

Comments

@eoursel
Copy link

eoursel commented Feb 6, 2021

Describe the bug
When i try to import this NodeSet2 file which is just a useless type which contains some custom datatypes i observe several issues

  • ObjectTypes are empty (HumanType, VehicleType, TricycleType). They should contains some variables in Children.
  • The field wheel in TriccyelDataType should be an array.

To Reproduce
Import the attached NodeSet2 file.
Opc.Ua.NodeSet2.TriCycleType_V1.1.xml.zip

Additional context
Last source code from master branch

Best regards

@mpostol mpostol self-assigned this Feb 7, 2021
@mpostol
Copy link
Owner

mpostol commented Feb 7, 2021

Blocker for mpostol/ASMD#158

@mpostol mpostol transferred this issue from mpostol/ASMD Feb 7, 2021
@mpostol mpostol added the Bug Bug in the code. label Feb 7, 2021
@mpostol
Copy link
Owner

mpostol commented Feb 7, 2021

Transfer because the main functionality is implemented here.

@mpostol
Copy link
Owner

mpostol commented Feb 7, 2021

Further development will be conducted on the branch https://github.com/mpostol/OPC-UA-OOI/tree/SemanticData-6.1.3

@mpostol
Copy link
Owner

mpostol commented Feb 8, 2021

Testing scope

  • possibility to import NodeSet file
  • possibility to validate NodeSet file
  • possibility to convert NodeSet to ModelDesign
  • possibility to open in the ASMD
  • possibility to compile ModelDesign and generate NodeSet file (round trip test)
  • validation against the schema and compliance with the specification (Part 3 and Part 5) of the NodeSet file
  • correct generation of the model documentation in MS Word format
  • correct generation of website content with the model documentation

mpostol added a commit that referenced this issue Feb 8, 2021
- added Unit test against model in concern
- UT 👍 - no unexpected errors reported
mpostol added a commit that referenced this issue Feb 14, 2021
Test Name:	eoursel510Test
Test Outcome:	Passed
Result StandardOutput:
Debug Trace:
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering IAddressSpaceContext.ValidateAndExportModel - starting for the http://tricycletypev1/ namespace.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information AddressSpaceContext.ValidateAndExportModel - selected 8 nodes to be added to the model.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing Validator.ValidateExportModel - the model contains 8 nodes.
@mpostol
Copy link
Owner

mpostol commented Feb 14, 2021

@eoursel the result of ValidateAndExportModel is as follows:

Test Name:	eoursel510Test
Test Outcome:	Passed
Result StandardOutput:	
Debug Trace:
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering IAddressSpaceContext.ValidateAndExportModel - starting for the http://tricycletypev1/ namespace.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information AddressSpaceContext.ValidateAndExportModel - selected 8 nodes to be added to the model.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing Validator.ValidateExportModel - the model contains 8 nodes.

generic export implementation has been used for testing.

@eoursel
Copy link
Author

eoursel commented Feb 14, 2021

hi @mpostol,
in the nodeset files there are more than 8 nodes. looks like the variables in the ObjectTypes are lost. Could you post a copy of the generated model design file?

Thanks again

@mpostol
Copy link
Owner

mpostol commented Feb 15, 2021

@eoursel thanks, I haven't yet generated the ModelDesign file. The task possibility to convert NodeSet to ModelDesign is scheduled next` (see Testing scope above). Thanks for the feedback, I will check it before forwarding it to the next step, because if any node is not selected in this step it will be omitted in the phase of ModelDesign generation for sure. We must fix the problem now. I will be back.
Thanks,

@mpostol
Copy link
Owner

mpostol commented Feb 15, 2021

@eoursel Except for simple errors in the code (let exclude this case for a while), the reason for the problem you have reported is conditions used to select nodes to validate and export. Now the method ValidateAndExportModel uses conditions defined in Part 5 8 Standard Objects and their Variables - it exports selected model from an internal Address Space. It means that

  • if the node is not on the browse path available for an abstract client it is neglected.

Related issues:

[ ] reconsider conditions used for selection of Nodes
[ ] clarify what the the model contains 8 nodes actually means
[ ] add a warning that the AS contains nodes orphaned and inaccessible for browsing starting from the Root node

Comments are welcome, but for now, I will report the above-mentioned problems.

@eoursel
Copy link
Author

eoursel commented Feb 15, 2021

hi @mpostol, i don't get your point. All the nodes are on the browse path. Do you have a concrete example of such a node? . I am using Siemens Siome which do not display any error in the validation of the NodeSet file.

Regards

@mpostol
Copy link
Owner

mpostol commented Feb 15, 2021

@eoursel It is just a hypothesis. I will check it. At least this scenario is now not subject of validation, but in my opinion should be, shouldn't it? After improving this validation method I will get the list of nodes if any. I will work on it using above mentioned independent issues. I will be back with the results soon.

@mpostol
Copy link
Owner

mpostol commented Feb 16, 2021

@eoursel do you have a copy of the document OPC UA Companion Specification Template. It is not available on the OPC website - the page is empty for my account.

I am collecting materials from the spec that can be used to resolve some questions related to inheritance, inherited member, and instance declaration in the context of your problem.

@eoursel
Copy link
Author

eoursel commented Feb 16, 2021

Hi @mpostol should be in the harmonization working group. I am checking.

@mpostol
Copy link
Owner

mpostol commented Feb 17, 2021

Hi @eoursel, I have improved the error processing and logging infrastructure and the current result for the model in concern processing is as follows:

Test Name:	eoursel510Test
Test Outcome:	Passed
Result StandardOutput:	
Debug Trace:
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContext creator - starting creation the OPC UA Address Space.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Address Space - the OPC UA defined has been uploaded.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContext.ImportNodeSet - starting import UAOOI.SemanticData.UANodeSetValidation.XML.UAModelContext.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information AddressSpaceContext.ImportNodeSet - the context for the imported model is created and starting import nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing AddressSpaceContext.ImportNodeSet - imported 3909 nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Address Space - has bee created successfully.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContextService.ImportUANodeSet - importing form file
Trace: Information, Error Focus:Diagnostic, ErrorID: P0-0001010000 Info: The XML attribute or element is not supported and neglected. Extensions is omitted during the import
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContext.ImportNodeSet - starting import UAOOI.SemanticData.UANodeSetValidation.XML.UAModelContext.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information AddressSpaceContext.ImportNodeSet - the context for the imported model is created and starting import nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing AddressSpaceContext.ImportNodeSet - imported 54 nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering IAddressSpaceContext.ValidateAndExportModel - starting for the http://tricycletypev1/ namespace.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Selected 8 types to be validated.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Selected 0 instances referenced by the ObjectsFolder to be validated.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing Validator.ValidateExportModel - the model contains 8 nodes, and nor errors reported
After removing inherited and instance declaration nodes the recovered information model contains 8

The result says that the model contains 54 nodes, but only 8 types and 0 instances referenced by the ObjectsFolder are selected at the beginning to be processed. I have not implemented reporting orphans - it is more complicated than expected but we can assume that there is a lot of nodes no selected for processing or by design removed as modeling rules, instance declaration, or inherited. To be processed a node must be selected at the very beginning or must be on any reference chain. Initial choice aggregates all types and instances referenced by ObjectFolder standard object. To properly implement an orphaned nodes search mechanism let me analyze a snipped from the code

    <UAObjectType NodeId="ns=1;i=12" BrowseName="1:VehicleType">
        <DisplayName>VehicleType</DisplayName>
        <References>
            <Reference ReferenceType="HasSubtype" IsForward="false">i=58</Reference>
        </References>
    </UAObjectType>
    <UAVariable DataType="DateTime" NodeId="ns=1;i=13" BrowseName="buildDate" ParentNodeId="ns=1;i=12">
        <DisplayName>buildDate</DisplayName>
        <References>
            <Reference ReferenceType="HasProperty" IsForward="false">ns=1;i=12</Reference>
            <Reference ReferenceType="HasTypeDefinition">i=63</Reference>
            <Reference ReferenceType="HasModellingRule">i=78</Reference>
        </References>
    </UAVariable>

The UAVariable NodeId="ns=1;i=13" is not referenced at all. It is not derived from the inheritance chain. Therefore it is recognized as orphaned and not considered for further processing. According to the spec Table F.6 – UANodeSet Instance Nodes ParentNodeId="ns=1;i=12"> cannot be used to expose a member in the Address Space. The Node is redundant, so we cannot say that the UANodeSet document is not valid. It is like a not referenced variable in software development.

Any comments are welcome. I will wait for your opinion.

@eoursel
Copy link
Author

eoursel commented Feb 17, 2021

Hi @mpostol but there is the HasProperty reference to link i=13 to i=12 with the IsForward = false. As a result the i=13 node is defined as a property of the object type i=12.

And so this dammed variable is not orphaned but attached by a reverse reference.

Because it is not forbidden by the schema to make reverse references some nodeset authors use this construct which is completely brain damaged I must admit.

@mpostol
Copy link
Owner

mpostol commented Feb 17, 2021

@eoursel I see. You are right and it makes sense to add randomly new components (members). I was sure that IsForward is restricted for HasProperty and must be true. It looks like a bug in my references resolving mechanism - it is a really challenging algorithm. I will check it and let you know after tomorrow. Therefore I like to say ModelDesign recovery instead of ModelDesign export - it is reverse engineering, as the UANodeSet documents are always generated by a compiler.

@mpostol
Copy link
Owner

mpostol commented Feb 19, 2021

@eoursel working on the section Address Space Concept Executive Summary - in a document where I try to work out a conclusion of our findings - I try to figure out a scenario (sequence of services defined in Part 4)) for the OPC UA Client that it shall apply to get access to the NodeId="ns=1;i=13".

Can you help me in this respect?

@eoursel
Copy link
Author

eoursel commented Feb 19, 2021

@mpostol I can't get your point. For sure if you browse the Node VehicleType you should find a reference HasProperty to the node BuildDate. I just checked with UaExpert or any other client.

I think that the way the NodeSet import is working in a server is that the two options are semantically identical

  1. Node A -> HasReference Is Forward True -> Node B
  2. Node B -> HasReference is Forward False -> Node A

The server will create in memory
Node A -> HasReference -> Node B

This is true for any any kind of reference. Not only HasProperty.

@mpostol
Copy link
Owner

mpostol commented Feb 19, 2021

@eoursel I am not saying that something is wrong but I ony try to recover a services call sequence a client shall apply to browse to NodeId="ns=1;i=13". I 100% agree that the two options are semantically identical.

As you know I am not checking the results against any selected product, but rather against the specification. Of course, any results from the existing products are more than welcome.

In the context of references, I am using three terms: SourceNode, TargetNode, and ParentNode. In the specification Part 3 4.3.4 References we can read:

The Node that contains the Reference is referred to as the `SourceNode` and the Node that is referenced is referred to as the `TargetNode`.

Of course, it is not the only place defining the reference term but it doesn't mention something about direction. From this text, we can learn that the reference shall be added only to one Node - I call it ParentNode. Additionally, it seems to me that the meaning of the ParentNode is different for the model (design-time) and OPC UA Client API exposed by a server. In the method GetMyReferences (see attachment) I get used ParentNode. After changing it to SourceNode I am getting a different result. My concern is if I should generate a warning in this respect - as you said "which is completely brain damaged".

ParentVsSource

mpostol added a commit that referenced this issue Feb 19, 2021
- fixed errors and improved diagnostic information
mpostol added a commit that referenced this issue Feb 20, 2021
- cleanup code - removed useless comments - unimportant changes
- UT 👎
mpostol added a commit that referenced this issue Feb 21, 2021
- updated NodeSet files using current ModelCompiler implementation
- Removed graph HasEncoding references - added warning, because the round trip test cannot be resolved (Key extension is same for different nodes)
- improved unit tests
- UT :21:
mpostol added a commit that referenced this issue Mar 4, 2021
- added SemanticData/BuildingErrorsHandling/IIdentifierSyntax.md to formaly define syntax of the build errors.
- no code changes
@mpostol
Copy link
Owner

mpostol commented Mar 4, 2021

@eoursel, The BuildError.Identifier provides details allowing to select of the appropriate section in the specification that has been used to derive this validation rule. The BuildError entries are preliminary and subject to further modifications. In the last commit, I have added a formal description of the Identifier. In the document, you can find an example related to this particular case.

It is hard to say if a reported problem is a critical one or just a warning because the specification is inconsistent, incomplete and work is in progress. At the very beginning, as a corporate member, I had submitted a preliminary list of IM validation problems as a starting point for further work on IM compliance to promote the reusability of the models at design-time. Unfortunately, there was no follow-up, so the work was put on hold. The list is prepared in XML to guarantee portability between different environments.

@mpostol
Copy link
Owner

mpostol commented Mar 17, 2021

Hi @eoursel, I hope you are still following me. To move forward the discussion related to the errors severity level I have abstracted a bunch of snippets referring to the BrowseName from the specification.

I am trying to find a rule that can be applied to the model BrowseNameInheritedFrom0.xml to distinguish that EngineeringUnits is the BrowseName defined by OPCUA but tickness is defined by a custom model, but not OPCUA.
Any comments are welcome.

@mpostol mpostol added this to the SemanticData_6.1.3 milestone Mar 17, 2021
mpostol added a commit that referenced this issue Mar 18, 2021
- used Object.ReferenceEquals to compare references
- refactoring definitions - unimportant changes
mpostol added a commit that referenced this issue Mar 18, 2021
Import simple NodeSet2 file is incomplete #510
mpostol added a commit that referenced this issue Mar 18, 2021
- tested possibility to convert NodeSet to ModelDesign
mpostol added a commit that referenced this issue Mar 18, 2021
Import simple NodeSet2 file is incomplete #510
mpostol added a commit that referenced this issue Mar 18, 2021
- tested possibility to open in the ASMD 👍
mpostol added a commit that referenced this issue Mar 18, 2021
Import simple NodeSet2 file is incomplete #510
mpostol added a commit that referenced this issue Mar 18, 2021
- possibility to compile ModelDesign and generate NodeSet file (round trip test) 👎

Error generated by the ModelCompiler

At Build: 851392402 Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information;
At Build: 851392402 Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information; !ERROR!
The following has occurred:
The BaseType reference for node TrailerType is not valid: VehicleType.
   at Opc.Ua.ModelCompiler.ModelCompilerValidator.FindNode(XmlQualifiedName symbolicId, Type requiredType, String sourceName, String referenceName)
   at Opc.Ua.ModelCompiler.ModelCompilerValidator.ImportType(TypeDesign type)
   at Opc.Ua.ModelCompiler.ModelCompilerValidator.Import(NodeDesign node, NodeDesign parent)
   at Opc.Ua.ModelCompiler.ModelCompilerValidator.Validate2(IList`1 designFilePaths, String identifierFilePath, Boolean generateIds)
   at Opc.Ua.ModelCompiler.ModelGenerator2.ValidateAndUpdateIds(IList`1 d
@mpostol
Copy link
Owner

mpostol commented Mar 18, 2021

The BaseType reference for node TrailerType is not valid: VehicleType.

Compilation fails if the base type (VehicleType) is located in the ModelDesign file later than the derived type (TrailerType). Because the ModelDesign is not standard this behavior could not be recognized as an error.

@mpostol
Copy link
Owner

mpostol commented Mar 18, 2021

@eoursel it looks like your model has circular references created using the HasChild. Can you confirm?

From the spec:

P03-0705 HasChild ReferenceType - the HasChild ReferenceType is an abstract ReferenceType; only subtypes of it can be used. It is a subtype of HierarchicalReferences.
The semantic is to indicate that References of this type span a non-looping hierarchy.
Starting from Node “A” and only following References of the subtypes of the HasChild ReferenceType it shall never be possible to return to “A”. But it is allowed that following the References there may be more than one path leading to another Node “B”.

My point is that it should be recognized as a critical error.
Waiting for your opinion.

HasOrderedComponentCircularReference

mpostol added a commit that referenced this issue Mar 18, 2021
- possibility to compile ModelDesign and generate NodeSet file (round trip test) 👍

After swapping the definition of the types (the base type must be located before the derived types) the `ModelCompiler` works.

ASMD `Browse View` works in any case.
@eoursel
Copy link
Author

eoursel commented Mar 19, 2021 via email

@mpostol
Copy link
Owner

mpostol commented Mar 19, 2021

@eoursel thanks for the feedback. Of course, my concern is not the model itself. I know that the model is just an example without any production value. My concern is how to prove that the methodology is correct. Recovering ModelDesign from UANodeSet is a complex process, therefore first I try to prove that the result is as expected.

From your comment, I can derive that the recovered Opc.Ua.NodeSet2.TriCycleType_V1.1.ModelDesign.xml
model is wrong. Therefore I must fix it first to avoid problems like that with production models. Of course, the models you mentioned are on my roadmap.

Refereing to the C# code genrator. My point is that it is much easier to implement UANodeSet => C# generator than UANodeSet =>ModelDesign=>C# multi-stage, multitool process. In all of the cases, we are discussing the domain-specific language (DSL). Following this idea consider starting from C#, and in this case, we need C#=>ModelDesign converter. The question is which one DSL should be the source. There is no universal answer for sure. The answer must be a result of research but not just a decision.

Anyway, I am open to any suggestions. This work is community-driven so your opinion matters.

mpostol added a commit that referenced this issue Apr 4, 2021
- after fixing the #558 generated the following mode. As prviously compiler cannot compile it because of the sequence of type declarations
mpostol added a commit that referenced this issue Apr 4, 2021
- final model recovered from the origin with changed sequence of nodes definition against the compiler
mpostol added a commit that referenced this issue Apr 4, 2021
- Project has been built successfully
@mpostol mpostol added Feature request Consider to apply and removed Bug Bug in the code. labels Apr 10, 2021
mpostol added a commit that referenced this issue Apr 10, 2021
- recovered ModelDesign Opc.Ua.NodeSet2.TriCycleType_V1.1.ModelDesign.xml
mpostol added a commit that referenced this issue Apr 10, 2021
Import simple NodeSet2 file is incomplete #510
@mpostol
Copy link
Owner

mpostol commented Apr 10, 2021

@eoursel let me inform you that I have just generated the recovered ModelDesign. This folder also contains other associated files. The diagnostic log is attached to this comment.

Hopefully, the content is as expected. Let me know how it works for you.

RecoveryTrace.log

@mpostol
Copy link
Owner

mpostol commented Apr 11, 2021

@eoursel it will be continued in mpostol/UA-Nodeset#4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature request Consider to apply
Projects
None yet
Development

No branches or pull requests

2 participants