Improve support for the Component.Evidence.Identity field in CycloneDX 1.6

[darioandre]

In CycloneDX 1.5, the Component.Evidence.Identity field was specified as a single Identity object. In CycloneDX 1.6 this has been deprecated in favor of an array of Identity objects.
The specifications can be compared here:
1.5: https://cyclonedx.org/docs/1.5/json/#components_items_evidence_identity
1.6: https://cyclonedx.org/docs/1.6/json/#components_items_evidence_identity

cyclonedx-go still defines Identity as *EvidenceIdentity so it fails to unmarshal SBOMs which have an array of identities in place of a single one; the error is: cannot unmarshal array into Go struct field Evidence.components.evidence.identity of type cyclonedx.EvidenceIdentity.

This currently happens with some SBOMs generated by cdxgen (https://github.com/CycloneDX/cdxgen) when using the --spec-version 1.6 argument. This is an SBOM affected by the issue. It was generated from a skeleton Poetry project, with just pytest added, using this command:

docker run --rm -v /tmp:/tmp -v $(pwd):/app:rw -t ghcr.io/cyclonedx/cdxgen -r /app -o /app/bom.json --spec-version 1.6`

[eruvanos]

We also run into that issue, trivy inherits this issue :/

[DmitriyLewen]

Hi everyone!
I'm working on this task.
Can you assign it to me?

I'll try to finish it today or next week.

[DmitriyLewen]

Created #204

[darioandre]

Just my thoughts as a user of the library, regarding @DmitriyLewen's approach:

• Existing code using the Identity field will need to be refactored
• If the Identity field is populated, marshalling will only create CycloneDX 1.6 compliant files, which, depending on the case, may not be desired (OK, I know this is explicitly mentioned in the readme)
• The logic for decoding and converting between versions is getting increasingly complex

@nscuro Perhaps it could make sense to consider introducing different BOM structures for different versions (or range of versions) of the specification, with the possibility of explicitly converting one to the other (most likely from a version to the next one).
Of course, this means some duplication in the structure definitions, but overall I think it would make the implementation and the use of the library clearer:

• No breaking changes: the user chooses the version of structure to use and has a well defined upgrade path if needed
• No workarounds in the JSON/XML decoders
• All logic to convert between versions would be in clearly defined, separate conversion functions (JSON/XML agnostic) with clear semantics
• Overall, it would be more straightforward for both maintainers and users to support newer CycloneDX versions in the future

One of our use cases (probably a typical one) is decoding an external JSON/XML file, which might be of any version, and use it via a specific version of the structure (which may or may not be the latest). To support this, there could be a helper function which decodes the file to the correct structure, matching the file version (e.g. extracted with gjson), and then converts it using the conversion functions, possibly multiple times, until the version wanted by the user is reached.

Note that a similar approach has been taken for SPDX files: https://github.com/spdx/tools-golang

[DmitriyLewen]

I agree with your thoughts @darioandre.
Your solution seems more correct.

But this problem exists now. We have a problem - users can't scan the CycloneDX file (aquasecurity/trivy#6902 (comment)), so we want to fix this problem as soon as possible.

[jkowalleck]

see also: #207

[esnible]

It surprises me that the schema authors are able to change the type of a field from object to array when going from 1.5 to 1.6. This is the kind of breaking change I would expect going from 1.x to 2.x. Thanks to whomever is working on this.

[jkowalleck]

It surprises me that the schema authors are able to change the type of a field from object to array when going from 1.5 to 1.6.

@esnible, Where did this happen? Could you point me to the lines in the schema?

[nscuro]

@darioandre Perhaps it could make sense to consider introducing different BOM structures for different versions (or range of versions) of the specification, with the possibility of explicitly converting one to the other (most likely from a version to the next one).
Of course, this means some duplication in the structure definitions, but overall I think it would make the implementation and the use of the library clearer:

• No breaking changes: the user chooses the version of structure to use and has a well defined upgrade path if needed
• No workarounds in the JSON/XML decoders
• All logic to convert between versions would be in clearly defined, separate conversion functions (JSON/XML agnostic) with clear semantics
• Overall, it would be more straightforward for both maintainers and users to support newer CycloneDX versions in the future

One of our use cases (probably a typical one) is decoding an external JSON/XML file, which might be of any version, and use it via a specific version of the structure (which may or may not be the latest). To support this, there could be a helper function which decodes the file to the correct structure, matching the file version (e.g. extracted with gjson), and then converts it using the conversion functions, possibly multiple times, until the version wanted by the user is reached.

This sounds reasonable, however I don't currently have the bandwidth to perform such a major refactoring. If anyone would like to give this a go (pun intended!), it would be much appreciated.

[esnible]

@jkowalleck I found the schema changes difficult to follow. Do these help?

• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.6.schema.json#L277
• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.6.xsd#L2471
  and
• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.5.schema.json#L575
• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.5.xsd#L575

@nscuro , would it be possible to change

cyclonedx-go/cyclonedx.go

Line 626 in 464d426

Identity *EvidenceIdentity `json:"identity,omitempty" xml:"identity,omitempty"`

to something like

	RawIdentity    json.RawMessage     `json:"identity,omitempty" xml:"identity,omitempty"`
	Identity    *EvidenceIdentity     `json:"-"`
	Identity1_6    []EvidenceIdentity     `json:"-"`

... possibly with a custom deserialize to populate Identity and Identity1_6? I am not using the Identity fields in my code but I can't deserialize some of the 1.6 SBOMs I'm seeing from cdxgen because of this.

[jkowalleck]

@jkowalleck I found the schema changes difficult to follow. Do these help?

• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.6.schema.json#L277
• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.6.xsd#L2471
  and
• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.5.schema.json#L575
• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.5.xsd#L575

@esnible ,
Are you sure you found the correct line numbers? they don't even describe the same JSON-path nor XPaths.

• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.6.schema.json#L277 is
  $.declarations.claims[].evidence
• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.5.schema.json#L575 is
• component.evidence

and

• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.6.xsd#L2471 is
  componentEvidenceType/identity
• https://github.com/CycloneDX/specification/blob/master/schema/bom-1.5.xsd#L575 is component/evidence

Maybe it helps if you'd compare the structures in a more human-readable form.
for example https://cyclonedx.org/docs/1.6/json/ VS https://cyclonedx.org/docs/1.5/json/

---

I am really interested in learning about any undocumented or unexpected schema changes, since I invested much effort in documenting any changes, while gate-keeping unexpected/breaking changes. See for example this extensive change log: https://github.com/CycloneDX/specification/releases/tag/1.6

Please don't hesitate to report unexpected or questionable schema changes here: https://github.com/CycloneDX/specification/issues