CycloneDX convertion into Syft improperly handles SPDX licenses #3172

NyanKiyoshi · 2024-08-29T13:42:04Z

When converting a CycloneDX JSON BOM into the Syft format, SPDX licenses are dropped.

The CycloneDX specifications define that either (one of) the following values should be set under components[].licenses:

CycloneDX format: [{"license": {"id": "...", "name": "..."}}, ...]
Or (exclusive or), SPDX format, thus only [{"expression": "..."}, ...]

But Syft expects to have the following value which is invalid in the specs: {"expression": "...", "license": {...}}.

This is caused by the lines 57 to 59 at:

syft/syft/format/internal/cyclonedxutil/helpers/licenses.go

Lines 50 to 73 in 11d77b4

    
           func decodeLicenses(c *cyclonedx.Component) []pkg.License { 
        
           	licenses := make([]pkg.License, 0) 
        
           	if c == nil || c.Licenses == nil { 
        
           		return licenses 
        
           	} 
        
           	for _, l := range *c.Licenses { 
        
           		if l.License == nil { 
        
           			continue 
        
           		} 
        
           		// these fields are mutually exclusive in the spec 
        
           		switch { 
        
           		case l.License.ID != "": 
        
           			licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.ID, l.License.URL)) 
        
           		case l.License.Name != "": 
        
           			licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.Name, l.License.URL)) 
        
           		case l.Expression != "": 
        
           			licenses = append(licenses, pkg.NewLicenseFromURLs(l.Expression, l.License.URL)) 
        
           		default: 
        
           		} 
        
           	} 
        
           	return licenses 
        
           }

What you expected to happen:

When an SPDX license is provided inside a CycloneDX component, Syft should not drop it.

Potential Solution

We could drop the if l.License == nil { continue } and change the case conditions to handle the nil case properly:

func decodeLicenses(c *cyclonedx.Component) []pkg.License {
	// [...]

	for _, l := range *c.Licenses {
		switch {
		case l.License != nil && l.License.ID != "":
			licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.ID, l.License.URL))
		case l.License != nil && l.License.Name != "":
			licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.Name, l.License.URL))
		case l.Expression != "":
			licenses = append(licenses, pkg.NewLicense(l.Expression))
		default:
		}
	}

	return licenses
}

Diffs:

--- a/licenses.go
+++ b/licenses.b.go
@@ -54,17 +54,13 @@ func decodeLicenses(c *cyclonedx.Component) []pkg.License {
        }
 
        for _, l := range *c.Licenses {
-               if l.License == nil {
-                       continue
-               }
-               // these fields are mutually exclusive in the spec
                switch {
-               case l.License.ID != "":
+               case l.License != nil && l.License.ID != "":
                        licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.ID, l.License.URL))
-               case l.License.Name != "":
+               case l.License != nil && l.License.Name != "":
                        licenses = append(licenses, pkg.NewLicenseFromURLs(l.License.Name, l.License.URL))
                case l.Expression != "":
-                       licenses = append(licenses, pkg.NewLicenseFromURLs(l.Expression, l.License.URL))
+                       licenses = append(licenses, pkg.NewLicense(l.Expression))
                default:
                }
        }

Steps to reproduce the issue:

Prerequisites

Create two files into a directory:

valid.json (what Syft doesn't handle properly) [download]

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.5",
  "serialNumber": "urn:uuid:bc59a2de-780c-40ae-9316-4ad1287f1bb2",
  "version": 1,
  "metadata": {
    "timestamp": "2024-08-29T12:36:56Z",
    "tools": {
      "components": [
        {
          "group": "@cyclonedx",
          "name": "cdxgen",
          "version": "10.9.5",
          "purl": "pkg:npm/%40cyclonedx/[email protected]",
          "type": "application",
          "bom-ref": "pkg:npm/@cyclonedx/[email protected]",
          "author": "OWASP Foundation",
          "publisher": "OWASP Foundation"
        }
      ]
    },
    "authors": [
      {
        "name": "OWASP Foundation"
      }
    ],
    "lifecycles": [
      {
        "phase": "build"
      }
    ],
    "component": {
      "group": "",
      "name": "app",
      "version": "latest",
      "type": "application",
      "bom-ref": "pkg:gem/app@latest",
      "purl": "pkg:gem/app@latest"
    },
    "properties": [
      {
        "name": "cdx:bom:componentTypes",
        "value": "pypi"
      }
    ]
  },
  "components": [
    {
      "author": "\"Jeffrey A. Clark\" <[email protected]>",
      "group": "",
      "name": "pillow",
      "version": "10.4.0",
      "description": "Python Imaging Library (Fork)",
      "hashes": [
        {
          "alg": "SHA-256",
          "content": "4d9667937cfa347525b319ae34375c37b9ee6b525440f3ef48542fcf66f2731e"
        }
      ],
      "licenses": [
        {
          "expression": "Historical Permission Notice and Disclaimer (HPND)"
        }
      ],
      "purl": "pkg:pypi/[email protected]",
      "type": "library",
      "bom-ref": "pkg:pypi/[email protected]",
      "evidence": {
        "identity": {
          "field": "purl",
          "confidence": 1,
          "methods": [
            {
              "technique": "instrumentation",
              "confidence": 1,
              "value": "/tmp/cdxgen-venv-St5Cu7"
            }
          ]
        }
      },
      "properties": [
        {
          "name": "SrcFile",
          "value": "/app/requirements.txt"
        }
      ]
    }
  ],
  "services": [],
  "dependencies": [
    {
      "ref": "pkg:pypi/app@latest",
      "dependsOn": []
    },
    {
      "ref": "pkg:pypi/[email protected]",
      "dependsOn": []
    }
  ]
}

invalid.json (what Syft expects) [download]

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.5",
  "serialNumber": "urn:uuid:bc59a2de-780c-40ae-9316-4ad1287f1bb2",
  "version": 1,
  "metadata": {
    "timestamp": "2024-08-29T12:36:56Z",
    "tools": {
      "components": [
        {
          "group": "@cyclonedx",
          "name": "cdxgen",
          "version": "10.9.5",
          "purl": "pkg:npm/%40cyclonedx/[email protected]",
          "type": "application",
          "bom-ref": "pkg:npm/@cyclonedx/[email protected]",
          "author": "OWASP Foundation",
          "publisher": "OWASP Foundation"
        }
      ]
    },
    "authors": [
      {
        "name": "OWASP Foundation"
      }
    ],
    "lifecycles": [
      {
        "phase": "build"
      }
    ],
    "component": {
      "group": "",
      "name": "app",
      "version": "latest",
      "type": "application",
      "bom-ref": "pkg:gem/app@latest",
      "purl": "pkg:gem/app@latest"
    },
    "properties": [
      {
        "name": "cdx:bom:componentTypes",
        "value": "pypi"
      }
    ]
  },
  "components": [
    {
      "author": "\"Jeffrey A. Clark\" <[email protected]>",
      "group": "",
      "name": "pillow",
      "version": "10.4.0",
      "description": "Python Imaging Library (Fork)",
      "hashes": [
        {
          "alg": "SHA-256",
          "content": "4d9667937cfa347525b319ae34375c37b9ee6b525440f3ef48542fcf66f2731e"
        }
      ],
      "licenses": [
        {
          "expression": "Historical Permission Notice and Disclaimer (HPND)",
          "license": {
            "id": "HPND",
            "url": "https://opensource.org/license/historical-php"
          }
        }
      ],
      "purl": "pkg:pypi/[email protected]",
      "type": "library",
      "bom-ref": "pkg:pypi/[email protected]",
      "evidence": {
        "identity": {
          "field": "purl",
          "confidence": 1,
          "methods": [
            {
              "technique": "instrumentation",
              "confidence": 1,
              "value": "/tmp/cdxgen-venv-St5Cu7"
            }
          ]
        }
      },
      "properties": [
        {
          "name": "SrcFile",
          "value": "/app/requirements.txt"
        }
      ]
    }
  ],
  "services": [],
  "dependencies": [
    {
      "ref": "pkg:pypi/app@latest",
      "dependsOn": []
    },
    {
      "ref": "pkg:pypi/[email protected]",
      "dependsOn": []
    }
  ]
}

Differences between valid.json and invalid.json:

# diff -u valid.json invalid.json 
--- valid.json  2024-08-29 14:46:59
+++ invalid.json        2024-08-29 14:45:56
@@ -59,7 +59,11 @@
       ],
       "licenses": [
         {
-          "expression": "Historical Permission Notice and Disclaimer (HPND)"
+          "expression": "Historical Permission Notice and Disclaimer (HPND)",
+          "license": {
+            "id": "HPND",
+            "url": "https://opensource.org/license/historical-php"
+          }
         }
       ],
       "purl": "pkg:pypi/[email protected]",

JSON schema validation showing that it is indeed invalid (using https://github.com/CycloneDX/sbom-utility/releases/tag/v0.16.0):

$ sbom-utility validate --input-file valid.json --quiet # OK
$ sbom-utility validate --input-file invalid.json --quiet # Not OK
1. {
        "type": "number_one_of",
        "field": "components.0.licenses",
        "context": "(root).components.0.licenses",
        "description": "Must validate one and only one schema (oneOf)",
        "value": [
            {
                "expression": "Historical Permission Notice and Disclaimer (HPND)",
                "license": {
                    "id": "HPND",
                    "url": "https://opensource.org/license/historical-php"
                }
            }
        ]
    }
2. {
        "type": "additional_property_not_allowed",
        "field": "components.0.licenses.0",
        "context": "(root).components.0.licenses.0",
        "description": "Additional property expression is not allowed",
        "value": "Historical Permission Notice and Disclaimer (HPND)"
    }

Steps

Convert CycloneDX JSON BOM into Syft JSON format:

docker run --rm -v "$(pwd)":/app/ -ti ghcr.io/anchore/syft:v1.11.1 convert /app/valid.json --quiet -o syft-json > syft-format.json

No licenses should be present in the Syft JSON format (the bug):

$ jq . syft-format.json
{
  "artifacts": [
    [...]
    {
      "id": "b2e94cea47d09252",
      "name": "pillow",
      "version": "10.4.0",
      "type": "python",
      "foundBy": "",
      "locations": null,
      "licenses": [],  /// <---- The issue
      "language": "python",
      "cpes": [],
      "purl": "pkg:pypi/[email protected]"
    }
  ],
  [...]
  "schema": {
    "version": "16.0.15",
    "url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-16.0.15.json"
  }
}

Convert the invalid CycloneDX JSON BOM into the Syft format:

docker run --rm -v "$(pwd)":/app/ -ti ghcr.io/anchore/syft:v1.11.1 convert /app/invalid.json --quiet -o syft-json > syft-format.json

The license should now be present despite being invalid:

$ jq . syft-format.json
{
  "artifacts": [
    [...]
    {
      "id": "6eb502d35e5c15e7",
      "name": "pillow",
      "version": "10.4.0",
      "type": "python",
      "foundBy": "",
      "locations": null,
      "licenses": [
        {
          "value": "HPND",  <---- It's now here
          "spdxExpression": "HPND",
          "type": "declared",
          "urls": [
            "https://opensource.org/license/historical-php"
          ],
          "locations": []
        }
      ],
      "language": "python",
      "cpes": [],
      "purl": "pkg:pypi/[email protected]"
    }
  ],
  "artifactRelationships": [],
  [...]
  "schema": {
    "version": "16.0.15",
    "url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-16.0.15.json"
  }
}

By mitigating the issue (see the 'Potential Solution' section), we get the expected result:

$ syft convert valid.json --quiet -o syft-json > syft-format.json
$ jq . syft-format.json
{
  "artifacts": [
    [...]
    {
      "id": "ceefc6aa54ba5e1c",
      "name": "pillow",
      "version": "10.4.0",
      "type": "python",
      "foundBy": "",
      "locations": null,
      "licenses": [
        {
          "value": "Historical Permission Notice and Disclaimer (HPND)", // <---- This is correct now
          "spdxExpression": "",
          "type": "declared",
          "urls": [],
          "locations": []
        }
      ],
      "language": "python",
      "cpes": [],
      "purl": "pkg:pypi/[email protected]"
    }
  ],
   [...]
  "schema": {
    "version": "16.0.15",
    "url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-16.0.15.json"
  }
}

Anything else we need to know?

The issue was found in https://github.com/anchore/grant, it causes the grant to be flag some dependencies as having no license (version: 4362dc2.

Unmitigated:

$ grant check /app/bom2.json -o table --show-packages
* /app/bom2.json
  * No License Violations Found for Rule default-deny-all
  * packages found with no licenses
    * app
    * pillow <--- Bug

Mitigated ('Potential Solution' section), it works properly:

$ grant check /app/bom2.json -o table --show-packages
* /app/bom2.json
  * license matches for rule: default-deny-all; matched with pattern *
    * Historical Permission Notice and Disclaimer (HPND)  <--- Works!
      * pillow
  * packages found with no licenses
    * app
check failed

Environment:

Output of syft version:

Application: syft
Version:    1.11.1
BuildDate:  2024-08-20T16:25:20Z
GitCommit:  95b4a88256bddebb91831250f28f602f8c36552a
GitDescription: v1.11.1
Platform:   linux/amd64
GoVersion:  go1.22.6
Compiler:   gc

OS: Mac OS 14.6.1 (Sonoma)

The text was updated successfully, but these errors were encountered:

kzantow · 2024-08-29T13:54:56Z

Thanks for the detailed report, @NyanKiyoshi ! I've added this to the ready issues, but we would definitely welcome a PR, too.

This fixes the issue reported at anchore#3172, where Syft would drop SPDX licenses due a logic error in the decoder. CycloneDX specifications require `components[].licenses[].license` to be nil when `components[].licenses[].expression` (SPDX) is non nil. Signed-off-by: Mikail Kocak <[email protected]>

NyanKiyoshi · 2024-08-29T15:10:28Z

Resolved via #3175

NyanKiyoshi added the bug Something isn't working label Aug 29, 2024

anchoretoolsops added this to OSS Aug 29, 2024

kzantow moved this to Ready in OSS Aug 29, 2024

kzantow added the good-first-issue Good for newcomers label Aug 29, 2024

NyanKiyoshi mentioned this issue Aug 29, 2024

Fix improper decoding of SPDX license expressions in the CycloneDX format #3175

Merged

NyanKiyoshi closed this as completed Aug 29, 2024

github-project-automation bot moved this from Ready to Done in OSS Aug 29, 2024

BrewTestBot mentioned this issue Sep 11, 2024

syft 1.12.2 Homebrew/homebrew-core#187161

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CycloneDX convertion into Syft improperly handles SPDX licenses #3172

CycloneDX convertion into Syft improperly handles SPDX licenses #3172

NyanKiyoshi commented Aug 29, 2024

kzantow commented Aug 29, 2024

NyanKiyoshi commented Aug 29, 2024

CycloneDX convertion into Syft improperly handles SPDX licenses #3172

CycloneDX convertion into Syft improperly handles SPDX licenses #3172

Comments

NyanKiyoshi commented Aug 29, 2024

What you expected to happen:

Potential Solution

Steps to reproduce the issue:

Prerequisites

Steps

Anything else we need to know?

Environment:

kzantow commented Aug 29, 2024

NyanKiyoshi commented Aug 29, 2024