Hi @tomersein, thanks for the report! I was able to reproduce this issue. I appreciate the inclusion of a Dockerfile - it makes reproducing the issue a lot easier.

1. Make a Dockerfile exactly as above (thanks!)
2. Run docker build -t syft3088 .
3. Run syft -o json syft3088 > syft-sbom.json
4. Run cat syft-sbom.json | jq '.artifacts[] | select(.name=="numpy") | { name: .name, licenses: .licenses }'

You can see that the license set has a value that's the whole text of the license and the SPDX expression is blank.

Interestingly, on GitHub, the license is also shown as just "license" rather than having some automatic identification: https://github.com/numpy/numpy?tab=License-1-ov-file

my suggestion is to use split \n and take the 1st part of the value in cases like that.
what do you think?

👋 thanks for the license issue @tomersein!

It looks like syft is pulling this value from the following path in the container:
/usr/local/lib/python3.9/site-packages/numpy-1.26.4.dist-info/METADATA

This happens when the Python cataloger runs and we construct package details from the Egg/Wheel metadata:

Here is the field definition from we key off of from the python Metadata specification:
https://packaging.python.org/en/latest/specifications/core-metadata/#license

In this case the package distributor opted to put the full text of the license in this field.

This is a valid value for when we go and decode the map structure of this file here:

I appreciate the suggestion of using a split \n to truncate longer values in this case, but I am afraid that might be inadequate in the general sense.

If you look at the contents of this license block the Author's take a good amount of effort to address different distributions they consume as a part of NumPy:
First the general clause for redistribution of the software

        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
        "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
        LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
        A PARTICULAR PURPOSE ARE DISCLAIMED.

More information (not in their github license.txt) about what they distribute

        The NumPy repository and source distributions bundle several libraries that are
        compatibly licensed.  We list these here.

        Name: lapack-lite
        Files: numpy/linalg/lapack_lite/*
        License: BSD-3-Clause
          For details, see numpy/linalg/lapack_lite/LICENSE.txt

        Name: tempita
        Files: tools/npy_tempita/*
        License: MIT
          For details, see tools/npy_tempita/license.txt

        Name: dragon4
        Files: numpy/core/src/multiarray/dragon4.c
        License: MIT
          For license text, see numpy/core/src/multiarray/dragon4.c

        Name: libdivide
        Files: numpy/core/include/numpy/libdivide/*
        License: Zlib
          For license text, see numpy/core/include/numpy/libdivide/LICENSE.txt


        Note that the following files are vendored in the repository and sdist but not
        installed in built numpy packages:

        Name: Meson
        Files: vendored-meson/meson/*
        License: Apache 2.0
          For license text, see vendored-meson/meson/COPYING

        Name: spin
        Files: .spin/cmds.py
        License: BSD-3
          For license text, see .spin/LICENSE
        ----

There are also considerations for the following (there are more truncated for brevity):

        Name: OpenBLAS
        Files: numpy.libs/libopenblas*.so
        Description: bundled as a dynamically linked library
        Availability: https://github.com/OpenMathLib/OpenBLAS/
        License: BSD-3-Clause
          Copyright (c) 2011-2014, The OpenBLAS Project
          All rights reserved.

        Name: LAPACK
        Files: numpy.libs/libopenblas*.so
        Description: bundled in OpenBLAS
        Availability: https://github.com/OpenMathLib/OpenBLAS/
        License: BSD-3-Clause-Attribution

        Name: libquadmath
        Files: numpy.libs/libquadmath*.so
        Description: dynamically linked to files compiled with gcc
        Availability: https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=libquadmath
        License: LGPL-2.1-or-later

Cutting the license off at the new line here would remove this information from the SBOM.

I know long field values are ugly and bloat the document, but in this case the value of the license file is as accurate as we can make it without resorting to some specialized parsing for this unique case.

@anchore/tools should we NOT be including this when we discover it associated with the package?

Maybe we should add a new fullText field on the existing package License structure, so that we can separate when we do detect the different use cases here into a separate field. This can be leveraged directly in SPDX too, which is nice.

hi @wagoodman , what is the decision?
may I have a way to help with the solution?

Hey @tomersein!

We talked about this on our livestream the other day. We're moving forward with FullText field being added to the license struct. What do you think is the best way forward detecting this? Just doing a simple len(arbitraryNumber) to see if it's the full text?

https://www.youtube.com/@Anchore/streams

i suggest \n character. I don't think saving twice the long license value is a good practice.

@spiffcs , https://peps.python.org/pep-0639/ has been provisionally accepted which will hopefully make this metadata field more standardised for future python package releases at least

Regarding:

What do you think is the best way forward detecting this? Just doing a simple len(arbitraryNumber) to see if it's the full text?

My suggestion would be to do this:
a) fuzzy match the license to an SPDX ID - don't include "full text" if we're just given an SPDX ID
b) for all other cases: include the license text we used as a fullText field, as was suggested above, and additionally try to classify the license using the library and if it returns a license that we can match to SPDX ID include that ID as well as the original license text