Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add package dependency completness field #3402

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Conversation

wagoodman
Copy link
Contributor

@wagoodman wagoodman commented Oct 31, 2024

Description

Adds a field that describes the completeness of a package's direct dependencies:

  • complete: the package has all of its direct dependencies resolved and related to this package.
  • incomplete: the package does not have all of its direct dependencies resolved.
  • complete-with-indirect: a superset of complete -- indicates that the package has all of its direct dependencies resolved as well as some or all of indirect dependencies. What is notable about this is that direct and indirect dependencies are linked directly to this package and are not separable (you cannot distinguish between a direct and indirect dependency from the perspective of this package).
  • unknown: indicates when dependency resolution mechanism for this package is not well understood.

In addition to adding this new field, all catalogers were updated to raise up accurate values for this field:

Cataloger Completeness Comment
alpm-db-cataloger complete
apk-db-cataloger complete
binary-classifier-cataloger unknown classifiers are limited to identifying package identities, but have no information about dependencies (though there may be dependencies via shared libs and other mechanisms [static lib at compile time])
cargo-auditable-binary-cataloger incomplete
cocoapods-cataloger incomplete
conan-cataloger conanfile.txt: incomplete, conan.lock: complete
conan-info-cataloger complete
dart-pubspec-lock-cataloger incomplete
dotnet-deps-cataloger complete
dotnet-portable-executable-cataloger incomplete
dpkg-db-cataloger complete
elf-binary-package-cataloger unknown though we can look for shared libs, we cannot see static dependencies nor dynamic dependencies using dlopen. This means that, even in cases where the dep info is actually complete, we can't programmatically determine that.
elixir-mix-lock-cataloger incomplete
erlang-otp-application-cataloger incomplete
erlang-rebar-lock-cataloger incomplete
github-action-workflow-usage-cataloger complete the only known exception to this is shared workflows
github-actions-usage-cataloger incomplete
go-module-binary-cataloger complete-with-indirect / incomplete the main module gets the complete-with-indirect but all dependencies get incomplete
go-module-file-cataloger incomplete there is no main module discovered to make relationships for
graalvm-native-image-cataloger unknown or if anything is decoded from the internal SBOM, then that value is used
haskell-cataloger incomplete
java-archive-cataloger complete/incomplete all packages are assumed to be incomplete unless searching for transitive dependencies configuration option has been enabled
java-gradle-lockfile-cataloger incomplete
java-jvm-cataloger incomplete
java-pom-cataloger incomplete
javascript-lock-cataloger incomplete
javascript-package-cataloger incomplete we find all dependency nodes but are not crafting relationships
linux-kernel-cataloger incomplete
lua-rock-cataloger incomplete
nix-store-cataloger incomplete
opam-cataloger incomplete
php-composer-installed-cataloger incomplete
php-composer-lock-cataloger incomplete
php-pecl-serialized-cataloger incomplete
portage-cataloger incomplete
python-installed-package-cataloger incomplete
python-package-cataloger requirements.txt: incomplete, setup.py: incomplete, poetry.lock: complete, Pipfile.lock: incomplete
r-package-cataloger incomplete
rpm-archive-cataloger incomplete
rpm-db-cataloger complete
ruby-gemfile-cataloger incomplete
ruby-gemspec-cataloger incomplete
ruby-installed-gemspec-cataloger incomplete
rust-cargo-lock-cataloger complete
sbom-cataloger unknown this may need some discussion
swift-package-manager-cataloger incomplete
swipl-pack-cataloger incomplete
wordpress-plugins-cataloger incomplete

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist:

  • I have added unit tests that cover changed behavior
  • I have tested my code in common scenarios and confirmed there are no regressions
  • I have added comments to my code, particularly in hard-to-understand sections

@github-actions github-actions bot added the json-schema Changes the json schema label Oct 31, 2024
@wagoodman wagoodman force-pushed the note-dep-quality branch 2 times, most recently from bb1b51b to 141d19a Compare October 31, 2024 16:27
@wagoodman wagoodman marked this pull request as ready for review October 31, 2024 16:28
@wagoodman wagoodman requested a review from a team November 15, 2024 14:44
@willmurphyscode
Copy link
Contributor

@wagoodman I wanted to make sure I understand why the rust-cargo-lock-cataloger is labeled as "complete" not as "mixed".

It seems like as we make a set of dependency relationships more complete, it passes from complete, to mixed, and back to complete, which is surprising:

  1. It finds all the direct dependencies and nothing else -> "complete"
  2. It finds all the direct and some or all indirect dependencies, but you can't tell how direct the dependencies are -> "mixed"
  3. It finds all direct and indirect dependencies, and you can tell which ones are direct -> "complete"

It seems weird to me that a cataloger can effectively get promoted from complete to mixed, and then from mixed to complete, by adding relationship data.

When I first read the descriptions above, I thought, "rust isn't complete, it's mixed, because it includes transitive dependencies."

Am I understanding correctly here?

@wagoodman
Copy link
Contributor Author

wagoodman commented Nov 15, 2024

Here each node in the cargo.lock is describing only direct dependencies so this is complete

@willmurphyscode
Copy link
Contributor

Cross posting #3010 (comment) so that it gets seen here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
json-schema Changes the json schema
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Supply "depth" information when including relationships
2 participants