Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: container support hashing .jar files of size up to buffer.constants.MAX_LENGTH #5403

Merged
merged 2 commits into from
Aug 14, 2024

Conversation

adrobuta
Copy link
Contributor

@adrobuta adrobuta commented Aug 6, 2024

Pull Request Submission

Please check the boxes once done.

The pull request must:

  • Reviewer Documentation
    • follow CONTRIBUTING rules
    • be accompanied by a detailed description of the changes
    • contain a risk assessment of the change (Low | Medium | High) with regards to breaking existing functionality. A change e.g. of an underlying language plugin can completely break the functionality for that language, but appearing as only a version change in the dependencies.
    • highlight breaking API if applicable
    • contain a link to the automatic tests that cover the updated functionality.
    • contain testing instructions in case that the reviewer wants to manual verify as well, to add to the manual testing done by the author.
    • link to the link to the PR for the User-facing documentation
  • User facing Documentation
    • update any relevant documentation in gitbook by submitting a gitbook PR, and including the PR link here
    • ensure that the message of the final single commit is descriptive and prefixed with either feat: or fix: , others might be used in rare occasions as well, if there is no need to document the changes in the release notes. The changes or fixes should be described in detail in the commit message for the changelog & release notes.
  • Testing
    • Changes, removals and additions to functionality must be covered by acceptance / integration tests or smoke tests - either already existing ones, or new ones, created by the author of the PR.

Pull Request Review

All pull requests must undergo a thorough review process before being merged.
The review process of the code PR should include code review, testing, and any necessary feedback or revisions.
Pull request reviews of functionality developed in other teams only review the given documentation and test reports.

Manual testing will not be performed by the reviewing team, and is the responsibility of the author of the PR.

For Node projects: It’s important to make sure changes in package.json are also affecting package-lock.json correctly.

If a dependency is not necessary, don’t add it.

When adding a new package as a dependency, make sure that the change is absolutely necessary. We would like to refrain from adding new dependencies when possible.
Documentation PRs in gitbook are reviewed by Snyk's content team. They will also advise on the best phrasing and structuring if needed.

Pull Request Approval

Once a pull request has been reviewed and all necessary revisions have been made, it is approved for merging into
the main codebase. The merging of the code PR is performed by the code owners, the merging of the documentation PR
by our content writers.

What does this PR do?

The bumped version of snyk-docker-plugin fixes the exception thrown by the crypto module The RangeError: data is too long by reading and computing the sha1 of .jar files in chunks. This allows snyk container to scan embedded .jar files of up to buffer.constants.MAX_LENGTH. On 32-bit architectures, this value currently is 230 - 1 (about 1 GiB). On 64-bit architectures, this value currently is 232 (about 4 GiB).

Where should the reviewer start?

How should this be manually tested?

This fix can be validated by testing a container image that contains a .jar file > 2 GB.
Cli behavior without the fix:

SNYK_API=https://app.snyk.io/api/v1 snyk container test large-jar:latest --platform=linux/amd64          
Something unexpected went wrong: RangeError: data is too long
    at Hash._transform (node:internal/crypto/hash:86:17)
    at Hash.Transform._write (node:internal/streams/transform:154:8)
    at writeOrBuffer (node:internal/streams/writable:392:12)
    at _write (node:internal/streams/writable:333:10)
    at Hash.Writable.write (node:internal/streams/writable:337:10)
    at Readable.ondata (node:internal/streams/readable:766:22)
    at Readable.emit (node:events:537:28)
    at Readable.emit (node:domain:482:12)
    at Readable.read (node:internal/streams/readable:539:10)
    at flow (node:internal/streams/readable:1023:34)
    at resume_ (node:internal/streams/readable:1004:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'ERR_OUT_OF_RANGE'
}
Exit code: 2

Cli behaviour with this fix:

SNYK_API=https://app.snyk.io/api/v1 node index.js container test large-jar:latest --platform=linux/amd64 --print-deps
alexandradrobut@C20G5979YN cli % echo $?
130

Any background context you want to provide?

What are the relevant tickets?

https://snyksec.atlassian.net/browse/UNIFY-158

Screenshots

Additional questions

@adrobuta adrobuta requested a review from a team as a code owner August 6, 2024 17:30
@adrobuta adrobuta force-pushed the fix/container-hash-large-jars branch 2 times, most recently from fd3c2fb to 8708202 Compare August 12, 2024 08:53
@sandor-trombitas sandor-trombitas force-pushed the fix/container-hash-large-jars branch from 8708202 to 6fe6b91 Compare August 14, 2024 08:55
@adrobuta adrobuta force-pushed the fix/container-hash-large-jars branch from 6fe6b91 to a103bbb Compare August 14, 2024 11:41
@sandor-trombitas sandor-trombitas force-pushed the fix/container-hash-large-jars branch from a103bbb to 1bb0ebf Compare August 14, 2024 11:52
The bumped version of snyk-docker-plugin fixes the exception thrown by the crypto module
`The RangeError: data is too long` by reading and computing the sha1 of .jar files in chunks.
This allows snyk container to scan embedded jar files of up to buffer.constants.MAX_LENGTH.
On 32-bit architectures, this value currently is 230 - 1 (about 1 GiB).
On 64-bit architectures, this value currently is 232 (about 4 GiB).
@sandor-trombitas sandor-trombitas force-pushed the fix/container-hash-large-jars branch from 1bb0ebf to 6f82231 Compare August 14, 2024 12:42
Copy link
Contributor

Warnings
⚠️

"fix: container support hashing .jar files of size up to buffer.constants.MAX_LENGTH" is too long. Keep the first line of your commit message under 72 characters.

Generated by 🚫 dangerJS against 689e1fe

@sandor-trombitas sandor-trombitas merged commit 7f39795 into main Aug 14, 2024
11 checks passed
@sandor-trombitas sandor-trombitas deleted the fix/container-hash-large-jars branch August 14, 2024 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants