Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support --severity-threshold for calculating exit code #5220

Closed
wants to merge 4 commits into from

Conversation

thisislawatts
Copy link
Member

@thisislawatts thisislawatts commented May 6, 2024

Pull Request Submission

Please check the boxes once done.

The pull request must:

  • Reviewer Documentation
    • follow CONTRIBUTING rules
    • be accompanied by a detailed description of the changes
    • contain a risk assessment of the change (Low | Medium | High) with regards to breaking existing functionality. A change e.g. of an underlying language plugin can completely break the functionality for that language, but appearing as only a version change in the dependencies.
    • highlight breaking API if applicable
    • contain a link to the automatic tests that cover the updated functionality.
    • contain testing instructions in case that the reviewer wants to manual verify as well, to add to the manual testing done by the author.
    • link to the link to the PR for the User-facing documentation
  • User facing Documentation
    • update any relevant documentation in gitbook by submitting a gitbook PR, and including the PR link here
    • ensure that the message of the final single commit is descriptive and prefixed with either feat: or fix: , others might be used in rare occasions as well, if there is no need to document the changes in the release notes. The changes or fixes should be described in detail in the commit message for the changelog & release notes.
  • Testing
    • Changes, removals and additions to functionality must be covered by acceptance / integration tests or smoke tests - either already existing ones, or new ones, created by the author of the PR.

Pull Request Review

All pull requests must undergo a thorough review process before being merged.
The review process of the code PR should include code review, testing, and any necessary feedback or revisions.
Pull request reviews of functionality developed in other teams only review the given documentation and test reports.

Manual testing will not be performed by the reviewing team, and is the responsibility of the author of the PR.

For Node projects: It’s important to make sure changes in package.json are also affecting package-lock.json correctly.

If a dependency is not necessary, don’t add it.

When adding a new package as a dependency, make sure that the change is absolutely necessary. We would like to refrain from adding new dependencies when possible.
Documentation PRs in gitbook are reviewed by Snyk's content team. They will also advise on the best phrasing and structuring if needed.

Pull Request Approval

Once a pull request has been reviewed and all necessary revisions have been made, it is approved for merging into
the main codebase. The merging of the code PR is performed by the code owners, the merging of the documentation PR
by our content writers.

What does this PR do?

Adds support --severity-threshold when determining the exit code. This means that you can use snyk code test --severity-threshold=high with the go based workflow and get the success exit code if only low severity issues are found.

How should this be manually tested?

Run the freshly built binary against a project that only contains low|medium severity issues, in this case I am running against https://github.com/snyk-fixtures/shallow-goof where .gitignore has been removed after installing so the scanner runs over node_modules.

~/snyk/cli/binary-releases/snyk-macos-arm64 code test ~/snyk/shallow-goof --org=<org-uuid-with-feature-flag> --severity-threshold=medium
echo $? // Should be 0

What are the relevant tickets?

CLI-304

The exit code derived from test summary should respect any
severity-threshold that has been configured by the client.

[CLI-317]
@thisislawatts thisislawatts force-pushed the feat/do-not-error-on-low-severity-issues branch 4 times, most recently from b530ae5 to 7fa2735 Compare May 6, 2024 12:07
@thisislawatts thisislawatts force-pushed the feat/do-not-error-on-low-severity-issues branch from 7fa2735 to 01f1cd5 Compare May 6, 2024 12:08
@thisislawatts thisislawatts changed the title feat: do not error on low severity issues feat: support --severity-threshold May 6, 2024
@thisislawatts thisislawatts changed the title feat: support --severity-threshold feat: support --severity-threshold for calculating exit code May 6, 2024
@thisislawatts thisislawatts marked this pull request as ready for review May 6, 2024 12:30
@thisislawatts thisislawatts requested a review from a team as a code owner May 6, 2024 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant