-
Notifications
You must be signed in to change notification settings - Fork 575
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: enable http timeout #1777
fix: enable http timeout #1777
Conversation
Otherwise grype db commands can hang if the CDN is having issues. Signed-off-by: Will Murphy <[email protected]>
From an offline discussion, we probably want have this configurable
I think there's some legitimate concern that 30 seconds is not always long enough to download the 150 mb grype-db, and the timeouts should definitely be configurable. I'll push an update to:
The particular complaint in #1731 is that the download starts promptly but is very slow. This symptom means that we can't use a dial timeout or a response headers timeout, since the delay is in actually sending the bytes of the response body. However, it might be worth also trying to tune these finer grained timeouts a bit, with the goal that if the network or the database or listing file downloads are just unreachable, grype would block for as little time as possible. |
Signed-off-by: Will Murphy <[email protected]>
Signed-off-by: Will Murphy <[email protected]>
Signed-off-by: Will Murphy <[email protected]>
return | ||
case err := <-errs: | ||
t.Error(err) | ||
case <-timeout: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we're doing a test that combines the calls of IsUpdateAvailable
and UpdateTo
is there a chance this can race incorrectly?
I guess the gap is so large from the configured timeouts and failAfter
(more than 10x) that it doesn't matter.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@spiffcs I wanted to make sure I understand what you mean by "race incorrectly." I think you're saying that since (1) the test fails on a timeout and (2) the assertion expects a different timeout, we could get the wrong timeout. Is that correct?
I think that since we're effectively racing a 0.8 second timer against a 10 second timer, the test should be pretty stable. I put in the 10 second timer because otherwise removing one of the timeouts causes the test suite to hang for 10 minutes while the overall testing timer runs out, which seems like worse behavior.
Please let me know if I've addressed these concerns adequately and then I'll merge.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep! That's exactly what I considered (0.8 vs 10). Test looks good and I don't think it will give us trouble in the future. LGTM
Otherwise grype db commands can hang if the CDN is having issues.
Related to #1731
Some notes
We don't want commands like
grype db update
to hang indefinitely, and we certain don't wantgrype <some-source>
to hang indefinitely.Grype is already intended to continue on error if db updates are not available: https://github.com/anchore/grype/blob/fix%2Foverall-http-timeout/grype/db/curator.go#L142, so after this change, a timeout error when checking for update or downloading the database will result in a logged warning.
30 seconds overall timeout seems be reasonable. The compressed grype db as of this writing is ~152MB .tar.gz file when downloaded. I've done a few
time curl -o /dev/null "${DB_URL}"
for grype dbs published on different days, and the highest I've seen is about 8 seconds.Questions
Manual testing done:
With this change: