When ctx is canceled, StreamPrediction will throw an error

[yan5xu]

If the ctx used by StreamPrediction is canceled, StreamPrediction will throw an error.
The expectation is that StreamPrediction stops processing without throwing an error.

[mattt]

Hi @yan5xu. Thanks for reporting this. It was our intention for context cancellation to return without an error. Can you share what error is being produced?

[yan5xu]

Hi @yan5xu. Thanks for reporting this. It was our intention for context cancellation to return without an error. Can you share what error is being produced?

lol, below is the error msg I collected in the log

failed to send request: Get "https://streaming-api.svc.us.c.replicate.net/v1/streams/fge5z76wnqszkj5rqi3cxr4w6bwirlju6vxfvqxeoaiqnrij7sga\": context canceled

[mattt]

@yan5xu Thank you for clarifying. If the error happens because the context is canceled for the original HTTP request for the stream, I think it's reasonable to bubble that up. And on reflection, I think it's less confusing if context errors are always returned, even when used to cancel a stream. Would you agree with that?

In the meantime, my recommendation would be to check for context cancellation errors and handle those differently:

if errors.Is(err, context.Canceled) {
    // ...
}