Upload Retry Object not created

[jtagvera]

I'm trying to upload a large (1.5 Gb) file using gcs_upload(), and am following your instructions in the library vignette as follows:

## write a big object to a file
big_file <- "big_filename.csv"
write.csv(big_object, file = big_file)

## attempt upload
upload_try <- gcs_upload(big_file)

I get an error as following, but upload_try is not saved as a retry object. In fact, upload_try is not saved as an object at all.

Therefore there's no retry object for me to enter into gcs_retry_upload(upload_try). Is there anything else I can do to create a retry object?

ℹ 2022-10-24 21:02:14 > File size detected as  1.3 Gb
ℹ 2022-10-24 21:02:14 > Found resumeable upload URL:  https://www.googleapis.com/upload/storage/v1/b/i.....
ℹ 2022-10-24 21:13:56 > File upload failed, trying to resume...
ℹ 2022-10-24 21:13:56 > Retry 3 of 3
Error in value[[3L]](cond) : 
  Must supply either retry_object or all of upload_url, file and type

FWIW, I can access the GCS bucket fine to read and write smaller objects. It's just with large files which require a resumable upload when this code breaks.

[MarkEdmondson1234]

What does str(upload_try) return? That object should be a class object you use in the second attempt.

[jtagvera]

Thanks so much for the quick response! The error message for upload_try <- gcs_upload(big_file) completes without saving upload_try as an object, so there's no such object in the environment. Hope that makes sense.

I also tried to manually complete gcs_retry_upload by providing the upload_url, file and type but I got this error instead:

Error in gcs_retry_upload(upload_url = "https://www.googleapis.com/upload/storage/v1/[url redacted]",  : 
  Couldn't get upload status

[MarkEdmondson1234]

Hmm ok that's odd, it should return something. It's always been hard to test this one without a way to trigger a failed upload.

[jtagvera]

Yeah it’s hard to create a reproducible example. Is there a way I can create a retry object without relying on the gcs_upload 3rd attempt failure?

[claytonperry]

Hi! Writing to +1 this issue, I've just run into the same problem when trying to upload a 5.7MB file (painfully close to the upload limit). I also noticed that adjusting the upload limit didn't change the behavior (I tried to adjust the upload limit up to 10MBs to no avail). Again, hard to reproduce, but I'm having the same issue where the upload_try object isn't even created when gcs_upload() fails. Also got the same Couldn't get upload status message when trying the Resumable Upload URL.

[MarkEdmondson1234]

The upload limit should get you around it, can I see some code? Perhaps the option is not being applied.