Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use update instead of CreateInBatches in CompleteMultipartUpload #776

Merged
merged 11 commits into from
Nov 30, 2023

Conversation

ChrisSchinnerl
Copy link
Member

@ChrisSchinnerl ChrisSchinnerl commented Nov 30, 2023

This PR uses an update instead of create now when completing a multipart upload.
That's possible now since we recently added the object_index to the slices and we no longer rely on the ID for the insertion order.

This also gets rid of the separate PartialSlabs field in the object.Object since workers won't be able to deduct the right download order. Instead partial slabs are now part of the regular slabs but without any slices.

Closes #769

@ChrisSchinnerl ChrisSchinnerl self-assigned this Nov 30, 2023
@peterjan
Copy link
Member

should we mention #769 here?

@mike76-dev
Copy link
Contributor

mike76-dev commented Nov 30, 2023

Do you think we need to have the slab order included in the object metadata?
Currently, the partial slabs are listed after the complete slabs, with no sequential order. However, with the multipart uploads it happens very often that a partial slab stands between two complete slabs. When the metadata is imported (from a satellite, or using a curl command), there is no way to assemble the slabs correctly without knowing their order.

bus/bus.go Show resolved Hide resolved
stores/multipart_test.go Outdated Show resolved Hide resolved
object/slab.go Outdated Show resolved Hide resolved
stores/metadata_test.go Outdated Show resolved Hide resolved
@ChrisSchinnerl ChrisSchinnerl merged commit 5a77190 into master Nov 30, 2023
6 checks passed
@ChrisSchinnerl ChrisSchinnerl deleted the chris/multipart-upload branch November 30, 2023 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Multipart upload parts assembled out of order
3 participants