Skip to content

Commit

Permalink
miniogw: make ListObjects/V2 aware of easier cases
Browse files Browse the repository at this point in the history
This change changes listObjectsExhaustive (that ListObjects /
ListObjectsV2 is using) to not list too many items if it doesn't have
to. Specifically, if the prefix contains forward slashes, it will always
try to limit the scope of its listing by listing from the last forward
slash in the prefix.

Updates storj/customer-issues#184

Change-Id: Iae3fa0bc509cf089404a348bdaa9bae1c127966e
  • Loading branch information
amwolff committed Aug 5, 2022
1 parent 72a306c commit 1d11e6a
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions miniogw/gateway.go
Original file line number Diff line number Diff line change
Expand Up @@ -462,13 +462,17 @@ func (layer *gatewayLayer) listObjectsExhaustive(
// have to comply with (*uplink.Project).ListObjects' API.
var listPrefix string

if strings.HasSuffix(prefix, "/") {
listPrefix = prefix
// There is a good chance we don't need to list the entire bucket if the
// prefix contains forward slashes! If it does, let's list from the last
// one. If the satellite doesn't give us anything after that chopped-off
// prefix, we won't return anything anyway.
if i := strings.LastIndex(prefix, "/"); i != -1 {
listPrefix = prefix[:i] + "/"
}

list := project.ListObjects(ctx, bucket, &uplink.ListObjectsOptions{
Prefix: listPrefix,
Recursive: delimiter != "/" || (strings.Contains(prefix, "/") && !strings.HasSuffix(prefix, "/")),
Recursive: delimiter != "/",
System: true,
Custom: layer.compatibilityConfig.IncludeCustomMetadataListing,
})
Expand Down

0 comments on commit 1d11e6a

Please sign in to comment.