Mutipart upload high level vs low level apis questions

[pegiribidi]

Hi,

I'm trying out aws-sdk v2 apis so as to make the aws-sdk library upgrade successful in our project.

For aws-sdk v2 multi part upload I'm referring to the below java example which is using the high level api.

https://github.com/awsdocs/aws-doc-sdk-examples/blob/eb1f1c5b8d2cb15e4d78a4e8196584395d3b8818/javav2/example_code/s3/src/main/java/com/example/s3/transfermanager/UploadFile.java#L59

Is this the recommended way to do it or we have to use the low level apis ?
I'm giving the below code snippet that I used in my example with the low level apis.

CreateMultipartUploadRequest createMultipartUploadRequest = CreateMultipartUploadRequest.builder()
.bucket(s3BucketName)
.key(key)
.build();

    CreateMultipartUploadResponse response = getS3Client().createMultipartUpload(createMultipartUploadRequest);
    String uploadId = response.uploadId();

    FileInputStream fis = new FileInputStream (file);

    List<CompletedPart> listCompletedPart = processFileIntoParts(fis, key, uploadId); // a private method to process the file in chunks to get multiple parts

    fis.close();

    CompletedMultipartUpload completedMultipartUpload = CompletedMultipartUpload.builder()
            .parts(sorted)
            .build();

    CompleteMultipartUploadRequest completeMultipartUploadRequest =
            CompleteMultipartUploadRequest.builder()
                    .bucket(s3BucketName)
                    .key(key)
                    .uploadId(uploadId)
                    .multipartUpload(completedMultipartUpload)
                    .build();

    getS3Client().completeMultipartUpload(completeMultipartUploadRequest);

======

In which scenario the low level apis are recommended over the high level apis ?

The other question that I have is about the use of targetThroughputInGbps() while building the s3AsyncClient.
I had noticed that the upload (using high level API) was failing for large file > ~980MB when I used targetThroughputInGbps(20.0). After changing the value to 5.0, I was able to upload these large files.
Please note that I've setup my project to upload file from local system to S3.

So what would that value be set to on a local machine for local testing vs on a EC2 instance ?

Please provide your input to my questions.

Thanks,
Priya

[debora-ito]

The high-level APIs will use the low-level APIs under the hood. Advantages of using TransferManager include the possibility to pause, resume and monitor the progress of a transfer, and operation abstractions - for example, if used together with the CRT-based S3 client, TransferManager upload, download and copy operations will automatically use multipart depending on the file size and the part size defined in the S3 client.

There's a lot more information in the TransferManager GA announcement blog post and in the Developer Guide.

The S3CrtAsyncClient javadoc provides some guidance around targetThroughputInGbps.