Using S3 Express One Zone from the Rust SDK

[ysaito1001]

This guide describes making use of S3 Express One Zone from the Rust SDK. For more details on S3 Express One Zone, see this documentation.

Interacting with S3 Express One Zone buckets

You can use the Rust SDK to interact with S3 Express One Zone buckets ("S3 Express bucket" for short for the remaining of this guide) much the same way you do with regular S3 buckets.

To start, you can create an S3 Express bucket as follows:

use aws_config::BehaviorVersion;
use aws_sdk_s3::types::{
    BucketInfo, BucketType, CreateBucketConfiguration, DataRedundancy, LocationInfo, LocationType,
};
use aws_sdk_s3::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let sdk_config = aws_config::load_defaults(BehaviorVersion::latest()).await;
    let client = Client::new(&sdk_config);

    let bucket_cfg = CreateBucketConfiguration::builder()
        .location(
            LocationInfo::builder()
                .name("usw2-az1")
                .r#type(LocationType::AvailabilityZone)
                .build(),
        )   
        .bucket(
            BucketInfo::builder()
                .data_redundancy(DataRedundancy::SingleAvailabilityZone)
                .r#type(BucketType::Directory)
                .build(),
        )   
        .build();

    // This name is just for an example purpose, use your own express bucket name
    let express_bucket = "s3express-example-bucket--usw2-az1--x-s3";

    let _ = client
        .create_bucket()
        .create_bucket_configuration(bucket_cfg)
        .bucket(express_bucket)
        .send()
        .await
        .expect("should create a directory bucket");
       
    // --snip--
    
    Ok(())
}

Assuming your default region is us-west-2, the code above will create an S3 Express bucket with the name s3express-example-bucket--usw2-az1--x-s3 as a directory bucket.

You can then call S3 APIs with S3 Express One Zone just as you do with regular S3 buckets. Make sure to specify the name of an S3 Express bucket. For instance:

// The same `use` statements in the snippet above

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Continuing from the `create_bucket` in the snippet above

    let key = "object-key";

    // Put an object
    client
        .put_object()
        .bucket(express_bucket)
        .key(key)
        .body(aws_sdk_s3::primitives::ByteStream::from_static(b"object"))
        .send()
        .await
        .expect("object should be placed in the directory bucket");

    // Get the object
    let output = client
        .get_object()
        .bucket(express_bucket)
        .key(key)
        .send()
        .await
        .expect("object should be retrieved from the directory bucket");
        
    // --snip--
    
    Ok(())
}

Overriding a credentials provider for S3 Express One Zone

S3 Express One Zone internally uses a default-implemented credentials provider. You can override it with your own custom credentials provider for S3 Express One Zone, calling .express_credentials_provider on an S3 client. The method can be called either at a client level or at an operation level.

Overriding an S3 Express credentials provider at a client level:

    let conf = aws_sdk_s3::Config::builder()
        .express_credentials_provider(/* pass your own express credentials provider */)
        .build();
    let client = Client::from_conf(conf);
    // --snip--

Overriding an S3 Express credentials provider at an operation level:

    let sdk_config = aws_config::load_defaults(aws_config::BehaviorVersion::latest()).await;
    let client = aws_sdk_s3::Client::new(&sdk_config);
    
    let _output = client
        .list_objects_v2()
        .customize()
        .config_override(aws_sdk_s3::Config::builder()
            .express_credentials_provider(/* pass your own express credentials provider */))
        .send()
        .await
        .expect("objects should be listed");

    // --snip--

Keep in mind that when overriding an S3 Express credentials provider, the default caching implementation for S3 Express credentials is no longer available. Thus, you need to provide you own caching for S3 Express credentials internal to a custom S3 Express credentials provider.

Disabling S3 Express One Zone

S3 Express One Zone has its own auth session flow that uses a modified SigV4 and it is enabled by default. You can disable S3 Express One Zone with one of the following ways:

highest precedence

• calling s3_disable_express_session_auth on an S3 client config (either at a client level or at an operation level)
• setting an environment variable AWS_S3_DISABLE_EXPRESS_SESSION_AUTH (accepting case-insensitive "true" or "false", and ignoring any other values)
• (not supported yet) specifying s3_disable_express_session_auth in the config file at ~/.aws/config

lowest precedence

If one of the places is set to true/false, subsequent places with lower precedence will not be considered.

Something to be aware of when setting the disable option through the environment variable. The environment variable is only checked during a client construction, meaning that if a customer sets it after the client has been created the SDK will not take the environment variable value into account, i.e. the following snippet will NOT disable the S3 Express session auth:

    let sdk_config = aws_config::load_defaults(aws_config::BehaviorVersion::latest()).await;
    let client = aws_sdk_s3::Client::new(&sdk_config);

    // Set the env variable to true after `client` has been created
    std::env::set_var("AWS_S3_DISABLE_EXPRESS_SESSION_AUTH", "true");

    let _ = client
        .list_objects_v2()
        .bucket(/* S3 express bucket name */)
        .send()
        .await;