Skip to content

Commit

Permalink
Add docu on s3fsstore.
Browse files Browse the repository at this point in the history
Include STS example.
  • Loading branch information
thomasmarwitz committed Dec 20, 2024
1 parent eef82c8 commit 8c0140f
Show file tree
Hide file tree
Showing 2 changed files with 68 additions and 0 deletions.
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ Table of contents
boto
azure
gcstorage
s3fsstore
memory
db
git
Expand Down
67 changes: 67 additions & 0 deletions docs/s3fsstore.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@

S3 Store
========

The S3FSStore class in the minimalkv library provides a mechanism to interact with an S3-compatible (e.g. AWS or minio) storage system using the s3fs library.

The preferred initialization method is to use the ``get_store_from_url`` function, which will parse the URL and return an instance of the appropriate store class.

Possible options for the URL are (newline separated for readability):

::

s3://access_key:secret_key@endpoint/bucket
[?create_if_missing=true]
[&region_name=region-name]
[&force_bucket_suffix=true|false]
[&session_token=session-token]
[&is_sts_credentials=true|false]
[&sts_assume_role__RoleArn=role-arn]
[&sts_assume_role__RoleSessionName=session-name]
[&sts_assume_role__DurationSeconds=duration-in-seconds]
[&sts_assume_role__[KEY]=[VALUE]]
[&verify=true|false]

::

from minimalkv import get_store_from_url

store = get_store_from_url(
"s3://access_key_id:secret_access_key@endpoint/bucket-name"
)
store.put("example-key", b"example content")
print(store.get("example-key"))


STS Credentials
===============

::

from minimalkv import get_store_from_url

store = get_store_from_url(
"s3://access_key_id:secret_access_key@endpoint/bucket-name"
"?is_sts_credentials=true"
"&sts_assume_role__RoleArn=arn:aws:iam::123456789012:role/MyRole"
"&sts_assume_role__RoleSessionName=MySession"
"&sts_assume_role__DurationSeconds=900"
)

store.put("example-key", b"example content")
print(store.get("example-key"))


If you want to pass nested attributes or lists as parameters, you should import
the ``S3FSStore`` directly, i.e. ``from minimalkv.net.s3fsstore import S3FSStore``.


Local Testing
=============

::

AWS_PROFILE={$MINIMALKV_PROFILE} pixi run pytest [--log-cli-level=INFO]

Include AWS_PROFILE or provide AWS credentials in the environment variables, otherwise
integration tests for AWS S3 will be skipped.

0 comments on commit 8c0140f

Please sign in to comment.