-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a Boto3Store class #106
Conversation
This still requires testing and a few todo list items need to be completed.
69dbe6d
to
81c1023
Compare
81c1023
to
f0ee14d
Compare
Converted this to a draft. It's been a while since I opened it but I'm actively working towards fixing the unit tests and adding the missing URL store support like the existing |
Alright, this is all done now and ready for review. It covers all of the same features in a backwards compatible way as the original |
Thanks for the merge! |
This implements #84 and adds a
Boto3Store
class using theboto3
library to use S3 as a storage backend.The code and tests were largely based on the existing
BotoStore
backend.I'm hoping these tests pass, but I'm getting some errors running them on my system and I can't quite tell if that's expected or not due to the authentication. If the tests need to be updated just let me know and I'd be happy to take care of that.I left theUrlMixin
out because in addition to a bucket, you'd need aboto3.client
instance configured for S3. I couldn't find an obvious way to create this from the bucket instance, so the user would need to initialize that every time, or we could initialize it for them but that starts to make assumptions about how they handle authentication. As a third option we could raiseRuntimeError
ifurl_for
is called and an client wasn't passed to__init__
. I opted to leave it out entirely but if you like one of those options more, just let me know.Update: The
UrlMixin
support is all set and all of the unit tests should be passing. One thing I wanted to note regarding the URL support is that theurl_valid_time
parameter is only used if it's set and the object is not an existing object that is publicly accessible. This allows the user to setpublic=True
and then obtain URLs that will be valid indefinitely as opposed to constrained by the 1 week limit that S3 imposes.Like the original
BotoStore
class, thebucket
argument is expected to be aboto3
Bucket instance. However, if it's a string it will be used as the bucket name and a new Bucket instance will be created as long as it already exists.Thanks!