Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option for S3 direct downloads with presigned urls #2175

Open
silenium-dev opened this issue Jul 24, 2024 · 1 comment
Open

Add option for S3 direct downloads with presigned urls #2175

silenium-dev opened this issue Jul 24, 2024 · 1 comment
Labels
investigation Issue is currently investigeted, e.g. author is trying to reproduce problem triage

Comments

@silenium-dev
Copy link

silenium-dev commented Jul 24, 2024

Request details

S3 can be used with presigned urls, which would enable redirecting to the object storage instead of proxying the download, saving bandwidth on the Reposilite server.
This is especially useful when using a different server (Minio on different machine) or external service (S3, GCS) and the bandwidth for object storage is independent from Reposilite.

AWS docs: https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
This is usually supported by S3-compatible providers, e.g. GCS, R2 and Minio

Of course, this only works if the object storage is reachable from the maven client.

@dzikoysk dzikoysk added the investigation Issue is currently investigeted, e.g. author is trying to reproduce problem label Aug 18, 2024
@solonovamax
Copy link
Contributor

from the discord where this topic came up at one point:

also gradle/maven running on older java version does not follow 3xx statuses at all by default

but, maybe smth that could be done is to parse the user agent to identify the java version (afaik it should include that in the user agent)
if on a modern jdk, return a 302, otherwise fall back to streaming it

tho there's always the possibility of people using build tools that 1. dont send a proper user agent and 2. dont support redirects; so imo there should be a config option to always stream. (bonus, this would allow caching of artifacts locally rather than having to query S3 all the time)

that's why I said fallback to streaming it

if the useragent can't be parsed nicely, assume the worst-case scenario

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
investigation Issue is currently investigeted, e.g. author is trying to reproduce problem triage
Projects
None yet
Development

No branches or pull requests

3 participants