Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support non-streamed downloads #14591

Conversation

Robertbaelde
Copy link

Description

Some hosting environments do not support streamed downloads (mainly environments running on lambda like Laravel Vapor or Bref). This makes the export functionality not work because the downloaders assume we can do a streamed download.

This PR adds a config to enable/disable streamed downloads. When disabled, the downloader will write the contents to one file and uploads it to the disk (so we don't have to re-do the generation on multiple downloads).

Visual changes

N/A

Functional changes

  • Code style has been fixed by running the composer cs command.
  • Changes have been tested to not break existing functionality.
  • Documentation is up-to-date.

@danharrin danharrin added the bug Something isn't working label Oct 22, 2024
@danharrin danharrin added this to the v3 milestone Oct 22, 2024
Copy link
Member

@danharrin danharrin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please can you remove the part that saves new files to the disk? I am conscious we are already writing a lot of this data to the disk and XLSX files are already generated ahead of time, so we can probably avoid that for now while still allowing non-streamed downloads.

Please be aware that you may hit the memory limit more often if you don't stream

I would also double check that Vapor definitely doesnt support streamed downloads, because that doesn't really sound right to me 🤔

@austincarpenter
Copy link
Contributor

To get things working on Vapor, you need to add the X-Vapor-Base64-Encode header in the response. (See https://docs.vapor.build/projects/development.html#binary-responses)

I only use the XLSX downloader so I've simply subclassed XlsxDownloader, replacing the __invoke() function to add the header:

public function __invoke(Export $export): StreamedResponse
{
    return tap(parent::__invoke($export), function (StreamedResponse $response) {
        $response->headers->set('X-Vapor-Base64-Encode', true);
    });
}
$this->app->bind(BaseXlsxDownloader::class, XlsxDownloader::class);

@danharrin
Copy link
Member

Is there a way we can detect Vapor within Filament and do this on the users behalf? Instead of disabling streaming?

@Robertbaelde
Copy link
Author

Robertbaelde commented Oct 23, 2024

Is there a way we can detect Vapor within Filament and do this on the users behalf? Instead of disabling streaming?

For Vapor: isset($_ENV['VAPOR_SSM_PATH']) || isset($_SERVER['VAPOR_SSM_PATH'])
For Bref: env('LAMBDA_TASK_ROOT') (detects Lambda runtime) or env('BREF_LOOP_MAX')

Will try to reshape this PR a bit with the given feedback. I'm afraid for Bref downloads larger than 6MB should come from a disk (s3), see: https://bref.sh/docs/use-cases/http/binary-requests-responses

Not sure if/how Vapor mitigates that though
Edit: Vapor also has the 6MB file limit

@danharrin
Copy link
Member

Hi, please reopen this if you decide to implement the requested changes. Thank you!

@danharrin danharrin closed this Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants