Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for xlsx export #25

Open
Dawnkai opened this issue Dec 17, 2023 · 0 comments
Open

Support for xlsx export #25

Dawnkai opened this issue Dec 17, 2023 · 0 comments
Labels
backend Tasks related to backend frontend Tasks related to frontend non-essential Tasks that are not required, but would be nice to have

Comments

@Dawnkai
Copy link
Owner

Dawnkai commented Dec 17, 2023

Currently the backend supports only .csv file export. That is because, as it turns out, it is not possible to split .xlsx files into chunks, which then can be sent to the user using generator and streams.

If such method of splitting exists, a librarly called XlsxWriter could be used to write into a file without eating up entire memory for very large files, as per documentation:
https://xlsxwriter.readthedocs.io/working_with_memory.html (especially the part about constant-memory mode)
https://flask.palletsprojects.com/en/2.3.x/patterns/streaming/

It is not something that is a must, as .csv files can be imported into Excel just fine, but it is worth considering for the future.
As for the detailed reason why this is impossible to achieve, I'll copy and paste ChatGPT's answer, because why not:

Unfortunately, streaming XLSX files in chunks directly from an Excel file isn't as straightforward as it is with plain text formats like CSV. The XLSX format is a ZIP archive of XML files, and the structure of an Excel file relies on having the correct relationships and dependencies, making it challenging to stream incrementally.

However, you can consider a workaround where you generate the XLSX file in chunks and then stream each chunk individually. This involves creating multiple XLSX files, each containing a portion of the data, and then sending them one by one. The client can then concatenate the received files.

Keep in mind that this is a workaround, and the client will need to combine the received chunks to reconstruct the complete Excel file. Additionally, you may need to adjust the delay and chunking logic based on your specific use case and requirements.

@Dawnkai Dawnkai added backend Tasks related to backend non-essential Tasks that are not required, but would be nice to have frontend Tasks related to frontend labels Dec 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Tasks related to backend frontend Tasks related to frontend non-essential Tasks that are not required, but would be nice to have
Projects
None yet
Development

No branches or pull requests

1 participant