Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open methods - only stream up to length #310

Merged
merged 2 commits into from
May 4, 2024
Merged

Open methods - only stream up to length #310

merged 2 commits into from
May 4, 2024

Conversation

ZJONSSON
Copy link
Owner

The stream method under the Open method did not utilize the fact that we know the size of the individual compressed file and, therefore, can avoid having the stream unbounded, reducing I/O.

Each entry comprises the local file header followed by the compressed file content. The size of the local header is dynamic; the base variables are 30 bytes, but then we have a dynamically sized filename and extra field. For some reason, I have to add a small buffer on top (a few bytes) to get all test cases passing.

Local file header (source Wikipedia)
image

Thanks to @jpambrun for catching this. Closes #308

@ZJONSSON
Copy link
Owner Author

@jpambrun can you check this out confirm whether this solution works

@jpambrun
Copy link

Yes, this branch works in my current project and minimal example in #309.

This is not exaustive testing by any means.

@ZJONSSON ZJONSSON merged commit 637df97 into master May 4, 2024
7 checks passed
@ZJONSSON ZJONSSON deleted the open-length branch May 4, 2024 19:57
@ZJONSSON
Copy link
Owner Author

ZJONSSON commented May 4, 2024

published as + [email protected]

@jribbens
Copy link

jribbens commented May 7, 2024

This change severely breaks things - see #311

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consider bounding the stream request when streaming entries
3 participants