Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PERF] Used owned Stream in Parquet Page Iterator #1365

Merged
merged 5 commits into from
Sep 11, 2023
Merged

[PERF] Used owned Stream in Parquet Page Iterator #1365

merged 5 commits into from
Sep 11, 2023

Conversation

samster25
Copy link
Member

@samster25 samster25 commented Sep 11, 2023 •
edited
Loading

  • Improves wall clock time for reading parquet by 5-15%
  • Reduces memory usage by < 5%

@codecov
Copy link

codecov bot commented Sep 11, 2023 •
edited
Loading

Codecov Report

Merging #1365 (b301241) into main (c8ffd03) will not change coverage.
Report is 1 commits behind head on main.
The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1365   +/-   ##
=======================================
  Coverage   87.22%   87.22%           
=======================================
  Files          61       61           
  Lines        6004     6004           
=======================================
  Hits         5237     5237           
  Misses        767      767

see 1 file with indirect coverage changes

@samster25 samster25 marked this pull request as ready for review September 11, 2023 02:40
@samster25 samster25 changed the title [PERF] Owned page stream for arr iter [PERF] Used owned Stream in Parquet Page Iterator Sep 11, 2023
@samster25 samster25 requested a review from jaychia September 11, 2023 04:54
@samster25 samster25 merged commit 6ffff30 into main Sep 11, 2023
@samster25 samster25 deleted the sammy/owned-page-stream branch September 11, 2023 05:41
@jaychia jaychia mentioned this pull request Sep 12, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jaychia jaychia Awaiting requested review from jaychia

Assignees
No one assigned
Labels
performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@samster25