Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAT] Estimate materialized size of ScanTask better from Parquet reads #3302

Closed
wants to merge 10 commits into from

Rename for clarity

b6a7b7f
Select commit
Loading
Failed to load commit list.
Closed

[FEAT] Estimate materialized size of ScanTask better from Parquet reads #3302

Rename for clarity
b6a7b7f
Select commit
Loading
Failed to load commit list.
Codecov / codecov/project succeeded Nov 22, 2024 in 1s

77.44% (+0.05%) compared to b6695eb

View this Pull Request on Codecov

77.44% (+0.05%) compared to b6695eb

Details

Codecov Report

Attention: Patch coverage is 96.47887% with 15 lines in your changes missing coverage. Please review.

Project coverage is 77.44%. Comparing base (b6695eb) to head (b6a7b7f).
Report is 12 commits behind head on main.

Files with missing lines Patch % Lines
src/daft-scan/src/glob.rs 91.02% 7 Missing ⚠️
src/daft-scan/src/size_estimations.rs 98.12% 6 Missing ⚠️
src/daft-scan/src/anonymous.rs 0.00% 1 Missing ⚠️
src/daft-scan/src/python.rs 66.66% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3302      +/-   ##
==========================================
+ Coverage   77.39%   77.44%   +0.05%     
==========================================
  Files         678      686       +8     
  Lines       83300    84006     +706     
==========================================
+ Hits        64469    65060     +591     
- Misses      18831    18946     +115     
Files with missing lines Coverage Δ
src/daft-micropartition/src/micropartition.rs 90.85% <100.00%> (+0.03%) ⬆️
src/daft-micropartition/src/ops/cast_to_schema.rs 100.00% <100.00%> (ø)
src/daft-scan/src/lib.rs 61.14% <100.00%> (+0.87%) ⬆️
src/daft-scan/src/scan_task_iters.rs 97.01% <100.00%> (+0.06%) ⬆️
src/daft-scan/src/anonymous.rs 0.00% <0.00%> (ø)
src/daft-scan/src/python.rs 76.64% <66.66%> (-0.09%) ⬇️
src/daft-scan/src/size_estimations.rs 98.12% <98.12%> (ø)
src/daft-scan/src/glob.rs 90.78% <91.02%> (+0.50%) ⬆️

... and 25 files with indirect coverage changes

---- 🚨 Try these New Features: