Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Fix writes for empty dataframes if target directory does not exist #3278

Merged
merged 3 commits into from
Nov 13, 2024

Conversation

colin-ho
Copy link
Contributor

Writes for empty dataframes currently error out if the target directory does not exist. This happens in the overwrite_files method when looking for files to delete, and also in write_empty_tabular when trying to write a file in a non-existent directory.

@github-actions github-actions bot added the bug Something isn't working label Nov 13, 2024
Copy link

codspeed-hq bot commented Nov 13, 2024

CodSpeed Performance Report

Merging #3278 will degrade performances by 15.73%

Comparing colin/fix-empty-overwrites (4a0f041) with main (5115c32)

Summary

⚡ 1 improvements
❌ 2 regressions
✅ 14 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main colin/fix-empty-overwrites Change
test_count[1 Small File] 3.4 ms 3.9 ms -12.43%
test_iter_rows_first_row[100 Small Files] 260.7 ms 309.3 ms -15.73%
test_show[100 Small Files] 42.4 ms 24 ms +76.64%

Copy link

codecov bot commented Nov 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.65%. Comparing base (1a4d259) to head (4a0f041).
Report is 13 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3278      +/-   ##
==========================================
+ Coverage   77.57%   77.65%   +0.08%     
==========================================
  Files         665      665              
  Lines       80900    80969      +69     
==========================================
+ Hits        62756    62880     +124     
+ Misses      18144    18089      -55     
Files with missing lines Coverage Δ
daft/filesystem.py 71.11% <100.00%> (+0.48%) ⬆️
daft/table/table_io.py 88.88% <100.00%> (+0.13%) ⬆️

... and 18 files with indirect coverage changes

@colin-ho colin-ho requested a review from kevinzwang November 13, 2024 15:34
Copy link
Member

@kevinzwang kevinzwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@colin-ho colin-ho merged commit c4e1ab2 into main Nov 13, 2024
43 of 44 checks passed
@colin-ho colin-ho deleted the colin/fix-empty-overwrites branch November 13, 2024 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants