[FEAT] Add smart planning of ScanTasks starting with merging by filesizes

[jaychia]

Refactors/changes required on ScanTask itself:

1. Added a ScanTask::merge
2. Added a ScanTask::partition_spec()
3. Added some validation in ScanTask::new to assert that all the underlying sources have the same partition spec

I then added a new module daft_scan::scan_task_iterators which contains functions that perform transformations on a Box<dyn Iterator<item = DaftResult<ScanTaskRef>>>.

TODO:

• Make the file_size configurable (as an environment variable/context flag) so that our unit-tests still run correctly when we do multi-file tests for multi-partition dataframes (see: [FEAT] Add daft.context.set_config function for setting configurations globally #1700 )

[codecov]

Codecov Report

Merging #1692 (3834fa2) into main (2fbf885) will increase coverage by 0.16%.
Report is 6 commits behind head on main.
The diff coverage is n/a.

❗ Current head 3834fa2 differs from pull request most recent head 5285b12. Consider uploading reports for the commit 5285b12 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1692      +/-   ##
==========================================
+ Coverage   84.89%   85.05%   +0.16%     
==========================================
  Files          55       55              
  Lines        5349     5354       +5     
==========================================
+ Hits         4541     4554      +13     
+ Misses        808      800       -8     

see 7 files with indirect coverage changes