perf: OOM investigations of map-only pipelines#3558

Draft

jaychia wants to merge 13 commits intomainfrom jay/big-materialization-ooms

+271-23

Commits on Dec 17, 2024

perf: Ray runner steps with big heap memory inflations
Jay Chia
committed
Add args
Jay Chia
committed
ints
Jay Chia
committed
set runner ray
Jay Chia
committed
Use pa large binary array
Jay Chia
committed
Upload logs even if job submission fails
Jay Chia
committed
Add memray outputs to traces
Jay Chia
committed
Add a test harness for testing estimations against actual
Jay Chia
committed
Add a __main__ script
Jay Chia
committed
Add build_partition test fixture
Jay Chia
committed
Add batched UDF tests
Jay Chia
committed
Add arrow return test
Jay Chia
committed
Add more failing tests
Jay Chia
committed