Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: cost estimator #123

Merged
merged 76 commits into from
Sep 23, 2024
Merged

refactor: cost estimator #123

merged 76 commits into from
Sep 23, 2024

Conversation

ratankaliani
Copy link
Member

Refactor the flow for how the cost estimator computes the execution reports.

  • L1 and L2 providers in the data fetcher were being dropped before the data was being fetched for long-running cost estimator tasks.
  • RAM usage would grow to 300-400GB for large cost estimator runs because the intermediate report data was not being dropped. Periodically add the data instead of collecting all of the execution report data at the end to avoid high memory usage.

@ratankaliani ratankaliani changed the base branch from custom-rollup-config to main September 20, 2024 21:03
@ratankaliani ratankaliani merged commit 1dcde31 into main Sep 23, 2024
6 checks passed
@ratankaliani ratankaliani deleted the ratan/refactor-cost-estimator branch September 24, 2024 23:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants