Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vine: submit more when hungry #3991

Merged
merged 4 commits into from
Dec 3, 2024

Conversation

JinZhou5042
Copy link
Member

@JinZhou5042 JinZhou5042 commented Nov 25, 2024

Proposed Changes

Fix #3957

The problem is that the DaskVine manager will take a while to initialize tasks when the graph is wide enough, this pr lets the manager to consider worker connections right after submitting a small number of tasks.

Merge Checklist

The following items must be completed before PRs can be merged.
Check these off to verify you have completed all steps.

  • make test Run local tests prior to pushing.
  • make format Format source code to comply with lint policies. Note that some lint errors can only be resolved manually (e.g., Python)
  • make lint Run lint on source code prior to pushing.
  • Manual Update: Update the manual to reflect user-visible changes.
  • Type Labels: Select a github label for the type: bugfix, enhancement, etc.
  • Product Labels: Select a github label for the product: TaskVine, Makeflow, etc.
  • PR RTM: Mark your PR as ready to merge.

@JinZhou5042 JinZhou5042 self-assigned this Nov 25, 2024
@JinZhou5042 JinZhou5042 changed the title vine: submit more tasks when daskvine manager is hungry vine: submit more when not hungry Nov 26, 2024
@JinZhou5042 JinZhou5042 changed the title vine: submit more when not hungry vine: submit more when hungry Nov 26, 2024
@JinZhou5042 JinZhou5042 requested a review from btovar November 26, 2024 22:32
@btovar
Copy link
Member

btovar commented Dec 2, 2024

@JinZhou5042 One issue with this solution is that vine_hungry() is potentially an expensive operation. #3990 makes it a little more efficient by considering only some of the waiting tasks, but it may still be slow if there are many workers connected. I got an idea on how to make this more efficient as the number of workers does not change much every time we call vine_wait, but I have not added it to #3990.

vine_hungry is supposed to return the number of tasks that would make the managernot hungry. So rather than calling vine_hungry every time, I suggest you call it once like min_to_sate = manager.hungry(), and decrease min_to_sate until it gets to less than 1.

@btovar btovar mentioned this pull request Dec 2, 2024
7 tasks
@btovar btovar merged commit 469f1da into cooperative-computing-lab:master Dec 3, 2024
9 of 10 checks passed
@JinZhou5042 JinZhou5042 deleted the hungry_submit branch December 10, 2024 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

vine: long initialization time for large task graphs in DaskVine
2 participants