Set lower CPU usage for test_shared_process_pool.py
to avoid slowing down the test
#1935
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
test_shared_process_pool.py
is taking longer than expected when running on remote-ci, causing multiple ci pipelines failed due to (1h) timeout.This is probably because:
SharedProcessPool
is set to0.5 * os.sched_getaffinity(0)
by default. If the test is running on a machine that has 64 cores, it will maintain 32 sub-processes.join()
all of them when tear down, which might be slow when the number of processes is large.The test should run faster by setting the number of workers to
0.1 * os.sched_getaffinity(0)
and it should avoid ci from being blocked by timeout.Closes #1936
By Submitting this PR I confirm: