support config gpu memory factor #28
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: peiniliu <[email protected]>
Thor-wl
reviewed
Aug 16, 2022
| | `--config-file` | `$CONFIG_FILE` | `""` | | ||
|
|
||
| when starting volcano-device-plugin.yml, users can specify these parameters by adding args to the container 'volcano-device-plugin'. | ||
| For example: | ||
| - args: ["--gpu-strategy=number"] will let device plugin using the gpu-number strategy | ||
| - args: ["--gpu-strategy=share","--gpu-memory-factor=10"] will let device plugin using the gpu-share strategy, and memory factor is 10MB |
There was a problem hiding this comment.
Are there some limit about the parameter gpu-memory-factor? For example, could it be a decimal？
There was a problem hiding this comment.
This is not recommended generally, the most common setting is 1MiB / 1 GiB
There was a problem hiding this comment.
Yes, so I mean it should be highlight.
Thor-wl
approved these changes
Aug 18, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As described in #19 and #22, the device plugin needs to support config GPU memory factor to control the number of virtual devices that Listandwatched by kubelet. By default, we support each memory block as 1MB, this PR support config this factor, so that users can specify their own GPU memory granularity.
Tested Results:
2022/08/15 08:57:18 Loading configuration.
2022/08/15 08:57:18
Running with config:
{
"version": "v1beta1",
"flags": {
"GPUStrategy": "share",
"GPUMemoryFactor": 10
}
}
ENV of pod1.yml
NVIDIA_VISIBLE_DEVICES=0
NVIDIA_DRIVER_CAPABILITIES=compute,utility
VOLCANO_GPU_MEMORY_TOTAL=15109
VOLCANO_GPU_ALLOCATED=10240
Signed-off-by: peiniliu [email protected]