Skip to content

Commit

Permalink
Rolback gpu mode
Browse files Browse the repository at this point in the history
Signed-off-by: Jean-Yves <[email protected]>
  • Loading branch information
docjyJ committed Dec 17, 2024
1 parent c64908e commit 0fdd137
Show file tree
Hide file tree
Showing 12 changed files with 54 additions and 31 deletions.
5 changes: 4 additions & 1 deletion community-containers/jellyfin/jellyfin.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,10 @@
"writeable": false
}
],
"enable_gpu": true,
"devices": [
"/dev/dri"
],
"enable_nvidia_gpu": true,
"backup_volumes": [
"nextcloud_aio_jellyfin"
]
Expand Down
5 changes: 4 additions & 1 deletion community-containers/memories/memories.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,10 @@
"writeable": false
}
],
"enable_gpu": true,
"devices": [
"/dev/dri"
],
"enable_nvidia_gpu": true,
"nextcloud_exec_commands": [
"php /var/www/html/occ app:install memories",
"php /var/www/html/occ app:enable memories",
Expand Down
5 changes: 4 additions & 1 deletion community-containers/plex/plex.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,10 @@
"writeable": false
}
],
"enable_gpu": true,
"devices": [
"/dev/dri"
],
"enable_nvidia_gpu": true,
"backup_volumes": [
"nextcloud_aio_plex"
]
Expand Down
3 changes: 2 additions & 1 deletion compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ services:
# NEXTCLOUD_STARTUP_APPS: deck twofactor_totp tasks calendar contacts notes # Allows to modify the Nextcloud apps that are installed on starting AIO the first time. See https://github.com/nextcloud/all-in-one#how-to-change-the-nextcloud-apps-that-are-installed-on-the-first-startup
# NEXTCLOUD_ADDITIONAL_APKS: imagemagick # This allows to add additional packages to the Nextcloud container permanently. Default is imagemagick but can be overwritten by modifying this value. See https://github.com/nextcloud/all-in-one#how-to-add-os-packages-permanently-to-the-nextcloud-container
# NEXTCLOUD_ADDITIONAL_PHP_EXTENSIONS: imagick # This allows to add additional php extensions to the Nextcloud container permanently. Default is imagick but can be overwritten by modifying this value. See https://github.com/nextcloud/all-in-one#how-to-add-php-extensions-permanently-to-the-nextcloud-container
# NEXTCLOUD_GPU_MODE: 'dri' # 'dri', 'nvidia-runtime' or 'nvidia-deploy':This allows to enable the GPU acceleration in the Nextcloud container. Make sure you follow https://github.com/nextcloud/all-in-one#how-to-enable-gpu-acceleration-for-nextcloud before setting this value.
# NEXTCLOUD_ENABLE_DRI_DEVICE: true # This allows to enable the /dev/dri device in the Nextcloud container. ⚠️⚠️⚠️ Warning: this only works if the '/dev/dri' device is present on the host! If it should not exist on your host, don't set this to true as otherwise the Nextcloud container will fail to start! See https://github.com/nextcloud/all-in-one#how-to-enable-hardware-transcoding-for-nextcloud
# NEXTCLOUD_NVIDIA_GPU_MODE: 'runtime' # 'runtime' or 'deploy': This allows to enable the [NVIDIA runtime](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) or [GPU access](https://docs.docker.com/compose/gpu-support/) in the Nextcloud container. Make sure you follow the instructions before setting this value. If you're using WSL2 and want to use the NVIDIA runtime, please follow the instructions to [install the NVIDIA Container Toolkit meta-version in WSL](https://docs.nvidia.com/cuda/wsl-user-guide/index.html#cuda-support-for-wsl-2).
# NEXTCLOUD_KEEP_DISABLED_APPS: false # Setting this to true will keep Nextcloud apps that are disabled in the AIO interface and not uninstall them if they should be installed. See https://github.com/nextcloud/all-in-one#how-to-keep-disabled-apps
# SKIP_DOMAIN_VALIDATION: false # This should only be set to true if things are correctly configured. See https://github.com/nextcloud/all-in-one?tab=readme-ov-file#how-to-skip-the-domain-validation
# TALK_PORT: 3478 # This allows to adjust the port that the talk container is using. See https://github.com/nextcloud/all-in-one#how-to-adjust-the-talk-port
Expand Down
2 changes: 1 addition & 1 deletion php/containers-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@
"pattern": "^/dev/[a-z]+$"
}
},
"enable_gpu": {
"enable_nvidia_gpu": {
"type": "boolean"
},
"apparmor_unconfined": {
Expand Down
5 changes: 4 additions & 1 deletion php/containers.json
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,10 @@
],
"stop_grace_period": 600,
"restart": "unless-stopped",
"enable_gpu": true,
"devices": [
"/dev/dri"
],
"enable_nvidia_gpu": true,
"backup_volumes": [
"nextcloud_aio_nextcloud"
],
Expand Down
6 changes: 3 additions & 3 deletions php/src/Container/Container.php
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ public function __construct(
private array $secrets,
/** @var string[] */
private array $devices,
private bool $enable_gpu,
private bool $enable_nvidia_gpu,
/** @var string[] */
private array $capAdd,
private int $shmSize,
Expand Down Expand Up @@ -93,8 +93,8 @@ public function GetDevices() : array {
return $this->devices;
}

public function GetGpuMode() : bool {
return $this->enable_gpu;
public function CanUseNidiaGpu() : bool {
return $this->enable_nvidia_gpu;
}

public function GetCapAdds() : array {
Expand Down
2 changes: 1 addition & 1 deletion php/src/ContainerDefinitionFetcher.php
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ private function GetDefinition(): array
$devices = $entry['devices'];
}

$enable_gpu = $entry['enable_gpu'] === true;
$enable_gpu = $entry['enable_nvidia_gpu'] === true;

$capAdd = [];
if (isset($entry['cap_add'])) {
Expand Down
24 changes: 17 additions & 7 deletions php/src/Data/ConfigurationManager.php
Original file line number Diff line number Diff line change
Expand Up @@ -968,24 +968,34 @@ public function GetEnabledCommunityContainers() : array {
return explode(' ', $this->GetCommunityContainers());
}


private function GetEnabledGPUMode() : string {
$envVariableName = 'NEXTCLOUD_GPU_MODE';
$configName = 'nextcloud_gpu_mode';
private function GetEnabledDriDevice() : string {
$envVariableName = 'NEXTCLOUD_ENABLE_DRI_DEVICE';
$configName = 'nextcloud_enable_dri_device';
$defaultValue = '';
return $this->GetEnvironmentalVariableOrConfig($envVariableName, $configName, $defaultValue);
}

public function isDriDeviceEnabled() : bool {
return $this->GetEnabledGPUMode() === 'dri';
if ($this->GetEnabledDriDevice() === 'true') {
return true;
} else {
return false;
}
}

private function GetEnabledGPUMode() : string {
$envVariableName = 'NEXTCLOUD_NVIDIA_GPU_MODE';
$configName = 'nextcloud_nvidia_gpu_mode';
$defaultValue = '';
return $this->GetEnvironmentalVariableOrConfig($envVariableName, $configName, $defaultValue);
}

public function isNvidiaRuntimeEnabled() : bool {
return $this->GetEnabledGPUMode() === 'nvidia-runtime';
return $this->GetEnabledGPUMode() === 'runtime';
}

public function isNvidiaDeployEnabled() : bool {
return $this->GetEnabledGPUMode() === 'nvidia-deploy';
return $this->GetEnabledGPUMode() === 'deploy';
}

private function GetKeepDisabledApps() : string {
Expand Down
17 changes: 9 additions & 8 deletions php/src/Docker/DockerActionManager.php
Original file line number Diff line number Diff line change
Expand Up @@ -481,13 +481,18 @@ public function CreateContainer(Container $container) : void {

$devices = [];
foreach($container->GetDevices() as $device) {
if ($device === '/dev/dri' && ! $this->configurationManager->isDriDeviceEnabled()) {
continue;
}
$devices[] = ["PathOnHost" => $device, "PathInContainer" => $device, "CgroupPermissions" => "rwm"];
}

if ($container->canUseNvidia()) {
if($this->configurationManager->isDriDeviceEnabled()) {
$devices[] = ["PathOnHost" => '/dev/dri', "PathInContainer" => '/dev/dri', "CgroupPermissions" => "rwm"];
} elseif ($this->configurationManager->isNvidiaRuntimeEnabled()) {
if (count($devices) > 0) {
$requestBody['HostConfig']['Devices'] = $devices;
}

if ($container->CanUseNidiaGpu()) {
if ($this->configurationManager->isNvidiaRuntimeEnabled()) {
$requestBody['HostConfig']['Runtime'] = 'nvidia';
} elseif ($this->configurationManager->isNvidiaDeployEnabled()) {
$requestBody['HostConfig']['DeviceRequests'] = [
Expand All @@ -500,10 +505,6 @@ public function CreateContainer(Container $container) : void {
}
}

if (count($devices) > 0) {
$requestBody['HostConfig']['Devices'] = $devices;
}

$shmSize = $container->GetShmSize();
if ($shmSize > 0) {
$requestBody['HostConfig']['ShmSize'] = $shmSize;
Expand Down
9 changes: 4 additions & 5 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -768,16 +768,15 @@ The [facerecognition app](https://apps.nextcloud.com/apps/facerecognition) requi
### How to enable GPU acceleration for Nextcloud?
Some container can use GPU acceleration to increase performance like [memories app](https://apps.nextcloud.com/apps/memories) allows to enable hardware transcoding for videos.
To enable it you need to add `--env NEXTCLOUD_GPU_MODE=<mode>` to the docker run command of the mastercontainer (but before the last line `nextcloud/all-in-one:latest`! If it was started already, you will need to stop the mastercontainer, remove it (no data will be lost) and recreate it using the docker run command that you initially used).
#### With open source drivers MESA for AMD, Intel and **new** drivers `Nouveau` for Nvidia
> [!WARNING]
> This only works if the `/dev/dri` device is present on the host! If it does not exists on your host, don't proceed as otherwise the Nextcloud container will fail to start! If you are unsure about this, better do not proceed with the instructions below. Make sure that your driver is correctly configured on the host.
A list of supported device can be fond in [MESA 3D documentation](https://docs.mesa3d.org/systems.html).
This methode use the [Direct Rendering Infrastructure](https://dri.freedesktop.org/wiki/) with the access to the `/dev/dri` device. To enable it, use `--env NEXTCLOUD_GPU_MODE=dri` as mode.
This methode use the [Direct Rendering Infrastructure](https://dri.freedesktop.org/wiki/) with the access to the `/dev/dri` device. In order to use that, you need to add `--env NEXTCLOUD_ENABLE_DRI_DEVICE=true` to the docker run command of the mastercontainer (but before the last line `nextcloud/all-in-one:latest`! If it was started already, you will need to stop the mastercontainer, remove it (no data will be lost) and recreate it using the docker run command that you initially used) which will mount the `/dev/dri` device into the container. There is now a community container which allows to easily add the transcoding container of Memories to AIO: https://github.com/nextcloud/all-in-one/tree/main/community-containers/memories

Check failure on line 778 in readme.md

View workflow job for this annotation

GitHub Actions / Check spelling

methode ==> method
#### With proprietary drivers for Nvidia :warning: BETA
Expand All @@ -788,9 +787,9 @@ This methode use the [Direct Rendering Infrastructure](https://dri.freedesktop.o
This methode use the [Nvidia Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/index.html) with the nvidia runtime.

Check failure on line 788 in readme.md

View workflow job for this annotation

GitHub Actions / Check spelling

methode ==> method
To enable it, use `--env NEXTCLOUD_GPU_MODE=nvidia-runtime` as mode.
To enable it, use `--env NEXTCLOUD_NVIDIA_GPU_MODE=runtime` to enable runtime.
You can also use [docker resource allocation](https://docs.docker.com/compose/gpu-support/) with `--env NEXTCLOUD_GPU_MODE=nvidia-deploy` as mode.
You can also use [docker resource allocation](https://docs.docker.com/compose/gpu-support/) with `--env NEXTCLOUD_NVIDIA_GPU_MODE=deploy` as mode.
If you're using WSL2 and want to use the NVIDIA runtime, please follow the instructions to [install the NVIDIA Container Toolkit meta-version in WSL](https://docs.nvidia.com/cuda/wsl-user-guide/index.html#cuda-support-for-wsl-2).
Expand Down
2 changes: 1 addition & 1 deletion tests/QA/060-environmental-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ See https://github.com/nextcloud/all-in-one#how-to-trust-user-defined-certificat
- [ ] When starting the mastercontainer with `--env NEXTCLOUD_STARTUP_APPS=deck`, the resulting Nextcloud should have only installed the deck app and not the other apps that get installed by default. Default are `deck twofactor_totp tasks calendar contacts notes`.
- [ ] When starting the mastercontainer with `--env NEXTCLOUD_ADDITIONAL_APKS=zip`, the resulting Nextcloud container should have the zip package installed and not imagemagick.
- [ ] When starting the mastercontainer with `--env NEXTCLOUD_ADDITIONAL_PHP_EXTENSIONS=inotify`, the resulting Nextcloud container should have the inotify extension installed and not the imagick extension.
- [ ] When starting the mastercontainer with `--env NEXTCLOUD_GPU_MODE=dri`, the resulting Nextcloud container should have the /dev/dri device mounted into the container. (Only works if a `/dev/dri` device is present on the host)
- [ ] When starting the mastercontainer with `--env NEXTCLOUD_ENABLE_DRI_DEVICE=true`, the resulting Nextcloud container should have the /dev/dri device mounted into the container. (Only works if a `/dev/dri` device is present on the host)
- [ ] When starting the mastercontainer with `--env NEXTCLOUD_KEEP_DISABLED_APPS=true` it should keep apps in Nextcloud that are disabled in the AIO interface. For example if Collabora is disabled in the AIO interface and you install the richdocuments app in Nextcloud, a restart should not uninstall the richdocuments app in Nextcloud anymore.
- [ ] When starting the mastercontainer with `--env AIO_COMMUNITY_CONTAINERS="fail2ban"`, it should add the fail2ban container to the container stack and show it in the AIO interface as well as start it, etc.

Expand Down

0 comments on commit 0fdd137

Please sign in to comment.