Configuring S3 for primary storage via Environment variables not working

[Caffe1neAdd1ct]

Steps to reproduce

1. Create a .env file with S3 storage variables configured:

OBJECTSTORE_S3_BUCKET=true
OBJECTSTORE_S3_SSL=true
OBJECTSTORE_S3_USEPATH_STYLE=false
OBJECTSTORE_S3_LEGACYAUTH=false
OBJECTSTORE_S3_AUTOCREATE=false
OBJECTSTORE_S3_BUCKET=bucket_name_here
OBJECTSTORE_S3_KEY=user_key_here
OBJECTSTORE_S3_SECRET=user_secret_here
OBJECTSTORE_S3_REGION=lowercase_region_here
OBJECTSTORE_S3_HOST=s3.host.com
OBJECTSTORE_S3_PORT=443
OBJECTSTORE_S3_OBJECT_PREFIX="urn:oid:"

2. Launch AIO main container

docker run \
    --sig-proxy=false \
    --name nextcloud-aio-mastercontainer \
    --restart always \
    --publish 80:80 \
    --publish 8080:8080 \
    --publish 8443:8443 \
    --volume nextcloud_aio_mastercontainer:/mnt/docker-aio-config \
    --volume /var/run/docker.sock:/var/run/docker.sock:ro \
    --env-file ./.env \
    nextcloud/all-in-one:latest

2. Launch all containers and get the stack running
3. Check S3 env variables are available to AIO main container:

docker exec -it nextcloud-aio-mastercontainer bash
env | grep OBJECTSTORE

These should be showing

4. Check S3 env vairables are available to the nextcloud container:

docker exec -it nextcloud-aio-nextcloud bash
env | grep OBJECTSTORE

Should be available and aren't, so configs in https://github.com/nextcloud/all-in-one/blob/main/Containers/nextcloud/config/s3.config.php via getenv() calls fail to a blank string / fallback option

Going to make a guess we need these configured here to be passed through to container creation:

all-in-one/php/containers.json

Line 117 in 946e214

"environment": [

Passing envs via -e OBJECTSTORE_S3_BUCKET=true also not working.

Expected behavior

S3 configured correctly as primary storage via Environment variables

Actual behavior

S3 not working as primary storage

Host OS

Rocky Linux 8

Nextcloud AIO version

Nextcloud AIO v4.1.0

Current channel

Latest

Other valuable info

Edit - typo

[szaimen]

Hi, moving to discussions since this is a feature request

[szaimen]

Hi, indeed it is known that these variables do not get forwarde to the Nextcloud container. It is actually intended since all Backup and Restore features will not work anymore when using primary object storage.

You can now either mount your bucket via the host (or a docker volume) in a location that can be used as datadir via https://github.com/nextcloud/all-in-one#how-to-change-the-default-location-of-nextclouds-datadir which brings the advantage that the built-in backup solution works, or you use this: https://github.com/nextcloud/all-in-one/tree/main/manual-install

[Caffe1neAdd1ct]

@szaimen thanks, only reason i wondered is i saw the tmp directory being configured here (

all-in-one/Containers/nextcloud/entrypoint.sh

Line 359 in d66002a

if [ -z "$OBJECTSTORE_S3_BUCKET" ] && [ -z "$OBJECTSTORE_SWIFT_URL" ]; then

) when object storage has been configured, however this container would never see this environment variable in the AIO setup.

Would be great if a similar conditional could disable the borg backup when primary object storage is detected plus pass these variables along to the nextcloud container.

I didn't come across any documentation to say primary object storage isn't supported by direct configuration (may have missed it) i'll double check and raise a pull if i still can't spot any references. Personally i'd much rather use primary object storage configured and used directly instead of going through a fuse / mounted option for performance.

Would you entertain the above suggestion if I raise as a change? If this doesn't feel right for the AIO project then i'll look at a manual setup or a patched main container 👍

[szaimen]

@szaimen thanks, only reason i wondered is i saw the tmp directory being configured here (

all-in-one/Containers/nextcloud/entrypoint.sh

Line 359 in d66002a

if [ -z "$OBJECTSTORE_S3_BUCKET" ] && [ -z "$OBJECTSTORE_SWIFT_URL" ]; then

This is actually only so that it does not break on manual-install.

) when object storage has been configured, however this container would never see this environment variable in the AIO setup.
Would be great if a similar conditional could disable the borg backup when primary object storage is detected plus pass these variables along to the nextcloud container.

There actually is a way to disable backup options but the intention is that this is only done if you have your own backup solution.

I didn't come across any documentation to say primary object storage isn't supported by direct configuration (may have missed it) i'll double check and raise a pull if i still can't spot any references. Personally i'd much rather use primary object storage configured and used directly instead of going through a fuse / mounted option for performance.

Yeah, we don't have a documentation on this because you are the first one that requests this.

Would you entertain the above suggestion if I raise as a change? If this doesn't feel right for the AIO project then i'll look at a manual setup or a patched main container 👍

I am not sure if we want to add support for this, honestly as you can already use the manual-install for this.

[Caffe1neAdd1ct]

This is actually only so that it does not break on manual-install.

Makes perfect sense 👌

There actually is a way to disable backup options but the intention is that this is only done if you have your own backup solution.

Agreed, enabling primary object storage and automatically disabling the advertised automatic backup solution would be a major gotcha, not sure if documentation is enough on that front.

On the flip side it would be great to continue using the AIO main container with the caveat that the backup solution won't work in this situation as it has some other major maintenance advantages over the manual install / docker-compose:

You lose the AIO interface
You lose update notifications and automatic updates
You need to know what you are doing, especially when modifying the docker-compose file
For updating, you need to strictly follow the at the bottom described update routine

Automatic updates, notifications, and low knowledge over setting up the docker(-compose) containers manually are still some great/big advantages.

Would the backup solution still provide some value, e.g DB backups? Object storage backup / replication can most likely be managed inside the relevant cloud / infrastructure solution.

I am not sure if we want to add support for this, honestly as you can already use the manual-install for this.

Thanks for spending time reviewing and coming back 👍

[szaimen]

Would the backup solution still provide some value, e.g DB backups?

It would simply not work as it expects your files do be in a certain location and would not start when it does not find them there. Also when restoring, you would usually need to restore the db and the files at the same time since otherwise the db and your files get out of sync which is one of the worst case scenarios that could happen when you use primary storage as scanning is not possible or at least very limited afaik.

[Caffe1neAdd1ct]

I've had a quick look at patching a master container and build/tag locally, would be a pain as the docker hub locations have been coded into the update checks for the master container, little point in maintaining more than just the containers.json env variables modification imo.

So unless adding the OBJECTSTORE_* environment variables to the master container nextcloud definition with the pitiful of knowing backups will need to be manually taken care of would be acceptable, primary S3 storage on AIO is out of possibility which is shame (for me at least).

Being able to fire up a nextcloud AIO instance on a VPS or Compute Instance with storage taken care of remotely on a cloud s3 or min.io cluster would be ideal for many setups i'd have thought, especially with the container updates taken care of.

AIO is well suited to mounted block storage with borg backup, but the primary object storage would need more of a think.

There is still the possibility of external user storage configured for object storage, downsides on that front are performance and manual configuration for each user which is something I'd personally want to avoid. Plus users can just choose to upload outside of the configured folder straight onto the instance storage which is what i'm trying to avoid (in-bound out-of-diskspace logwatch warnings... )

I'll go down the manual setup road for now 👍

[szaimen]

Hi, I just wanted to leave this here as someone figured out how to mount its object via the host: https://help.nextcloud.com/t/setting-up-object-storage-as-primary-storage-for-nextcloud-aio-docker-container/157178/8

[jahanson]

Tried this out and wow does it run poorly. I suggest another method if not supported natively by aio.

[szaimen]

Hi, you could check if the performance using https://github.com/awslabs/mountpoint-s3 is better

[jahanson]

I'll try it, and I think you can even set the endpoint url. Since i'm using a minio cluster on baremetal @ hetzner it may not be supported because it's not apart of their mission currently to prioritize non-aws S3 services awslabs/mountpoint-s3#144 (comment)

[HDW-Viakom]

Could you please provide an update on the current discussion? We are planning to use S3 as our primary storage.

Additionally, are there any tips or tricks from others who have experience with manually installing and integrating S3?

[Caffe1neAdd1ct]

Running via the manual docker compose yaml on primary S3 is working fine for me, however it's worth mentioning backups won't be performed automatically via aio and the auto-updates won't work, it's a manual update of the yaml and docker pull for the latest container versions.

I have a different solution for DB / container and S3 file backups to replace this functionality for now.

I think there may have been something like a manual file to be created to satisfy the aio start process, then S3 has been very stable and reliable as a primary storage option (for me).

[mitexleo]

I'm also using s3 as primary storage without any performance issue. How do you backup the configuration files and database? I couldn't find any guide/docs to setup automated/manual backup. I thought about creating backups of the /var/lib/docker/volumes directory manually. Not sure I will be able to restore it later or not.

[Caffe1neAdd1ct]

Personally i'm taking a Logical Volume snapshot of the entire VM running AIO / docker (slightly overkill but works for me.. )

Backing up the docker volumes and compose / config files should be enough if you know or can work out how to restore.

[szaimen]

Additionally see this doc: https://github.com/nextcloud/all-in-one?tab=readme-ov-file#how-to-enable-automatic-updates-without-creating-a-backup-beforehand

[rasos]

So if configuring S3 does not work via environment variables, what is the recommended approach for making S3 available for primary storage without having to build my own image? Simply add it to /var/lib/docker/volumes/nextcloud_aio_nextcloud/_data/config/config.php? I guess this is not persistent but overwritten. Or can I just replace s3.config.php with

'objectstore' => array(
        'class' => 'OC\\Files\\ObjectStore\\S3',
        'arguments' => array(
                'bucket' => 'nc-bucket',
                'autocreate' => true,
                'key'    => 'xxxxxxxxxxxxxxxxxxxx',
                'secret' => 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy',
                'hostname' => 's3.<region_in_lowercase>.io.cloud.ovh.net',
                'port' => 443,
                'use_ssl' => true,
                'region' => '<region_in_lowercase>',
                'use_path_style' => false
        ),
),

[Caffe1neAdd1ct]

The approach i've taken is to use a custom docker compose setup, so using the AIO containers but not the main updater or controller container. Here you can specify the container environment variables and setup S3 as primary, also the AIO backup solution doesn't work with this setup so you wouldn't want that running automatically.

@rasos see https://github.com/nextcloud/all-in-one/tree/main/manual-install

[qstraza]

Can you explain in more detail? I have edited manual-install/compose.yaml file by adding S3 environmental variables to the nextcloud-aio-nextcloud container, but it does not work.

[qstraza]

I think I figured it out. When I used OBJECTSTORE_S3_ variables in .env and in compose.yml, config.php got generated with information about the S3, which is OK. But it did not work (page does not load - server error...). I went to edit the config.php of the nextcloud (location of the file is mentioned at the bottom of the post) by deleting all extra keys in the arguments array inside objectstore key. I left only arguments which are also mentioned in the documentation here https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/primary_storage.html

This is what I left in the config.php:

'objectstore' =>
array (
'class' => '\OC\Files\ObjectStore\S3',
'arguments' =>
array (
'bucket' => '...',
'key' => '*',
'secret' => '...',
'region' => '****',
),
),

Location of the config.php is
/var/lib/docker/volumes/nextcloud_aio_nextcloud/_data/config/config.php

I did not investigate which argument causes this issue.