Selfhosting with docker on VPS with reverse proxy and docker networks.

[eyduh]

Hi,

intro:

I have a budibase instance running on a VPS where i run everything in docker containers. I prefer to keep all the ports locked down except for the ones that my reverse proxy use.

To do this, I use the very handy docker networks feature so that I can resolve various microservices using their containers names.

With a docker-compose.yml setup, everything in that cluster get added to the same network automagically, but what if I want to be able to restart the flat file CMS that I use for the front page without having the budibase instance go down? Enter docker networks!

how to:

To use this feature I first create an "external" docker network.

docker network create reverse-proxy
I call my network reverse-proxy because I use the same network for all services, however if you want a slightly more secure setup you can make one network for each service and add the reverse proxy to every network.

After you've made your docker network you simply add the following two sections line to all your docker-compose files:

    networks:
      - reverse-proxy 

to each "service" section
and

networks:
  reverse-proxy:
    external: yes

to the bottom of all the docker-compose.yml files.

When you have done this you can comment out the ports: section on all the services, in the case of budibase that\s only on the bbproxy container.

A modified budibase docker-compose.yml would look like this:

version: "3"

# optional ports are specified throughout for more advanced use cases.

services:
  app-service:
    restart: unless-stopped
    image: budibase.docker.scarf.sh/budibase/apps
    container_name: bbapps 
    environment:
      SELF_HOSTED: 1
      COUCH_DB_URL: http://${COUCH_DB_USER}:${COUCH_DB_PASSWORD}@couchdb-service:5984
      WORKER_URL: http://worker-service:4003
      MINIO_URL: http://minio-service:9000
      MINIO_ACCESS_KEY: ${MINIO_ACCESS_KEY}
      MINIO_SECRET_KEY: ${MINIO_SECRET_KEY}
      INTERNAL_API_KEY: ${INTERNAL_API_KEY}
      BUDIBASE_ENVIRONMENT: ${BUDIBASE_ENVIRONMENT}
      PORT: 4002
      JWT_SECRET: ${JWT_SECRET}
      LOG_LEVEL: info
      SENTRY_DSN: https://[email protected]/5338131
      ENABLE_ANALYTICS: "true"
      REDIS_URL: redis-service:6379
      REDIS_PASSWORD: ${REDIS_PASSWORD}
      BB_ADMIN_USER_EMAIL: ${BB_ADMIN_USER_EMAIL}
      BB_ADMIN_USER_PASSWORD: ${BB_ADMIN_USER_PASSWORD}
    depends_on:
      - worker-service
      - redis-service
    networks:
      - reverse-proxy 

  worker-service:
    restart: unless-stopped
    image: budibase.docker.scarf.sh/budibase/worker
    container_name: bbworker
    environment:
      SELF_HOSTED: 1
      PORT: 4003
      CLUSTER_PORT: ${MAIN_PORT}
      JWT_SECRET: ${JWT_SECRET}
      MINIO_ACCESS_KEY: ${MINIO_ACCESS_KEY}
      MINIO_SECRET_KEY: ${MINIO_SECRET_KEY}
      MINIO_URL: http://minio-service:9000
      APPS_URL: http://app-service:4002
      COUCH_DB_USERNAME: ${COUCH_DB_USER}
      COUCH_DB_PASSWORD: ${COUCH_DB_PASSWORD}
      COUCH_DB_URL: http://${COUCH_DB_USER}:${COUCH_DB_PASSWORD}@couchdb-service:5984
      SENTRY_DSN: https://[email protected]/5338131
      INTERNAL_API_KEY: ${INTERNAL_API_KEY}
      REDIS_URL: redis-service:6379
      REDIS_PASSWORD: ${REDIS_PASSWORD}
    depends_on:
      - redis-service
      - minio-service
      - couch-init
    networks:
      - reverse-proxy 

  minio-service:
    restart: unless-stopped
    image: minio/minio
    volumes:
      - minio_data:/data
    environment:
      MINIO_ACCESS_KEY: ${MINIO_ACCESS_KEY}
      MINIO_SECRET_KEY: ${MINIO_SECRET_KEY}
      MINIO_BROWSER: "off"
    command: server /data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3
    networks:
      - reverse-proxy 

  proxy-service:
    restart: unless-stopped
#   ports:
#     - ${MAIN_PORT}:10000
    container_name: bbproxy
    image: budibase/proxy
    depends_on:
      - minio-service
      - worker-service
      - app-service
      - couchdb-service
    networks:
      - reverse-proxy 

  couchdb-service:
    restart: unless-stopped
    image: ibmcom/couchdb3
    environment:
      - COUCHDB_PASSWORD=${COUCH_DB_PASSWORD}
      - COUCHDB_USER=${COUCH_DB_USER}
    volumes:
      - couchdb3_data:/opt/couchdb/data
    networks:
      - reverse-proxy 

  couch-init:
    image: curlimages/curl
    environment:
      PUT_CALL: "curl -u ${COUCH_DB_USER}:${COUCH_DB_PASSWORD} -X PUT couchdb-service:5984"
    depends_on:
      - couchdb-service
    command: ["sh","-c","sleep 10 && $${PUT_CALL}/_users && $${PUT_CALL}/_replicator; fg;"]
    networks:
      - reverse-proxy 

  redis-service:
    restart: unless-stopped
    image: redis
    command: redis-server --requirepass ${REDIS_PASSWORD}
    volumes:
      - redis_data:/data
    networks:
      - reverse-proxy 

  watchtower-service:
    restart: always
    image: containrrr/watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: --debug --http-api-update bbapps bbworker bbproxy
    environment:
      - WATCHTOWER_HTTP_API=true
      - WATCHTOWER_HTTP_API_TOKEN=budibase
      - WATCHTOWER_CLEANUP=true
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    networks:
      - reverse-proxy 

volumes:
  couchdb3_data:
    driver: local
  minio_data:
    driver: local
  redis_data:
    driver: local
networks:
  reverse-proxy:
    external: yes

reverse proxying

For reverse proxying with docker networks you need to add the reverse proxy to the same docker network.

npm

I often use nginx proxy manager, set up like so:

version: '3'
services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      - '80:80'
      - '10.6.0.1:81:81'
      - '443:443'
    networks:
      - reverse-proxy
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
networks:
  reverse-proxy:
    external: yes

Settings would be like so:


Note the use of http here, I tried with https as well but it wouldnt work. This this does not really matter as the unencrypted traffic is on the local docker network and not on an actual network.

Note:
the line
- '10.6.0.1:81:81' is mapping the web interface of nginx proxy manager to the ip address of the wireguard network I have setup on the VPS. If you have a different setup this mapping would be different. Alternatively, you can map it to 127.0.0.1 and use ssh tunnelling to access it on your local machine. ssh -L 127.0.0.1:81:[vps.vpn.ipv4.address]:81 user@[vps.vpn.ipv4.address]. Another way would be to comment this line out except for when you want to change the settings, then re-comment when done.

caddy

alternatively, you can use caddy for reverse proxying, the project below adds the ability to use docker container names to caddy reverse proxy:

version: "3.7"
services:
  caddy:
    image: lucaslorentz/caddy-docker-proxy:ci-alpine
    ports:
      - 80:80
      - 443:443
    environment:
      - CADDY_INGRESS_NETWORKS=reverse-proxy
    networks:
      - reverse-proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - caddy_data:/data
    restart: unless-stopped

networks:
  reverse-proxy:
    external: true

volumes:
  caddy_data: {}

in your caddy file, swap out localhost:10000 with bbproxy:10000

This will keep your setup slightly more secure by only having the necessary ports open to the outside world.

Enjoy!

[joebudi]

This is awesome! Thanks for contributing!