Skip to content

Upgrade the prod server

Arnaud Morvan edited this page Dec 7, 2016 · 12 revisions

Connect to the composition manager server

ssh compose0.exoscale.infra.camptocamp.org

a bunch of envvars with the list of all the servers

source /opt/c2corg-docker/server-list.env

one directory for each production server, self-descriptive names:

ls /opt/c2corg-docker/composition/production/

choose one

cd /opt/c2corg-docker/composition/production/api0/

Notice the hidden ./.env in this directory. https://docs.docker.com/compose/environment-variables/#/the-env-file This envvar file teaches docker-compose to connect to a remote docker instance. Each per-server directory has it's own .env file.

You can now edit the docker-compose.yml file. In this example, you'll want to change the image version to a more recent one which travis made available on dockerhub (https://hub.docker.com/r/c2corg/v6_api/tags/).

download the new image

docker-compose pull

shutdown the running container(s)

docker-compose down

re-create it with the new image version (you'll notice 2 monitoring

containers get stopped/started too)

docker-compose up -d

make sure everything is working properly

docker-compose ps docker-compose logs -f

don't forget to save your changes

git add . && git commit

BTW, please don't clone this repository elsewhere. It's full of credentials, and it's backuped on the c2c.org backup server.

Now repeat the docker-compose dance for the api1 and api2 subdirectories. Congrats, you've done a rolling upgrade of the service with zero downtime !

You can check haproxy's status here: http://www.camptocamp.org:8008/stats (assuming you have modified your /etc/hosts). You should notice one backend going offline if you refresh the page between docker-compose "down" and "up".

Also, I setup 2 basic kibana dashboards (more to come) you'll probably want to keep track of the first few days:

https://c2corgv6.logs.camptocamp.net/kibana/#/dashboard/elasticsearch/Python%20stacktraces https://c2corgv6.logs.camptocamp.net/kibana/#/dashboard/elasticsearch/HTTP%2050x