Terraform helper that allows you to apply rolling updates on resources with counts.
The first step is to partition the instances into cycles
partition, resource = 2, [ 'node-1', 'node-2', 'node-3', 'node-4', 'node-5' ]
The update of this resource was split into 3 cycles where len(cycle) <= partition
, meaning that no more than len(cycle)
nodes can be down at any given time
cycles = [ [ 'node-1', 'node-2' ], [ 'node-3', 'node-4' ], [ 'node-5' ] ]
In each cycle, a terraform apply
or terraform destroy+apply
is run, targeting only the instances from that cycle
terraform apply -target resource.name['node-1'] -target resource.name['node-2']
bash <<EOF
exit_code=$(healhcheck_command)
until [ $exit_code -eq 0 ]; do
exit_code=$(healhcheck_command)
done
EOF
continue
A cycle waits for each instance to pass its health checks before proceeding
Upgrading a Vault cluster:
Placeholder for video
Example of config.yaml
with good metadata:
# Override default command terraform or add flags to it
command: terraform -lock=true -no-color
# Name of the terraform resource to be updated
name: vsphere_virtual_machine.vault_server
# Maximum no. of instances to be updated in one cycle
partition: 1
# Force destroy of the instance, use where providers don't detect changes properly
recreate: yes
# Healtcheck condition that must be satisfied in order to proceed to the next cycle
healthcheck:
# Command used to check instance health, available environment variables are:
# $INDEX $COUNT $INSTANCE_IP $INSTANCE_NAME
exec: |
#!/bin/bash
http_code=$(curl -sw '%{http_code}' http://${INSTANCE_IP}:8200/v1/sys/health -o /dev/null)
if [ ${http_code} -eq 200 ]; then
exit 0
fi
exit 1
# Or provide a script file instead of the exec, it must be executable,
# have a shebang and be present in the root folder
script: health.py
# Initial delay after finishing an apply and before starting the checks
delay: 5m
# How much to wait between healthchecks
period: 15s
Make sure you have terraform installed
Just run the following command at the root of your project:
rollotf apply -config vault.yaml
Or provide the config from stdin:
cat <<EOF | rollotf apply -
name: vsphere_virtual_machine.vault_server
partition: 1
recreate: yes
healthcheck:
script: health.py
delay: 1m
period: 15s
EOF
Generate default config:
rollotf config > default.yaml
This code is released under the MIT License. See LICENSE.