Missing certs in kube-controller-manager

[indradhanush]

Currently we run the following to configure the kube-controller-manager component:

# set up the kube-controller-manager
vagrant ssh node-0 -- sudo hab svc load core/kubernetes-controller-manager --channel "${channel}"

cat <<'EOF' | vagrant ssh node-0 -- bash
for f in /vagrant/certificates/{ca.pem,ca-key.pem}; do sudo hab file upload kubernetes-controller-manager.default $(date +%s) "${f}"; done
EOF
vagrant ssh node-0 -- sudo hab config apply kubernetes-controller-manager.default $(date +%s) /vagrant/config/svc-kubernetes-controller-manager.toml

However, the certificates do not get uploaded to /hab/svc/kubernetes-controller-manager/files and as a result the controller keeps getting restarted. This is an intermittent issue and hard to reproduce. Conditions that reproduce it are unknown as of yet.

Running the command again after ssh-ing into the node-0 works:

for f in /vagrant/certificates/{ca.pem,ca-key.pem}; do sudo hab file upload kubernetes-controller-manager.default $(date +%s) "${f}"; done

Here's more information that may or may not be relevant:

$ uname -a
Linux xps 4.18.5-arch1-1-ARCH #1 SMP PREEMPT Fri Aug 24 12:48:58 UTC 2018 x86_64 GNU/Linux

$ vagrant -v
Vagrant 2.1.4

$ virtualbox -h
Oracle VM VirtualBox Manager 5.2.18