Support raft for HA storage

[Pondidum]

I saw issue #197 about adding Vault's Integrated Storage for HA, and wanted to try it.

I am not certain that using the $instance_ip_address is the best thing for the node_id, but wasn't sure what else to use (and it works...).

I added two flags: --enable-raft-backend and --raft-dir. --raft-dir should work identically to the --config-dir type flags, and the default direction is created by the install-vault script too.

I don't see any tests to verify this, other than it is working in my test aws account.

[brikis98]

Thanks for the PR! We need to resolve #196 first and can then take a look at this PR. Hoping to get to that in the next week or two. If anyone has cycles to look into it before then, help is very appreciated!

[Pondidum]

You're welcome!

One thing I have noticed with the raft ha storage is that if I stop Vault on a node, or terminate the node (gracefully), the Vault instance never leaves the raft list of peers when checked on other nodes.

[brikis98]

OK, thank you for your patience! We merged fixes in #195. Could you rebase?

The code changes here look great 👍

Would you be up for switching one of the examples in the examples folder to using Raft? Or alternatively, adding a new example + test? I want to make sure we have tests that are executing this code path.

[Pondidum]

I might have some time to do that later this week - I have just done the rebase for now.

Thanks

[jboero]

Is there an update on this? @Pondidum

[Pondidum]

This had completely slipped my mind (I switched to using the dynamodb backend for my deployment), but I think I will have some time over the next week - I have some demos on Wednesday, so it would be after that I think.

I'll set a reminder this time...

[Pondidum]

@jboero I've added an example which uses the raft backend, but there is an issue, which I also hit when trying to write the test for it too: The raft version needs the certificate on each machine to be valid for the machine's private IP, not just 127.0.0.1 and vault.service.consul, as otherwise the machines fail to cluster due to TLS errors.

I am not sure how to go about generating the right certificates ahead of time, as the private IP is unknown until the machines boot.

[RicoToothless]

@jboero I've added an example which uses the raft backend, but there is an issue, which I also hit when trying to write the test for it too: The raft version needs the certificate on each machine to be valid for the machine's private IP, not just 127.0.0.1 and vault.service.consul, as otherwise the machines fail to cluster due to TLS errors.

I am not sure how to go about generating the right certificates ahead of time, as the private IP is unknown until the machines boot.

I have a similar issue with integrating Vault plugin.

The plugin uses api_addr (default is private IP) to communicate with the vault server.

[jboero]

Yes it seems like this is a pickle. We shouldn't be using direct IPs when possible. Other than a nasty wildcard cert I'm not sure what the answer is.

[hashicorp-cla]

All committers have signed the CLA.