F add provision guides (hashicorp#40)

* Rename

* Update Vault best-practices guides

* Fix output warnings

* Fix TLS

* Use curl bash to run dev scripts

* Move provision guides under operations and cleanup repo

* Remove shared directory and use curl bash instead

* Use cleanup script

* Update outputs and PATH

* Rename files

* Update module source

* Use HTTPS instead of SSH for module sources

* Add READMEs and rename folders

* Update module outputs

* Update module sources

* Update vars

* f-refactor to master

* Update versions

* Update underscores with hyphens

* Cleanup repo

.gitignore

@@ -8,6 +8,7 @@
 # State file in any sub-folder
 **/terraform.tfstate
 **/terraform.tfstate.backup
+*.tfstate.lock.info
 
 # Terraform Variables
 *.tfvars
@@ -16,4 +17,4 @@
 *.pem
 
 *.DS_Store
-*.vagrant
+*.vagrant

README.md

@@ -7,33 +7,36 @@
 
 <img width="300" alt="Vault Logo" src="https://cloud.githubusercontent.com/assets/416727/24112835/03b57de4-0d58-11e7-81f5-9056cac5b427.png">
 
-----  
+----
 # Vault-Guides
-This repository aims to assist individuals in learning how to install, configure, and administer HashiCorp Vault.
 
-## provision
-This area will contain instructions to provision Vault and Consul as a first step to start using these tools.
+This repository provides the technical content to support the [Vault Guides](https://www.vaultproject.io/guides/index.html) site.
+
+## Operations
+
+This area will contain instructions to operatioanlize Vault.
+
+- [Provision a Dev Vault Cluster locally with Vagrant](operations/provision-vault/dev/vagrant-local)
+- [Provision a Dev Vault Cluster on AWS with Terraform](operations/provision-vault/dev/terraform-aws)
+- [Provision a Quick Start Vault & Consul Cluster on AWS with Terraform](operations/provision-vault/quick-start/terraform-aws)
+- [Provision a Best Practices Vault & Consul Cluster on AWS with Terraform](operations/provision-vault/best-practices/terraform-aws)
+
+## Secrets
 
-These may include use cases installing Vault in cloud services via Terraform, or within virtual environments using Vagrant, or running Vault in a local development mode.  
+This directory contains example use cases involving [secrets management](https://www.vaultproject.io/docs/secrets/index.html).
 
-## encryption
-This directory contains example usage of the [Vault Transit backend](https://www.vaultproject.io/docs/secrets/transit/index.html). Also referred to as 'Encryption as a Service' as it allows organizations to provide a centrally managed encryption service for their infrastructure.
+## Identity
 
-## secrets
-This directory contains example use cases involving [secrets management](https://www.vaultproject.io/docs/secrets/index.html). Secure secret storage of static secrets and sensitive information. Implementation of Dynamic Secrets.
+This directory contains example use cases involving [identity](https://www.vaultproject.io/docs/auth/index.html).
 
-## identity
-This directory contains examples of privilege access management, identity, access control.
+## Encryption
 
-## governance
-Guides related to governance within Vault stored here. Sentinel guides only available for Vault Enterprise Premium.
+This directory contains example use cases involving [encryption as a service](https://www.vaultproject.io/docs/secrets/transit/index.html).
 
-## shared
-This directory contains common scripts and configuration files used to provision environments used for the guides in this repository.
+## Assets
 
-## assets
 This directory contains graphics and other material for the repository.
 
+## Contributing
 
-### Contributing
-We welcome contributions and feedback!  For guide submissions, please see [the contributions guide](CONTRIBUTING.md)
+We welcome contributions and feedback! For guide submissions, please see [the contributions guide](CONTRIBUTING.md)

governance/README.md

@@ -1 +1 @@
-Guides related to governance within Vault stored here. Sentinel guides only available for Vault Enterprise Premium.
+Guides related to governance within Vault stored here. Sentinel guides only available for Vault Enterprise Premium.

identity/ssh_ca/vagrant/QUICKSTART.md → identity/ssh-ca/vagrant-local/QUICKSTART.md

@@ -3,11 +3,11 @@
 1. Perform `vagrant up` within this directory
 2. Login to Vault vm `vagrant ssh vault` and execute `/vagrant/1_server_setup.sh` as Vagrant user
 3. Login to Client vm `vagrant ssh client` and execute `/vagrant/2_client_setup.sh` as Vagrant user
-4. While logged into Client vm execute `ssh vault.example.com` as Vagrant user. It should successfully login to the Vault node using the SSH CA configuration.  
+4. While logged into Client vm execute `ssh vault.example.com` as Vagrant user. It should successfully login to the Vault node using the SSH CA configuration.
 
 # Notes
 
-Certificate login on the host logs user specific details allowing for service account usage to still be tied to a user.  
+Certificate login on the host logs user specific details allowing for service account usage to still be tied to a user.
 
 ```
 Oct 11 14:31:10 localhost sshd[5334]: Accepted publickey for vagrant from 192.168.50.101 port 51272 ssh2: RSA-CERT ID vault-clientrole-userpass-johnsmith-4b0473525e9941250c988f992b0204d1326885e5e51adca0b1d8debe5e102aad (serial 2914803897344261917) CA RSA 90:b5:59:62:fa:9e:0a:fa:92:75:6c:97:6c:d8:75:c7

identity/ssh_ca/vagrant/README.md → identity/ssh-ca/vagrant-local/README.md

@@ -1,28 +1,28 @@
 #  Vault SSH CA backend
-Manage users' remote access of Linux/Unix servers via SSH access.  
+Manage users' remote access of Linux/Unix servers via SSH access.
 
 ## Reference Material
 
-Typically SSH access to Linux/Unix servers is by private/public keys, and can prove difficult to manage in large environments for several reasons:  
+Typically SSH access to Linux/Unix servers is by private/public keys, and can prove difficult to manage in large environments for several reasons:
 
 - Sprawl of keys means that provisioning or revoking keys can be slow
 - Tracking and auditing use of keys is increasingly difficult as the number of managed systems grows
 - Enforcing short lived credentials is impossible
 
-In 2010, [OpenSSH introduced](http://www.openssh.com/txt/release-5.4) a method where authentication is governed by certificate authority authentication. The same cryptography used by x509 certificates can be leveraged to authenticate users. There are some interesting benefits of this technique:  
+In 2010, [OpenSSH introduced](http://www.openssh.com/txt/release-5.4) a method where authentication is governed by certificate authority authentication. The same cryptography used by x509 certificates can be leveraged to authenticate users. There are some interesting benefits of this technique:
 
-- Individual user public keys do not need to be managed on all servers that a user needs access to. This reduces management overhead.  
+- Individual user public keys do not need to be managed on all servers that a user needs access to. This reduces management overhead.
 - A user's access to servers can be time bound, based on expiration of their signed key. It is now possible to enforce short lived SSH credentials at scale.
 
 This functionality has been documented and used by a number of organizations:
 
-- [Uber SSH Certificate Authority](https://medium.com/uber-security-privacy/introducing-the-uber-ssh-certificate-authority-4f840839c5cc) also released a related [PAM module](https://github.com/uber/pam-ussh) 
+- [Uber SSH Certificate Authority](https://medium.com/uber-security-privacy/introducing-the-uber-ssh-certificate-authority-4f840839c5cc) also released a related [PAM module](https://github.com/uber/pam-ussh)
 - [Facebook's use of OpenSSH CA](https://code.facebook.com/posts/365787980419535/scalable-and-secure-access-with-ssh/)
 - [Netflix' BLESS project](https://github.com/Netflix/bless)
 - [Lyft](https://eng.lyft.com/blessing-your-ssh-at-lyft-a1b38f81629d) made use of the BLESS project and open sourced a [client side integration tool](https://github.com/lyft/python-blessclient)
 - [Red Hat Enterprise Linux documentation](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/s1-ssh-configuration) on OpenSSH CA authentication
 - [SSH protocol documentation](http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/ssh/PROTOCOL.certkeys?rev=HEAD)
-- [Another well documented page related to SSH CA authentication](https://blog.habets.se/2011/07/OpenSSH-certificates.html)  
+- [Another well documented page related to SSH CA authentication](https://blog.habets.se/2011/07/OpenSSH-certificates.html)
 - [Another documented page with SSH CA details](https://www.lorier.net/docs/ssh-ca.html)
 
 ## Estimated Time to Complete

identity/ssh-ca/vagrant-local/Vagrantfile

@@ -0,0 +1,89 @@
+
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+# Vault variable defaults
+vault_version = ENV['VAULT_VERSION'] || "0.10.0"
+vault_ent_url = ENV['VAULT_ENT_URL']
+vault_group = "vault"
+vault_user = "vault"
+vault_comment = "Vault"
+vault_home = "/srv/vault"
+
+$vault_env = <<VAULT_ENV
+sudo cat << EOF > /etc/profile.d/vault.sh
+export VAULT_ADDR="http://192.168.50.100:8200"
+export VAULT_SKIP_VERIFY=true
+EOF
+VAULT_ENV
+
+$vault_run = <<VAULT_RUN
+nohup /usr/local/bin/vault server -dev \
+  -dev-root-token-id="password" \
+  -dev-listen-address="0.0.0.0:8200" 0<&- &>/dev/null &
+VAULT_RUN
+
+$hosts_file = <<HOSTS_FILE
+sudo cat << EOF >> /etc/hosts
+192.168.50.100  vault vault.example.com
+192.168.50.101  client client.example.com
+EOF
+HOSTS_FILE
+
+Vagrant.configure("2") do |config|
+  config.vm.provider "virtualbox" do |vb|
+    vb.customize ["modifyvm", :id, "--memory", "512"]
+    vb.customize ["modifyvm", :id, "--cpus", "1"]
+    vb.customize ["modifyvm", :id, "--cpuexecutioncap", "50"]
+    vb.customize ["modifyvm", :id, "--chipset", "ich9"]
+    vb.customize ["modifyvm", :id, "--ioapic", "on"]
+  end
+  config.vm.define "vault" do |vault|
+    vault.vm.network :private_network, ip: "192.168.50.100"
+    vault.vm.box = "bento/centos-7.3"
+    vault.vm.box_version = "2.3.8"
+    vault.vm.hostname = "vault"
+    vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/base.sh | bash"
+    vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/setup-user.sh | bash",
+      env: {
+        "GROUP" => vault_group,
+        "USER" => vault_user,
+        "COMMENT" => vault_comment,
+        "HOME" => vault_home,
+      }
+    vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/vault/scripts/install-vault.sh | bash",
+      env: {
+        "VERSION" => vault_version,
+        "URL" => vault_ent_url,
+        "USER" => vault_user,
+        "GROUP" => vault_group,
+      }
+    vault.vm.provision "shell", inline: $vault_env
+    vault.vm.provision "shell", inline: $vault_run
+    vault.vm.provision "shell", inline: $hosts_file
+  end
+
+  config.vm.define "client" do |client|
+    client.vm.network :private_network, ip: "192.168.50.101"
+    client.vm.box = "bento/centos-7.3"
+    client.vm.box_version = "2.3.8"
+    client.vm.hostname = "client"
+    vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/base.sh | bash"
+    vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/setup-user.sh | bash",
+      env: {
+        "GROUP" => vault_group,
+        "USER" => vault_user,
+        "COMMENT" => vault_comment,
+        "HOME" => vault_home,
+      }
+    vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/vault/scripts/install-vault.sh | bash",
+      env: {
+        "VERSION" => vault_version,
+        "URL" => vault_ent_url,
+        "USER" => vault_user,
+        "GROUP" => vault_group,
+      }
+    client.vm.provision "shell", inline: $vault_env
+    client.vm.provision "shell", inline: $hosts_file
+  end
+end

operations/aws-kms-unseal/terraform/README.md → operations/aws-kms-unseal/terraform-aws/README.md

@@ -4,7 +4,6 @@ These assets are provided to perform the tasks described in the [Vault Auto-unse
 
 ---
 
-
 ## Demo Steps
 
 ### Setup

operations/aws-kms-unseal/terraform/userdata.tpl → operations/aws-kms-unseal/terraform-aws/userdata.tpl

@@ -63,10 +63,10 @@ logger "User setup complete"
 
 
 
-VAULT_ZIP="vault.zip"		
-VAULT_URL="${vault_url}"	
-curl --silent --output /tmp/$${VAULT_ZIP} $${VAULT_URL}		
-unzip -o /tmp/$${VAULT_ZIP} -d /usr/local/bin/		
+VAULT_ZIP="vault.zip"
+VAULT_URL="${vault_url}"
+curl --silent --output /tmp/$${VAULT_ZIP} $${VAULT_URL}
+unzip -o /tmp/$${VAULT_ZIP} -d /usr/local/bin/
 chmod 0755 /usr/local/bin/vault
 chown vault:vault /usr/local/bin/vault
 mkdir -pm 0755 /etc/vault.d

operations/provision-vault/README.md

@@ -0,0 +1,75 @@
+# Provision Vault
+
+The goal of this guide is to allows users to easily provision a Vault cluster in just a few short commands.
+
+## Reference Material
+
+- [Terraform](https://www.terraform.io/)
+- [Consul](https://www.consul.io/)
+- [Vault](https://www.vaultproject.io/)
+
+## Estimated Time to Complete
+
+5 minutes.
+
+## Personas
+
+### Operator
+
+The operator is responsible for producing the Vault cluster infrastructure and managing day 1 & 2 operations. This includes initial service administration, upgrades, logging/monitoring, and more.
+
+### Developer
+
+The developer will be consuming the Vault services and developing against it. This may be leveraging Vault for Secrets Management, Identity, and Encryption as a Service.
+
+### InfoSec
+
+Infosec will be creating and managing ACLs for Vault, this may include both ACLs and Sentinel policies.
+
+## Challenge
+
+There are many different ways to provision and configure a Vault cluster, making it difficult to get started.
+
+## Solution
+
+Provision a Vault cluster. This will enable users to easily provision a Vault cluster for their desired use case.
+
+### Dev
+
+The [Vault Dev Guides](./dev) are for **educational purposes only**. They're designed to allow you to quickly standup a single instance with Vault running in `-dev` mode in your desired provider. The single node is provisioned into a single public subnet that's completely open, allowing for easy (and insecure) access to the instance. Because Vault is running in `-dev` mode, all data is in-memory and not persisted to disk. If any agent fails or the node restarts, all data will be lost. This is in no way, shape, or form meant for Production use, please use with caution.
+
+### Quick Start
+
+The [Vault Quick Start Guide](./quick-start) provisions a 3 node Vault cluster and 3 node Consul cluster with all agents running in server mode in the provider of your choice.
+
+The Quick Start guide leverages the scripts in the [Guides Configuration Repo](https://github.com/hashicorp/guides-configuration) to do runtime configuration of Vault. Although using `curl bash` at runtime is _not_ best practices, this makes it quick and easy to standup a Vault cluster with no external dependencies like pre-built images. This guide will also forgo setting up TLS/encryption on Vault for the sake of simplicity.
+
+### Best Practices
+
+The [Vault Best Practices Guide](./best-practices) provisions a 3 node Vault cluster with a similar architecture to the [Quick Start](#quick-start) guide in the provider of your choice. The difference is this guide will setup TLS/encryption across Vault and depends on pre-built images rather than runtime configuration. You can find the Packer templates to create these Vault images in the [Guides Configuration Repo](https://github.com/hashicorp/guides-configuration/tree/master/vault).
+
+## Steps
+
+We will now provision the Vault cluster.
+
+### Step 1: Choose your Preferred Guide
+
+`cd` into one of the below guides from the root of the repository and follow the instructions from there.
+
+- [Vagrant dev](./dev/vagrant-local)
+- [AWS dev](./dev/terraform-aws)
+- [AWS quick-start](./quick-start/terraform-aws)
+- [AWS best-practices](./best-practices/terraform-aws)
+
+#### CLI
+
+```sh
+$ cd operations/provision-vault/dev/vagrant-local
+$ cd operations/provision-vault/dev/terraform-aws
+$ cd operations/provision-vault/quick-start/terraform-aws
+$ cd operations/provision-vault/best-practices/terraform-aws
+```
+
+## Next Steps
+
+Now that you've provisioned and configured Vault, start walking through the [Vault Guides](https://www.vaultproject.io/guides/index.html).