diff --git a/.gitignore b/.gitignore index 33e7524a..77c93943 100644 --- a/.gitignore +++ b/.gitignore @@ -8,6 +8,7 @@ # State file in any sub-folder **/terraform.tfstate **/terraform.tfstate.backup +*.tfstate.lock.info # Terraform Variables *.tfvars @@ -16,4 +17,4 @@ *.pem *.DS_Store -*.vagrant \ No newline at end of file +*.vagrant diff --git a/README.md b/README.md index 99d873f4..b4041857 100644 --- a/README.md +++ b/README.md @@ -7,33 +7,36 @@ Vault Logo ----- +---- # Vault-Guides -This repository aims to assist individuals in learning how to install, configure, and administer HashiCorp Vault. -## provision -This area will contain instructions to provision Vault and Consul as a first step to start using these tools. +This repository provides the technical content to support the [Vault Guides](https://www.vaultproject.io/guides/index.html) site. + +## Operations + +This area will contain instructions to operatioanlize Vault. + +- [Provision a Dev Vault Cluster locally with Vagrant](operations/provision-vault/dev/vagrant-local) +- [Provision a Dev Vault Cluster on AWS with Terraform](operations/provision-vault/dev/terraform-aws) +- [Provision a Quick Start Vault & Consul Cluster on AWS with Terraform](operations/provision-vault/quick-start/terraform-aws) +- [Provision a Best Practices Vault & Consul Cluster on AWS with Terraform](operations/provision-vault/best-practices/terraform-aws) + +## Secrets -These may include use cases installing Vault in cloud services via Terraform, or within virtual environments using Vagrant, or running Vault in a local development mode. +This directory contains example use cases involving [secrets management](https://www.vaultproject.io/docs/secrets/index.html). -## encryption -This directory contains example usage of the [Vault Transit backend](https://www.vaultproject.io/docs/secrets/transit/index.html). Also referred to as 'Encryption as a Service' as it allows organizations to provide a centrally managed encryption service for their infrastructure. +## Identity -## secrets -This directory contains example use cases involving [secrets management](https://www.vaultproject.io/docs/secrets/index.html). Secure secret storage of static secrets and sensitive information. Implementation of Dynamic Secrets. +This directory contains example use cases involving [identity](https://www.vaultproject.io/docs/auth/index.html). -## identity -This directory contains examples of privilege access management, identity, access control. +## Encryption -## governance -Guides related to governance within Vault stored here. Sentinel guides only available for Vault Enterprise Premium. +This directory contains example use cases involving [encryption as a service](https://www.vaultproject.io/docs/secrets/transit/index.html). -## shared -This directory contains common scripts and configuration files used to provision environments used for the guides in this repository. +## Assets -## assets This directory contains graphics and other material for the repository. +## Contributing -### Contributing -We welcome contributions and feedback! For guide submissions, please see [the contributions guide](CONTRIBUTING.md) \ No newline at end of file +We welcome contributions and feedback! For guide submissions, please see [the contributions guide](CONTRIBUTING.md) diff --git a/governance/README.md b/governance/README.md index 1bd24c51..1e42e9f2 100644 --- a/governance/README.md +++ b/governance/README.md @@ -1 +1 @@ -Guides related to governance within Vault stored here. Sentinel guides only available for Vault Enterprise Premium. \ No newline at end of file +Guides related to governance within Vault stored here. Sentinel guides only available for Vault Enterprise Premium. diff --git a/identity/ssh_ca/vagrant/.gitignore b/identity/ssh-ca/vagrant-local/.gitignore similarity index 100% rename from identity/ssh_ca/vagrant/.gitignore rename to identity/ssh-ca/vagrant-local/.gitignore diff --git a/identity/ssh_ca/vagrant/1_server_setup.sh b/identity/ssh-ca/vagrant-local/1_server_setup.sh similarity index 100% rename from identity/ssh_ca/vagrant/1_server_setup.sh rename to identity/ssh-ca/vagrant-local/1_server_setup.sh diff --git a/identity/ssh_ca/vagrant/2_client_setup.sh b/identity/ssh-ca/vagrant-local/2_client_setup.sh similarity index 100% rename from identity/ssh_ca/vagrant/2_client_setup.sh rename to identity/ssh-ca/vagrant-local/2_client_setup.sh diff --git a/identity/ssh_ca/vagrant/QUICKSTART.md b/identity/ssh-ca/vagrant-local/QUICKSTART.md similarity index 92% rename from identity/ssh_ca/vagrant/QUICKSTART.md rename to identity/ssh-ca/vagrant-local/QUICKSTART.md index 596ea5b2..a0d18083 100644 --- a/identity/ssh_ca/vagrant/QUICKSTART.md +++ b/identity/ssh-ca/vagrant-local/QUICKSTART.md @@ -3,11 +3,11 @@ 1. Perform `vagrant up` within this directory 2. Login to Vault vm `vagrant ssh vault` and execute `/vagrant/1_server_setup.sh` as Vagrant user 3. Login to Client vm `vagrant ssh client` and execute `/vagrant/2_client_setup.sh` as Vagrant user -4. While logged into Client vm execute `ssh vault.example.com` as Vagrant user. It should successfully login to the Vault node using the SSH CA configuration. +4. While logged into Client vm execute `ssh vault.example.com` as Vagrant user. It should successfully login to the Vault node using the SSH CA configuration. # Notes -Certificate login on the host logs user specific details allowing for service account usage to still be tied to a user. +Certificate login on the host logs user specific details allowing for service account usage to still be tied to a user. ``` Oct 11 14:31:10 localhost sshd[5334]: Accepted publickey for vagrant from 192.168.50.101 port 51272 ssh2: RSA-CERT ID vault-clientrole-userpass-johnsmith-4b0473525e9941250c988f992b0204d1326885e5e51adca0b1d8debe5e102aad (serial 2914803897344261917) CA RSA 90:b5:59:62:fa:9e:0a:fa:92:75:6c:97:6c:d8:75:c7 diff --git a/identity/ssh_ca/vagrant/README.md b/identity/ssh-ca/vagrant-local/README.md similarity index 94% rename from identity/ssh_ca/vagrant/README.md rename to identity/ssh-ca/vagrant-local/README.md index 35ae4958..caf9663f 100644 --- a/identity/ssh_ca/vagrant/README.md +++ b/identity/ssh-ca/vagrant-local/README.md @@ -1,28 +1,28 @@ # Vault SSH CA backend -Manage users' remote access of Linux/Unix servers via SSH access. +Manage users' remote access of Linux/Unix servers via SSH access. ## Reference Material -Typically SSH access to Linux/Unix servers is by private/public keys, and can prove difficult to manage in large environments for several reasons: +Typically SSH access to Linux/Unix servers is by private/public keys, and can prove difficult to manage in large environments for several reasons: - Sprawl of keys means that provisioning or revoking keys can be slow - Tracking and auditing use of keys is increasingly difficult as the number of managed systems grows - Enforcing short lived credentials is impossible -In 2010, [OpenSSH introduced](http://www.openssh.com/txt/release-5.4) a method where authentication is governed by certificate authority authentication. The same cryptography used by x509 certificates can be leveraged to authenticate users. There are some interesting benefits of this technique: +In 2010, [OpenSSH introduced](http://www.openssh.com/txt/release-5.4) a method where authentication is governed by certificate authority authentication. The same cryptography used by x509 certificates can be leveraged to authenticate users. There are some interesting benefits of this technique: -- Individual user public keys do not need to be managed on all servers that a user needs access to. This reduces management overhead. +- Individual user public keys do not need to be managed on all servers that a user needs access to. This reduces management overhead. - A user's access to servers can be time bound, based on expiration of their signed key. It is now possible to enforce short lived SSH credentials at scale. This functionality has been documented and used by a number of organizations: -- [Uber SSH Certificate Authority](https://medium.com/uber-security-privacy/introducing-the-uber-ssh-certificate-authority-4f840839c5cc) also released a related [PAM module](https://github.com/uber/pam-ussh) +- [Uber SSH Certificate Authority](https://medium.com/uber-security-privacy/introducing-the-uber-ssh-certificate-authority-4f840839c5cc) also released a related [PAM module](https://github.com/uber/pam-ussh) - [Facebook's use of OpenSSH CA](https://code.facebook.com/posts/365787980419535/scalable-and-secure-access-with-ssh/) - [Netflix' BLESS project](https://github.com/Netflix/bless) - [Lyft](https://eng.lyft.com/blessing-your-ssh-at-lyft-a1b38f81629d) made use of the BLESS project and open sourced a [client side integration tool](https://github.com/lyft/python-blessclient) - [Red Hat Enterprise Linux documentation](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/s1-ssh-configuration) on OpenSSH CA authentication - [SSH protocol documentation](http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/ssh/PROTOCOL.certkeys?rev=HEAD) -- [Another well documented page related to SSH CA authentication](https://blog.habets.se/2011/07/OpenSSH-certificates.html) +- [Another well documented page related to SSH CA authentication](https://blog.habets.se/2011/07/OpenSSH-certificates.html) - [Another documented page with SSH CA details](https://www.lorier.net/docs/ssh-ca.html) ## Estimated Time to Complete diff --git a/identity/ssh-ca/vagrant-local/Vagrantfile b/identity/ssh-ca/vagrant-local/Vagrantfile new file mode 100644 index 00000000..4df4466e --- /dev/null +++ b/identity/ssh-ca/vagrant-local/Vagrantfile @@ -0,0 +1,89 @@ + +# -*- mode: ruby -*- +# vi: set ft=ruby : + +# Vault variable defaults +vault_version = ENV['VAULT_VERSION'] || "0.10.0" +vault_ent_url = ENV['VAULT_ENT_URL'] +vault_group = "vault" +vault_user = "vault" +vault_comment = "Vault" +vault_home = "/srv/vault" + +$vault_env = < /etc/profile.d/vault.sh +export VAULT_ADDR="http://192.168.50.100:8200" +export VAULT_SKIP_VERIFY=true +EOF +VAULT_ENV + +$vault_run = </dev/null & +VAULT_RUN + +$hosts_file = <> /etc/hosts +192.168.50.100 vault vault.example.com +192.168.50.101 client client.example.com +EOF +HOSTS_FILE + +Vagrant.configure("2") do |config| + config.vm.provider "virtualbox" do |vb| + vb.customize ["modifyvm", :id, "--memory", "512"] + vb.customize ["modifyvm", :id, "--cpus", "1"] + vb.customize ["modifyvm", :id, "--cpuexecutioncap", "50"] + vb.customize ["modifyvm", :id, "--chipset", "ich9"] + vb.customize ["modifyvm", :id, "--ioapic", "on"] + end + config.vm.define "vault" do |vault| + vault.vm.network :private_network, ip: "192.168.50.100" + vault.vm.box = "bento/centos-7.3" + vault.vm.box_version = "2.3.8" + vault.vm.hostname = "vault" + vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/base.sh | bash" + vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/setup-user.sh | bash", + env: { + "GROUP" => vault_group, + "USER" => vault_user, + "COMMENT" => vault_comment, + "HOME" => vault_home, + } + vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/vault/scripts/install-vault.sh | bash", + env: { + "VERSION" => vault_version, + "URL" => vault_ent_url, + "USER" => vault_user, + "GROUP" => vault_group, + } + vault.vm.provision "shell", inline: $vault_env + vault.vm.provision "shell", inline: $vault_run + vault.vm.provision "shell", inline: $hosts_file + end + + config.vm.define "client" do |client| + client.vm.network :private_network, ip: "192.168.50.101" + client.vm.box = "bento/centos-7.3" + client.vm.box_version = "2.3.8" + client.vm.hostname = "client" + vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/base.sh | bash" + vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/setup-user.sh | bash", + env: { + "GROUP" => vault_group, + "USER" => vault_user, + "COMMENT" => vault_comment, + "HOME" => vault_home, + } + vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/vault/scripts/install-vault.sh | bash", + env: { + "VERSION" => vault_version, + "URL" => vault_ent_url, + "USER" => vault_user, + "GROUP" => vault_group, + } + client.vm.provision "shell", inline: $vault_env + client.vm.provision "shell", inline: $hosts_file + end +end diff --git a/identity/ssh_mfa/vagrant/.gitignore b/identity/ssh-mfa/vagrant/.gitignore similarity index 100% rename from identity/ssh_mfa/vagrant/.gitignore rename to identity/ssh-mfa/vagrant/.gitignore diff --git a/identity/ssh_mfa/vagrant/README.md b/identity/ssh-mfa/vagrant/README.md similarity index 100% rename from identity/ssh_mfa/vagrant/README.md rename to identity/ssh-mfa/vagrant/README.md diff --git a/identity/ssh_mfa/vagrant/Vagrantfile b/identity/ssh-mfa/vagrant/Vagrantfile similarity index 100% rename from identity/ssh_mfa/vagrant/Vagrantfile rename to identity/ssh-mfa/vagrant/Vagrantfile diff --git a/identity/ssh_mfa/vagrant/config/config.hcl b/identity/ssh-mfa/vagrant/config/config.hcl similarity index 100% rename from identity/ssh_mfa/vagrant/config/config.hcl rename to identity/ssh-mfa/vagrant/config/config.hcl diff --git a/identity/ssh_mfa/vagrant/config/sshd b/identity/ssh-mfa/vagrant/config/sshd similarity index 100% rename from identity/ssh_mfa/vagrant/config/sshd rename to identity/ssh-mfa/vagrant/config/sshd diff --git a/identity/ssh_mfa/vagrant/config/sshd_config b/identity/ssh-mfa/vagrant/config/sshd_config similarity index 100% rename from identity/ssh_mfa/vagrant/config/sshd_config rename to identity/ssh-mfa/vagrant/config/sshd_config diff --git a/identity/ssh_mfa/vagrant/scripts/server_endpoint_otp.sh b/identity/ssh-mfa/vagrant/scripts/server_endpoint_otp.sh similarity index 100% rename from identity/ssh_mfa/vagrant/scripts/server_endpoint_otp.sh rename to identity/ssh-mfa/vagrant/scripts/server_endpoint_otp.sh diff --git a/identity/ssh_mfa/vagrant/scripts/server_endpoint_ssh_ca.sh b/identity/ssh-mfa/vagrant/scripts/server_endpoint_ssh_ca.sh similarity index 100% rename from identity/ssh_mfa/vagrant/scripts/server_endpoint_ssh_ca.sh rename to identity/ssh-mfa/vagrant/scripts/server_endpoint_ssh_ca.sh diff --git a/identity/ssh_mfa/vagrant/scripts/vault_aws.sh b/identity/ssh-mfa/vagrant/scripts/vault_aws.sh similarity index 100% rename from identity/ssh_mfa/vagrant/scripts/vault_aws.sh rename to identity/ssh-mfa/vagrant/scripts/vault_aws.sh diff --git a/identity/ssh_mfa/vagrant/scripts/vault_config.sh b/identity/ssh-mfa/vagrant/scripts/vault_config.sh similarity index 100% rename from identity/ssh_mfa/vagrant/scripts/vault_config.sh rename to identity/ssh-mfa/vagrant/scripts/vault_config.sh diff --git a/identity/ssh_mfa/vagrant/scripts/vault_okta.sh b/identity/ssh-mfa/vagrant/scripts/vault_okta.sh similarity index 100% rename from identity/ssh_mfa/vagrant/scripts/vault_okta.sh rename to identity/ssh-mfa/vagrant/scripts/vault_okta.sh diff --git a/identity/ssh_mfa/vagrant/scripts/vault_ssh.sh b/identity/ssh-mfa/vagrant/scripts/vault_ssh.sh similarity index 100% rename from identity/ssh_mfa/vagrant/scripts/vault_ssh.sh rename to identity/ssh-mfa/vagrant/scripts/vault_ssh.sh diff --git a/identity/ssh_mfa/vagrant/vars.yaml.example b/identity/ssh-mfa/vagrant/vars.yaml.example similarity index 100% rename from identity/ssh_mfa/vagrant/vars.yaml.example rename to identity/ssh-mfa/vagrant/vars.yaml.example diff --git a/identity/ssh_otp/vagrant/README.md b/identity/ssh-otp/vagrant/README.md similarity index 100% rename from identity/ssh_otp/vagrant/README.md rename to identity/ssh-otp/vagrant/README.md diff --git a/identity/ssh_otp/vagrant/Vagrantfile b/identity/ssh-otp/vagrant/Vagrantfile similarity index 100% rename from identity/ssh_otp/vagrant/Vagrantfile rename to identity/ssh-otp/vagrant/Vagrantfile diff --git a/identity/ssh_otp/vagrant/config/config.hcl b/identity/ssh-otp/vagrant/config/config.hcl similarity index 100% rename from identity/ssh_otp/vagrant/config/config.hcl rename to identity/ssh-otp/vagrant/config/config.hcl diff --git a/identity/ssh_otp/vagrant/config/sshd b/identity/ssh-otp/vagrant/config/sshd similarity index 100% rename from identity/ssh_otp/vagrant/config/sshd rename to identity/ssh-otp/vagrant/config/sshd diff --git a/identity/ssh_otp/vagrant/config/sshd_config b/identity/ssh-otp/vagrant/config/sshd_config similarity index 100% rename from identity/ssh_otp/vagrant/config/sshd_config rename to identity/ssh-otp/vagrant/config/sshd_config diff --git a/identity/ssh_otp/vagrant/scripts/server_endpoint_otp.sh b/identity/ssh-otp/vagrant/scripts/server_endpoint_otp.sh similarity index 100% rename from identity/ssh_otp/vagrant/scripts/server_endpoint_otp.sh rename to identity/ssh-otp/vagrant/scripts/server_endpoint_otp.sh diff --git a/identity/ssh_otp/vagrant/scripts/server_endpoint_ssh_ca.sh b/identity/ssh-otp/vagrant/scripts/server_endpoint_ssh_ca.sh similarity index 100% rename from identity/ssh_otp/vagrant/scripts/server_endpoint_ssh_ca.sh rename to identity/ssh-otp/vagrant/scripts/server_endpoint_ssh_ca.sh diff --git a/identity/ssh_otp/vagrant/scripts/vault_config.sh b/identity/ssh-otp/vagrant/scripts/vault_config.sh similarity index 100% rename from identity/ssh_otp/vagrant/scripts/vault_config.sh rename to identity/ssh-otp/vagrant/scripts/vault_config.sh diff --git a/identity/ssh_otp/vagrant/scripts/vault_ssh.sh b/identity/ssh-otp/vagrant/scripts/vault_ssh.sh similarity index 100% rename from identity/ssh_otp/vagrant/scripts/vault_ssh.sh rename to identity/ssh-otp/vagrant/scripts/vault_ssh.sh diff --git a/identity/ssh_ca/vagrant/Vagrantfile b/identity/ssh_ca/vagrant/Vagrantfile deleted file mode 100644 index b84f97b3..00000000 --- a/identity/ssh_ca/vagrant/Vagrantfile +++ /dev/null @@ -1,64 +0,0 @@ - -# -*- mode: ruby -*- -# vi: set ft=ruby : - -# vault variable defaults -vault_version = ENV['VAULT_VERSION'] || "0.9.5" -vault_download_url = ENV['VAULT_ENT_URL'] || "https://releases.hashicorp.com/vault/#{vault_version}/vault_#{vault_version}_linux_amd64.zip" - -$vault_env = < /etc/profile.d/vault.sh -export VAULT_ADDR="http://192.168.50.100:8200" -export VAULT_SKIP_VERIFY=true -EOF -VAULT_ENV - -$vault_run = </dev/null & -VAULT_RUN - -$hosts_file = <> /etc/hosts -192.168.50.100 vault vault.example.com -192.168.50.101 client client.example.com -EOF -HOSTS_FILE - -Vagrant.configure("2") do |config| - config.vm.provider "virtualbox" do |vb| - vb.customize ["modifyvm", :id, "--memory", "512"] - vb.customize ["modifyvm", :id, "--cpus", "1"] - vb.customize ["modifyvm", :id, "--cpuexecutioncap", "50"] - vb.customize ["modifyvm", :id, "--chipset", "ich9"] - vb.customize ["modifyvm", :id, "--ioapic", "on"] - end - config.vm.define "vault" do |vault| - vault.vm.network :private_network, ip: "192.168.50.100" - vault.vm.box = "bento/centos-7.3" - vault.vm.box_version = "2.3.8" - vault.vm.hostname = "vault" - vault.vm.provision "shell", path: "../../../shared/scripts/base.sh" - vault.vm.provision "shell", path: "../../../shared/scripts/setup-user.sh", args: "vault" - vault.vm.provision "shell", path: "../../../shared/vault/scripts/install-vault.sh", - env: { "VERSION" => vault_version,"URL" => vault_download_url } - vault.vm.provision "shell", inline: $vault_env - vault.vm.provision "shell", inline: $vault_run - vault.vm.provision "shell", inline: $hosts_file - end - - config.vm.define "client" do |client| - client.vm.network :private_network, ip: "192.168.50.101" - client.vm.box = "bento/centos-7.3" - client.vm.box_version = "2.3.8" - client.vm.hostname = "client" - client.vm.provision "shell", path: "../../../shared/scripts/base.sh" - client.vm.provision "shell", path: "../../../shared/scripts/setup-user.sh", args: "vault" - client.vm.provision "shell", path: "../../../shared/vault/scripts/install-vault.sh", - env: { "VERSION" => vault_version,"URL" => vault_download_url } - client.vm.provision "shell", inline: $vault_env - client.vm.provision "shell", inline: $hosts_file - end -end - diff --git a/operations/README.md b/operations/README.md deleted file mode 100644 index 61105e0e..00000000 --- a/operations/README.md +++ /dev/null @@ -1,4 +0,0 @@ -Guides and examples of Vault operations will be stored here -- Process and procedural -- High Availability -- Performance and DR modes of Replication [(Vault Enterprise features)](https://www.vaultproject.io/docs/enterprise/replication/index.html) diff --git a/operations/aws-kms-unseal/terraform/.gitignore b/operations/aws-kms-unseal/terraform-aws/.gitignore similarity index 100% rename from operations/aws-kms-unseal/terraform/.gitignore rename to operations/aws-kms-unseal/terraform-aws/.gitignore diff --git a/operations/aws-kms-unseal/terraform/README.md b/operations/aws-kms-unseal/terraform-aws/README.md similarity index 99% rename from operations/aws-kms-unseal/terraform/README.md rename to operations/aws-kms-unseal/terraform-aws/README.md index a1238cd6..64613fe5 100644 --- a/operations/aws-kms-unseal/terraform/README.md +++ b/operations/aws-kms-unseal/terraform-aws/README.md @@ -4,7 +4,6 @@ These assets are provided to perform the tasks described in the [Vault Auto-unse --- - ## Demo Steps ### Setup diff --git a/operations/aws-kms-unseal/terraform/instance-profile.tf b/operations/aws-kms-unseal/terraform-aws/instance-profile.tf similarity index 100% rename from operations/aws-kms-unseal/terraform/instance-profile.tf rename to operations/aws-kms-unseal/terraform-aws/instance-profile.tf diff --git a/operations/aws-kms-unseal/terraform/instance.tf b/operations/aws-kms-unseal/terraform-aws/instance.tf similarity index 100% rename from operations/aws-kms-unseal/terraform/instance.tf rename to operations/aws-kms-unseal/terraform-aws/instance.tf diff --git a/operations/aws-kms-unseal/terraform/main.tf b/operations/aws-kms-unseal/terraform-aws/main.tf similarity index 100% rename from operations/aws-kms-unseal/terraform/main.tf rename to operations/aws-kms-unseal/terraform-aws/main.tf diff --git a/operations/aws-kms-unseal/terraform/ssh-key.tf b/operations/aws-kms-unseal/terraform-aws/ssh-key.tf similarity index 100% rename from operations/aws-kms-unseal/terraform/ssh-key.tf rename to operations/aws-kms-unseal/terraform-aws/ssh-key.tf diff --git a/operations/aws-kms-unseal/terraform/terraform.tfvars.example b/operations/aws-kms-unseal/terraform-aws/terraform.tfvars.example similarity index 100% rename from operations/aws-kms-unseal/terraform/terraform.tfvars.example rename to operations/aws-kms-unseal/terraform-aws/terraform.tfvars.example diff --git a/operations/aws-kms-unseal/terraform/userdata.tpl b/operations/aws-kms-unseal/terraform-aws/userdata.tpl similarity index 93% rename from operations/aws-kms-unseal/terraform/userdata.tpl rename to operations/aws-kms-unseal/terraform-aws/userdata.tpl index 5f051371..9f1fd7d4 100644 --- a/operations/aws-kms-unseal/terraform/userdata.tpl +++ b/operations/aws-kms-unseal/terraform-aws/userdata.tpl @@ -63,10 +63,10 @@ logger "User setup complete" -VAULT_ZIP="vault.zip" -VAULT_URL="${vault_url}" -curl --silent --output /tmp/$${VAULT_ZIP} $${VAULT_URL} -unzip -o /tmp/$${VAULT_ZIP} -d /usr/local/bin/ +VAULT_ZIP="vault.zip" +VAULT_URL="${vault_url}" +curl --silent --output /tmp/$${VAULT_ZIP} $${VAULT_URL} +unzip -o /tmp/$${VAULT_ZIP} -d /usr/local/bin/ chmod 0755 /usr/local/bin/vault chown vault:vault /usr/local/bin/vault mkdir -pm 0755 /etc/vault.d diff --git a/operations/aws-kms-unseal/terraform/variables.tf b/operations/aws-kms-unseal/terraform-aws/variables.tf similarity index 100% rename from operations/aws-kms-unseal/terraform/variables.tf rename to operations/aws-kms-unseal/terraform-aws/variables.tf diff --git a/operations/provision-vault/README.md b/operations/provision-vault/README.md new file mode 100644 index 00000000..e6e04881 --- /dev/null +++ b/operations/provision-vault/README.md @@ -0,0 +1,75 @@ +# Provision Vault + +The goal of this guide is to allows users to easily provision a Vault cluster in just a few short commands. + +## Reference Material + +- [Terraform](https://www.terraform.io/) +- [Consul](https://www.consul.io/) +- [Vault](https://www.vaultproject.io/) + +## Estimated Time to Complete + +5 minutes. + +## Personas + +### Operator + +The operator is responsible for producing the Vault cluster infrastructure and managing day 1 & 2 operations. This includes initial service administration, upgrades, logging/monitoring, and more. + +### Developer + +The developer will be consuming the Vault services and developing against it. This may be leveraging Vault for Secrets Management, Identity, and Encryption as a Service. + +### InfoSec + +Infosec will be creating and managing ACLs for Vault, this may include both ACLs and Sentinel policies. + +## Challenge + +There are many different ways to provision and configure a Vault cluster, making it difficult to get started. + +## Solution + +Provision a Vault cluster. This will enable users to easily provision a Vault cluster for their desired use case. + +### Dev + +The [Vault Dev Guides](./dev) are for **educational purposes only**. They're designed to allow you to quickly standup a single instance with Vault running in `-dev` mode in your desired provider. The single node is provisioned into a single public subnet that's completely open, allowing for easy (and insecure) access to the instance. Because Vault is running in `-dev` mode, all data is in-memory and not persisted to disk. If any agent fails or the node restarts, all data will be lost. This is in no way, shape, or form meant for Production use, please use with caution. + +### Quick Start + +The [Vault Quick Start Guide](./quick-start) provisions a 3 node Vault cluster and 3 node Consul cluster with all agents running in server mode in the provider of your choice. + +The Quick Start guide leverages the scripts in the [Guides Configuration Repo](https://github.com/hashicorp/guides-configuration) to do runtime configuration of Vault. Although using `curl bash` at runtime is _not_ best practices, this makes it quick and easy to standup a Vault cluster with no external dependencies like pre-built images. This guide will also forgo setting up TLS/encryption on Vault for the sake of simplicity. + +### Best Practices + +The [Vault Best Practices Guide](./best-practices) provisions a 3 node Vault cluster with a similar architecture to the [Quick Start](#quick-start) guide in the provider of your choice. The difference is this guide will setup TLS/encryption across Vault and depends on pre-built images rather than runtime configuration. You can find the Packer templates to create these Vault images in the [Guides Configuration Repo](https://github.com/hashicorp/guides-configuration/tree/master/vault). + +## Steps + +We will now provision the Vault cluster. + +### Step 1: Choose your Preferred Guide + +`cd` into one of the below guides from the root of the repository and follow the instructions from there. + +- [Vagrant dev](./dev/vagrant-local) +- [AWS dev](./dev/terraform-aws) +- [AWS quick-start](./quick-start/terraform-aws) +- [AWS best-practices](./best-practices/terraform-aws) + +#### CLI + +```sh +$ cd operations/provision-vault/dev/vagrant-local +$ cd operations/provision-vault/dev/terraform-aws +$ cd operations/provision-vault/quick-start/terraform-aws +$ cd operations/provision-vault/best-practices/terraform-aws +``` + +## Next Steps + +Now that you've provisioned and configured Vault, start walking through the [Vault Guides](https://www.vaultproject.io/guides/index.html). diff --git a/operations/provision-vault/best-practices/terraform-aws/README.md b/operations/provision-vault/best-practices/terraform-aws/README.md new file mode 100644 index 00000000..680f78bf --- /dev/null +++ b/operations/provision-vault/best-practices/terraform-aws/README.md @@ -0,0 +1,92 @@ +# Provision a Best Practices Vault Cluster in AWS + +The goal of this guide is to allows users to easily provision a best practices Vault & Consul cluster in just a few commands. + +## Reference Material + +- [Terraform Getting Started](https://www.terraform.io/intro/getting-started/install.html) +- [Terraform Docs](https://www.terraform.io/docs/index.html) +- [Consul Getting Started](https://www.consul.io/intro/getting-started/install.html) +- [Consul Docs](https://www.consul.io/docs/index.html) +- [Vault Getting Started](https://www.vaultproject.io/intro/getting-started/install.html) +- [Vault Docs](https://www.vaultproject.io/docs/index.html) + +## Estimated Time to Complete + +5 minutes. + +## Challenge + +There are many different ways to provision and configure an easily accessible best practices Vault & Consul cluster, making it difficult to get started. + +## Solution + +Provision a best practices Vault & Consul cluster in a private network with a bastion host. + +The AWS Best Practices Vault guide provisions a 3 node Vault and 3 node Consul cluster with a similar architecture to the [Quick Start](../quick-start) guide. The difference is this guide will setup TLS/encryption across Vault & Consul and depends on pre-built images rather than runtime configuration. You can find the Packer templates to create the [Consul image](https://github.com/hashicorp/guides-configuration/blob/master/consul/consul-aws.json) and [Vault image](https://github.com/hashicorp/guides-configuration/blob/master/vault/vault-aws.json) in the [Guides Configuration Repo](https://github.com/hashicorp/guides-configuration/). + +## Prerequisites + +- [Download Terraform](https://www.terraform.io/downloads.html) + +## Steps + +We will now provision the best practices Vault cluster. + +### Step 1: Initialize + +Initialize Terraform - download providers and modules. + +#### CLI + +[`terraform init` Command](https://www.terraform.io/docs/commands/init.html) + +##### Request + +```sh +$ terraform init +``` + +##### Response +``` +``` + +### Step 2: Plan + +Run a `terraform plan` to ensure Terraform will provision what you expect. + +#### CLI + +[`terraform plan` Command](https://www.terraform.io/docs/commands/plan.html) + +##### Request + +```sh +$ terraform plan +``` + +##### Response +``` +``` + +### Step 3: Apply + +Run a `terraform apply` to provision the HashiStack. One provisioned, view the `zREADME` instructions output from Terraform for next steps. + +#### CLI + +[`terraform apply` command](https://www.terraform.io/docs/commands/apply.html) + +##### Request + +```sh +$ terraform apply +``` + +##### Response +``` +``` + +## Next Steps + +Now that you've provisioned and configured a best practices Vault & Consul cluster, start walking through the [Consul Guides](https://www.consul.io/docs/guides/index.html). diff --git a/operations/provision-vault/best-practices/terraform-aws/main.tf b/operations/provision-vault/best-practices/terraform-aws/main.tf new file mode 100644 index 00000000..bb0594bc --- /dev/null +++ b/operations/provision-vault/best-practices/terraform-aws/main.tf @@ -0,0 +1,200 @@ +module "ssh_keypair_aws_override" { + source = "github.com/hashicorp-modules/ssh-keypair-aws" + + name = "${var.name}-override" +} + +module "consul_auto_join_instance_role" { + source = "github.com/hashicorp-modules/consul-auto-join-instance-role-aws" + + name = "${var.name}" +} + +resource "random_id" "consul_encrypt" { + byte_length = 16 +} + +module "root_tls_self_signed_ca" { + source = "github.com/hashicorp-modules/tls-self-signed-cert" + + name = "${var.name}-root" + ca_common_name = "${var.common_name}" + organization_name = "${var.organization_name}" + common_name = "${var.common_name}" + download_certs = "${var.download_certs}" + + validity_period_hours = "8760" + + ca_allowed_uses = [ + "cert_signing", + "key_encipherment", + "digital_signature", + "server_auth", + "client_auth", + ] +} + +module "leaf_tls_self_signed_cert" { + source = "github.com/hashicorp-modules/tls-self-signed-cert" + + name = "${var.name}-leaf" + organization_name = "${var.organization_name}" + common_name = "${var.common_name}" + ca_override = true + ca_key_override = "${module.root_tls_self_signed_ca.ca_private_key_pem}" + ca_cert_override = "${module.root_tls_self_signed_ca.ca_cert_pem}" + download_certs = "${var.download_certs}" + + validity_period_hours = "8760" + + dns_names = [ + "localhost", + "*.node.consul", + "*.service.consul", + "server.dc1.consul", + "*.dc1.consul", + "server.${var.name}.consul", + "*.${var.name}.consul", + ] + + ip_addresses = [ + "0.0.0.0", + "127.0.0.1", + ] + + allowed_uses = [ + "key_encipherment", + "digital_signature", + "server_auth", + "client_auth", + ] +} + +data "template_file" "bastion_user_data" { + template = "${file("${path.module}/../../templates/best-practices-bastion-systemd.sh.tpl")}" + + vars = { + name = "${var.name}" + provider = "${var.provider}" + local_ip_url = "${var.local_ip_url}" + ca_crt = "${module.root_tls_self_signed_ca.ca_cert_pem}" + leaf_crt = "${module.leaf_tls_self_signed_cert.leaf_cert_pem}" + leaf_key = "${module.leaf_tls_self_signed_cert.leaf_private_key_pem}" + consul_encrypt = "${random_id.consul_encrypt.b64_std}" + consul_override = "${var.consul_client_config_override != "" ? true : false}" + consul_config = "${var.consul_client_config_override}" + } +} + +module "network_aws" { + source = "github.com/hashicorp-modules/network-aws" + + name = "${var.name}" + vpc_cidr = "${var.vpc_cidr}" + vpc_cidrs_public = "${var.vpc_cidrs_public}" + nat_count = "${var.nat_count}" + vpc_cidrs_private = "${var.vpc_cidrs_private}" + release_version = "${var.bastion_release}" + consul_version = "${var.bastion_consul_version}" + vault_version = "${var.bastion_vault_version}" + os = "${var.bastion_os}" + os_version = "${var.bastion_os_version}" + bastion_count = "${var.bastion_servers}" + instance_profile = "${module.consul_auto_join_instance_role.instance_profile_id}" # Override instance_profile + instance_type = "${var.bastion_instance}" + image_id = "${var.bastion_image_id}" + user_data = "${data.template_file.bastion_user_data.rendered}" # Override user_data + ssh_key_name = "${module.ssh_keypair_aws_override.name}" + ssh_key_override = true + private_key_file = "${module.ssh_keypair_aws_override.private_key_filename}" + tags = "${var.network_tags}" +} + +data "template_file" "consul_user_data" { + template = "${file("${path.module}/../../templates/best-practices-consul-systemd.sh.tpl")}" + + vars = { + name = "${var.name}" + provider = "${var.provider}" + local_ip_url = "${var.local_ip_url}" + ca_crt = "${module.root_tls_self_signed_ca.ca_cert_pem}" + leaf_crt = "${module.leaf_tls_self_signed_cert.leaf_cert_pem}" + leaf_key = "${module.leaf_tls_self_signed_cert.leaf_private_key_pem}" + consul_bootstrap = "${length(module.network_aws.subnet_private_ids)}" + consul_encrypt = "${random_id.consul_encrypt.b64_std}" + consul_override = "${var.consul_client_config_override != "" ? true : false}" + consul_config = "${var.consul_client_config_override}" + } +} + +module "consul_aws" { + source = "github.com/hashicorp-modules/consul-aws" + + name = "${var.name}" # Must match network_aws module name for Consul Auto Join to work + vpc_id = "${module.network_aws.vpc_id}" + vpc_cidr = "${module.network_aws.vpc_cidr}" + subnet_ids = "${split(",", var.consul_public ? join(",", module.network_aws.subnet_public_ids) : join(",", module.network_aws.subnet_private_ids))}" + release_version = "${var.consul_release}" + consul_version = "${var.consul_version}" + os = "${var.consul_os}" + os_version = "${var.consul_os_version}" + count = "${var.consul_servers}" + instance_profile = "${module.consul_auto_join_instance_role.instance_profile_id}" # Override instance_profile + instance_type = "${var.consul_instance}" + image_id = "${var.consul_image_id}" + public = "${var.consul_public}" + use_lb_cert = true + lb_cert = "${module.leaf_tls_self_signed_cert.leaf_cert_pem}" + lb_private_key = "${module.leaf_tls_self_signed_cert.leaf_private_key_pem}" + lb_cert_chain = "${module.root_tls_self_signed_ca.ca_cert_pem}" + user_data = "${data.template_file.consul_user_data.rendered}" # Custom user_data + ssh_key_name = "${module.ssh_keypair_aws_override.name}" + tags = "${var.consul_tags}" + tags_list = "${var.consul_tags_list}" +} + +data "template_file" "vault_user_data" { + template = "${file("${path.module}/../../templates/best-practices-vault-systemd.sh.tpl")}" + + vars = { + name = "${var.name}" + provider = "${var.provider}" + local_ip_url = "${var.local_ip_url}" + ca_crt = "${module.root_tls_self_signed_ca.ca_cert_pem}" + leaf_crt = "${module.leaf_tls_self_signed_cert.leaf_cert_pem}" + leaf_key = "${module.leaf_tls_self_signed_cert.leaf_private_key_pem}" + consul_encrypt = "${random_id.consul_encrypt.b64_std}" + consul_override = "${var.consul_client_config_override != "" ? true : false}" + consul_config = "${var.consul_client_config_override}" + vault_encrypt = "${random_id.consul_encrypt.b64_std}" + vault_override = "${var.vault_server_config_override != "" ? true : false}" + vault_config = "${var.vault_server_config_override}" + } +} + +module "vault_aws" { + source = "github.com/hashicorp-modules/vault-aws" + + name = "${var.name}" # Must match network_aws module name for Consul Auto Join to work + vpc_id = "${module.network_aws.vpc_id}" + vpc_cidr = "${module.network_aws.vpc_cidr}" + subnet_ids = "${split(",", var.vault_public ? join(",", module.network_aws.subnet_public_ids) : join(",", module.network_aws.subnet_private_ids))}" + release_version = "${var.vault_release}" + vault_version = "${var.vault_version}" + consul_version = "${var.consul_version}" + os = "${var.vault_os}" + os_version = "${var.vault_os_version}" + count = "${var.vault_servers}" + instance_profile = "${module.consul_auto_join_instance_role.instance_profile_id}" # Override instance_profile + instance_type = "${var.vault_instance}" + image_id = "${var.vault_image_id}" + public = "${var.vault_public}" + use_lb_cert = true + lb_cert = "${module.leaf_tls_self_signed_cert.leaf_cert_pem}" + lb_private_key = "${module.leaf_tls_self_signed_cert.leaf_private_key_pem}" + lb_cert_chain = "${module.root_tls_self_signed_ca.ca_cert_pem}" + user_data = "${data.template_file.vault_user_data.rendered}" # Custom user_data + ssh_key_name = "${module.ssh_keypair_aws_override.name}" + tags = "${var.vault_tags}" + tags_list = "${var.vault_tags_list}" +} diff --git a/operations/provision-vault/best-practices/terraform-aws/outputs.tf b/operations/provision-vault/best-practices/terraform-aws/outputs.tf new file mode 100644 index 00000000..7eb5dea9 --- /dev/null +++ b/operations/provision-vault/best-practices/terraform-aws/outputs.tf @@ -0,0 +1,153 @@ +output "zREADME" { + value = <= 0100 || var.vault_url != "") ? format("Vault UI: http://%s %s", module.vault_aws.vault_lb_dns, var.vault_public ? "(Public)" : "(Internal)") : "", + var.consul_install ? format("Consul UI: http://%s %s", module.consul_lb_aws.consul_lb_dns, var.vault_public ? "(Public)" : "(Internal)") : "", + ), +))} + +You can SSH into the Vault node by updating the "PUBLIC_IP" and running the +below command. + + $ ${format("ssh -A -i %s %s@%s", module.ssh_keypair_aws.private_key_filename, module.vault_aws.vault_username, "PUBLIC_IP")} + +${module.vault_aws.zREADME} +README +} + +output "vpc_cidr" { + value = "${module.network_aws.vpc_cidr}" +} + +output "vpc_id" { + value = "${module.network_aws.vpc_id}" +} + +output "subnet_public_ids" { + value = "${module.network_aws.subnet_public_ids}" +} + +output "subnet_private_ids" { + value = "${module.network_aws.subnet_private_ids}" +} + +output "private_key_name" { + value = "${module.ssh_keypair_aws.private_key_name}" +} + +output "private_key_filename" { + value = "${module.ssh_keypair_aws.private_key_filename}" +} + +output "private_key_pem" { + value = "${module.ssh_keypair_aws.private_key_pem}" +} + +output "public_key_pem" { + value = "${module.ssh_keypair_aws.public_key_pem}" +} + +output "public_key_openssh" { + value = "${module.ssh_keypair_aws.public_key_openssh}" +} + +output "ssh_key_name" { + value = "${module.ssh_keypair_aws.name}" +} + +output "consul_lb_sg_id" { + value = "${module.consul_lb_aws.consul_lb_sg_id}" +} + +output "consul_tg_http_8500_arn" { + value = "${module.consul_lb_aws.consul_tg_http_8500_arn}" +} + +output "consul_lb_dns" { + value = "${module.consul_lb_aws.consul_lb_dns}" +} + +output "vault_asg_id" { + value = "${module.vault_aws.vault_asg_id}" +} + +output "vault_sg_id" { + value = "${module.vault_aws.vault_sg_id}" +} + +output "vault_lb_sg_id" { + value = "${module.vault_aws.vault_lb_sg_id}" +} + +output "vault_tg_http_8200_arn" { + value = "${module.vault_aws.vault_tg_http_8200_arn}" +} + +output "vault_lb_dns" { + value = "${module.vault_aws.vault_lb_dns}" +} diff --git a/operations/provision-vault/dev/terraform-aws/terraform.auto.tfvars b/operations/provision-vault/dev/terraform-aws/terraform.auto.tfvars new file mode 100644 index 00000000..d8c02f73 --- /dev/null +++ b/operations/provision-vault/dev/terraform-aws/terraform.auto.tfvars @@ -0,0 +1,70 @@ +# --------------------------------------------------------------------------------------------------------------------- +# General Variables +# --------------------------------------------------------------------------------------------------------------------- +# name = "vault-dev" +# ami_owner = "099720109477" # Base image owner, defaults to RHEL +# ami_name = "*ubuntu-xenial-16.04-amd64-server-*" # Base image name, defaults to RHEL + +# --------------------------------------------------------------------------------------------------------------------- +# Network Variables +# --------------------------------------------------------------------------------------------------------------------- +# vpc_cidr = "172.19.0.0/16" +# vpc_cidrs_public = ["172.19.0.0/20", "172.19.16.0/20", "172.19.32.0/20",] +# vpc_cidrs_private = ["172.19.48.0/20", "172.19.64.0/20", "172.19.80.0/20",] + +# nat_count = 1 # Defaults to 1 +# bastion_servers = 0 # Defaults to 0 +# bastion_image_id = "" # AMI ID override, defaults to base RHEL AMI + +# network_tags = {"owner" = "hashicorp", "TTL" = "24"} + +# --------------------------------------------------------------------------------------------------------------------- +# Consul Variables +# --------------------------------------------------------------------------------------------------------------------- +# consul_install = true # Install Consul +# consul_version = "1.0.6" # Consul Version for runtime install, defaults to 1.0.6 +# consul_url = "" # Consul Enterprise download URL for runtime install, defaults to Consul OSS + +# consul_config_override = < consul_group, + "USER" => consul_user, + "COMMENT" => consul_comment, + "HOME" => consul_home, + } + + # Install Consul + config.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/consul/scripts/install-consul.sh | bash", + env: { + "VERSION" => consul_version, + "URL" => consul_ent_url, + "USER" => consul_user, + "GROUP" => consul_group, + } + + config.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/consul/scripts/install-consul-systemd.sh | bash" + + # Setup Vault user + config.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/setup-user.sh | bash", + env: { + "GROUP" => vault_group, + "USER" => vault_user, + "COMMENT" => vault_comment, + "HOME" => vault_home, + } + + # Install Vault + config.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/vault/scripts/install-vault.sh | bash", + env: { + "VERSION" => vault_version, + "URL" => vault_ent_url, + "USER" => vault_user, + "GROUP" => vault_group, + } + + config.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/vault/scripts/install-vault-systemd.sh | bash" + + # Setup tests + config.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/setup-testing.sh | bash" + + # Run tests if `RUN_TESTS` env var is provided + if (run_tests) + config.vm.provision "shell", inline: "cd /tmp && rake vault:spec" + end + + # Cleanup if `CLEANUP` env var is provided + if (cleanup) + config.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/cleanup.sh | bash" + end + + # Increase memory for Parallels Desktop + config.vm.provider "parallels" do |p, o| + p.memory = "1024" + end + + # Increase memory for Virtualbox + config.vm.provider "virtualbox" do |vb| + vb.memory = "1024" + end + + # Increase memory for VMware + ["vmware_fusion", "vmware_workstation"].each do |p| + config.vm.provider p do |v| + v.vmx["memsize"] = "1024" + end + end + + config.vm.post_up_message = " +Your Vault dev cluster has been successfully provisioned! + +To SSH into a Vault host, run the below command. + + $ vagrant ssh + +You can interact with Vault using any of the CLI (https://www.vaultproject.io/docs/commands/index.html) or API (https://www.vaultproject.io/api/index.html) commands. + + # The Root token for your Vault -dev instance is set to `root` and placed in /srv/vault/.vault-token, the `VAULT_TOKEN` environment variable has already been set for you + $ echo $VAULT_TOKEN + $ sudo cat /srv/vault/.vault-token + + # Use the CLI to write and read a generic secret + $ vault kv put secret/cli foo=bar + $ vault kv get secret/cli + + # Use the API to write and read a generic secret + $ curl -H \"X-Vault-Token: $VAULT_TOKEN\" -X POST -d '{\"data\": {\"bar\":\"baz\"}}' http://127.0.0.1:8200/v1/secret/data/api | jq '.' + $ curl -H \"X-Vault-Token: $VAULT_TOKEN\" http://127.0.0.1:8200/v1/secret/data/api | jq '.' + +Visit the Consul UI: http://#{private_ip}:#{consul_host_port}/ +Visit the Vault UI: http://#{private_ip}:#{vault_host_port}/ + +Don't forget to tear your VM down after. + + $ vagrant destroy +" +end diff --git a/provision/kubernetes/minikube/.gitignore b/operations/provision-vault/kubernetes/minikube/.gitignore similarity index 100% rename from provision/kubernetes/minikube/.gitignore rename to operations/provision-vault/kubernetes/minikube/.gitignore diff --git a/provision/kubernetes/minikube/Makefile b/operations/provision-vault/kubernetes/minikube/Makefile similarity index 100% rename from provision/kubernetes/minikube/Makefile rename to operations/provision-vault/kubernetes/minikube/Makefile diff --git a/provision/kubernetes/minikube/README.markdown b/operations/provision-vault/kubernetes/minikube/README.markdown similarity index 100% rename from provision/kubernetes/minikube/README.markdown rename to operations/provision-vault/kubernetes/minikube/README.markdown diff --git a/provision/kubernetes/minikube/README.org b/operations/provision-vault/kubernetes/minikube/README.org similarity index 100% rename from provision/kubernetes/minikube/README.org rename to operations/provision-vault/kubernetes/minikube/README.org diff --git a/provision/kubernetes/minikube/build.sh b/operations/provision-vault/kubernetes/minikube/build.sh similarity index 100% rename from provision/kubernetes/minikube/build.sh rename to operations/provision-vault/kubernetes/minikube/build.sh diff --git a/provision/kubernetes/minikube/cleanup.sh b/operations/provision-vault/kubernetes/minikube/cleanup.sh similarity index 100% rename from provision/kubernetes/minikube/cleanup.sh rename to operations/provision-vault/kubernetes/minikube/cleanup.sh diff --git a/provision/kubernetes/minikube/config.sh b/operations/provision-vault/kubernetes/minikube/config.sh similarity index 100% rename from provision/kubernetes/minikube/config.sh rename to operations/provision-vault/kubernetes/minikube/config.sh diff --git a/provision/kubernetes/minikube/consul.yml b/operations/provision-vault/kubernetes/minikube/consul.yml similarity index 100% rename from provision/kubernetes/minikube/consul.yml rename to operations/provision-vault/kubernetes/minikube/consul.yml diff --git a/provision/kubernetes/minikube/data/draw.io/consul_k8s_sd_consul_off-cluster.xml b/operations/provision-vault/kubernetes/minikube/data/draw.io/consul_k8s_sd_consul_off-cluster.xml similarity index 100% rename from provision/kubernetes/minikube/data/draw.io/consul_k8s_sd_consul_off-cluster.xml rename to operations/provision-vault/kubernetes/minikube/data/draw.io/consul_k8s_sd_consul_off-cluster.xml diff --git a/provision/kubernetes/minikube/data/draw.io/consul_vault_on_cluster.xml b/operations/provision-vault/kubernetes/minikube/data/draw.io/consul_vault_on_cluster.xml similarity index 100% rename from provision/kubernetes/minikube/data/draw.io/consul_vault_on_cluster.xml rename to operations/provision-vault/kubernetes/minikube/data/draw.io/consul_vault_on_cluster.xml diff --git a/provision/kubernetes/minikube/deps.sh b/operations/provision-vault/kubernetes/minikube/deps.sh similarity index 100% rename from provision/kubernetes/minikube/deps.sh rename to operations/provision-vault/kubernetes/minikube/deps.sh diff --git a/provision/kubernetes/minikube/static/consul-enterprise/Dockerfile b/operations/provision-vault/kubernetes/minikube/static/consul-enterprise/Dockerfile similarity index 100% rename from provision/kubernetes/minikube/static/consul-enterprise/Dockerfile rename to operations/provision-vault/kubernetes/minikube/static/consul-enterprise/Dockerfile diff --git a/provision/kubernetes/minikube/static/consul-enterprise/assets/.gitkeep b/operations/provision-vault/kubernetes/minikube/static/consul-enterprise/assets/.gitkeep similarity index 100% rename from provision/kubernetes/minikube/static/consul-enterprise/assets/.gitkeep rename to operations/provision-vault/kubernetes/minikube/static/consul-enterprise/assets/.gitkeep diff --git a/provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Enterprise Logos/Consul_Enterprise_PrimaryLogo.png b/operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Enterprise Logos/Consul_Enterprise_PrimaryLogo.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Enterprise Logos/Consul_Enterprise_PrimaryLogo.png rename to operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Enterprise Logos/Consul_Enterprise_PrimaryLogo.png diff --git a/provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Enterprise Logos/Consul_Enterprise_VerticalLogo.png b/operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Enterprise Logos/Consul_Enterprise_VerticalLogo.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Enterprise Logos/Consul_Enterprise_VerticalLogo.png rename to operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Enterprise Logos/Consul_Enterprise_VerticalLogo.png diff --git a/provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Black/Consul_PrimaryLogo_Black.png b/operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Black/Consul_PrimaryLogo_Black.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Black/Consul_PrimaryLogo_Black.png rename to operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Black/Consul_PrimaryLogo_Black.png diff --git a/provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Full Color/Consul_PrimaryLogo_FullColor.png b/operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Full Color/Consul_PrimaryLogo_FullColor.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Full Color/Consul_PrimaryLogo_FullColor.png rename to operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Full Color/Consul_PrimaryLogo_FullColor.png diff --git a/provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Monochrome Tonal/Consul_PrimaryLogo_MonochromeTonal.png b/operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Monochrome Tonal/Consul_PrimaryLogo_MonochromeTonal.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Monochrome Tonal/Consul_PrimaryLogo_MonochromeTonal.png rename to operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/Monochrome Tonal/Consul_PrimaryLogo_MonochromeTonal.png diff --git a/provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/White/Consul_PrimaryLogo_White.png b/operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/White/Consul_PrimaryLogo_White.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/White/Consul_PrimaryLogo_White.png rename to operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Primary Logos/White/Consul_PrimaryLogo_White.png diff --git a/provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Black/Consul_VerticalLogo_Black.png b/operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Black/Consul_VerticalLogo_Black.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Black/Consul_VerticalLogo_Black.png rename to operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Black/Consul_VerticalLogo_Black.png diff --git a/provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Full Color/Consul_VerticalLogo_FullColor.png b/operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Full Color/Consul_VerticalLogo_FullColor.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Full Color/Consul_VerticalLogo_FullColor.png rename to operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Full Color/Consul_VerticalLogo_FullColor.png diff --git a/provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Monochrome Tonal/Consul_VerticalLogo_MonochromeTonal.png b/operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Monochrome Tonal/Consul_VerticalLogo_MonochromeTonal.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Monochrome Tonal/Consul_VerticalLogo_MonochromeTonal.png rename to operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/Monochrome Tonal/Consul_VerticalLogo_MonochromeTonal.png diff --git a/provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/White/Consul_VerticalLogo_White.png b/operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/White/Consul_VerticalLogo_White.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/White/Consul_VerticalLogo_White.png rename to operations/provision-vault/kubernetes/minikube/static/images/Consul/Logos/PNG/Vertical Logos/White/Consul_VerticalLogo_White.png diff --git a/provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Enterprise Logos/Vault_Enterprise_PrimaryLogo.png b/operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Enterprise Logos/Vault_Enterprise_PrimaryLogo.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Enterprise Logos/Vault_Enterprise_PrimaryLogo.png rename to operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Enterprise Logos/Vault_Enterprise_PrimaryLogo.png diff --git a/provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Enterprise Logos/Vault_Enterprise_VerticalLogo.png b/operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Enterprise Logos/Vault_Enterprise_VerticalLogo.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Enterprise Logos/Vault_Enterprise_VerticalLogo.png rename to operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Enterprise Logos/Vault_Enterprise_VerticalLogo.png diff --git a/provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Black/Vault_PrimaryLogo_Black.png b/operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Black/Vault_PrimaryLogo_Black.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Black/Vault_PrimaryLogo_Black.png rename to operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Black/Vault_PrimaryLogo_Black.png diff --git a/provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Full Color/Vault_PrimaryLogo_FullColor.png b/operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Full Color/Vault_PrimaryLogo_FullColor.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Full Color/Vault_PrimaryLogo_FullColor.png rename to operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Full Color/Vault_PrimaryLogo_FullColor.png diff --git a/provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Monochrome Tonal/Vault_PrimaryLogo_MonochromeTonal.png b/operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Monochrome Tonal/Vault_PrimaryLogo_MonochromeTonal.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Monochrome Tonal/Vault_PrimaryLogo_MonochromeTonal.png rename to operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/Monochrome Tonal/Vault_PrimaryLogo_MonochromeTonal.png diff --git a/provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/White/Vault_PrimaryLogo_White.png b/operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/White/Vault_PrimaryLogo_White.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/White/Vault_PrimaryLogo_White.png rename to operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Primary Logos/White/Vault_PrimaryLogo_White.png diff --git a/provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Black/Vault_VerticalLogo_Black.png b/operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Black/Vault_VerticalLogo_Black.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Black/Vault_VerticalLogo_Black.png rename to operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Black/Vault_VerticalLogo_Black.png diff --git a/provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Full Color/Vault_VerticalLogo_FullColor.png b/operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Full Color/Vault_VerticalLogo_FullColor.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Full Color/Vault_VerticalLogo_FullColor.png rename to operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Full Color/Vault_VerticalLogo_FullColor.png diff --git a/provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Monochrome Tonal/Vault_VerticalLogo_MonochromeTonal.png b/operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Monochrome Tonal/Vault_VerticalLogo_MonochromeTonal.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Monochrome Tonal/Vault_VerticalLogo_MonochromeTonal.png rename to operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/Monochrome Tonal/Vault_VerticalLogo_MonochromeTonal.png diff --git a/provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/White/Vault_VerticalLogo_White.png b/operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/White/Vault_VerticalLogo_White.png similarity index 100% rename from provision/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/White/Vault_VerticalLogo_White.png rename to operations/provision-vault/kubernetes/minikube/static/images/Vault/Logos/PNG/Vertical Logos/White/Vault_VerticalLogo_White.png diff --git a/provision/kubernetes/minikube/static/images/consul-vault-unsealed.png b/operations/provision-vault/kubernetes/minikube/static/images/consul-vault-unsealed.png similarity index 100% rename from provision/kubernetes/minikube/static/images/consul-vault-unsealed.png rename to operations/provision-vault/kubernetes/minikube/static/images/consul-vault-unsealed.png diff --git a/provision/kubernetes/minikube/static/images/consul_k8s_sd_consul_off-cluster.png b/operations/provision-vault/kubernetes/minikube/static/images/consul_k8s_sd_consul_off-cluster.png similarity index 100% rename from provision/kubernetes/minikube/static/images/consul_k8s_sd_consul_off-cluster.png rename to operations/provision-vault/kubernetes/minikube/static/images/consul_k8s_sd_consul_off-cluster.png diff --git a/provision/kubernetes/minikube/static/images/consul_k8s_sd_consul_off-cluster.svg b/operations/provision-vault/kubernetes/minikube/static/images/consul_k8s_sd_consul_off-cluster.svg similarity index 100% rename from provision/kubernetes/minikube/static/images/consul_k8s_sd_consul_off-cluster.svg rename to operations/provision-vault/kubernetes/minikube/static/images/consul_k8s_sd_consul_off-cluster.svg diff --git a/provision/kubernetes/minikube/static/images/consul_ui.png b/operations/provision-vault/kubernetes/minikube/static/images/consul_ui.png similarity index 100% rename from provision/kubernetes/minikube/static/images/consul_ui.png rename to operations/provision-vault/kubernetes/minikube/static/images/consul_ui.png diff --git a/provision/kubernetes/minikube/static/images/consul_vault_on_cluster.jpg b/operations/provision-vault/kubernetes/minikube/static/images/consul_vault_on_cluster.jpg similarity index 100% rename from provision/kubernetes/minikube/static/images/consul_vault_on_cluster.jpg rename to operations/provision-vault/kubernetes/minikube/static/images/consul_vault_on_cluster.jpg diff --git a/provision/kubernetes/minikube/static/images/consul_vault_on_cluster.png b/operations/provision-vault/kubernetes/minikube/static/images/consul_vault_on_cluster.png similarity index 100% rename from provision/kubernetes/minikube/static/images/consul_vault_on_cluster.png rename to operations/provision-vault/kubernetes/minikube/static/images/consul_vault_on_cluster.png diff --git a/provision/kubernetes/minikube/static/images/consul_vault_on_cluster.svg b/operations/provision-vault/kubernetes/minikube/static/images/consul_vault_on_cluster.svg similarity index 100% rename from provision/kubernetes/minikube/static/images/consul_vault_on_cluster.svg rename to operations/provision-vault/kubernetes/minikube/static/images/consul_vault_on_cluster.svg diff --git a/provision/kubernetes/minikube/static/images/k8s/kubernetes.png b/operations/provision-vault/kubernetes/minikube/static/images/k8s/kubernetes.png similarity index 100% rename from provision/kubernetes/minikube/static/images/k8s/kubernetes.png rename to operations/provision-vault/kubernetes/minikube/static/images/k8s/kubernetes.png diff --git a/provision/kubernetes/minikube/static/images/minikube-consul-consul.png b/operations/provision-vault/kubernetes/minikube/static/images/minikube-consul-consul.png similarity index 100% rename from provision/kubernetes/minikube/static/images/minikube-consul-consul.png rename to operations/provision-vault/kubernetes/minikube/static/images/minikube-consul-consul.png diff --git a/provision/kubernetes/minikube/static/images/vault-auth-root.png b/operations/provision-vault/kubernetes/minikube/static/images/vault-auth-root.png similarity index 100% rename from provision/kubernetes/minikube/static/images/vault-auth-root.png rename to operations/provision-vault/kubernetes/minikube/static/images/vault-auth-root.png diff --git a/provision/kubernetes/minikube/static/images/vault-auth.png b/operations/provision-vault/kubernetes/minikube/static/images/vault-auth.png similarity index 100% rename from provision/kubernetes/minikube/static/images/vault-auth.png rename to operations/provision-vault/kubernetes/minikube/static/images/vault-auth.png diff --git a/provision/kubernetes/minikube/static/vault-enterprise/Dockerfile b/operations/provision-vault/kubernetes/minikube/static/vault-enterprise/Dockerfile similarity index 100% rename from provision/kubernetes/minikube/static/vault-enterprise/Dockerfile rename to operations/provision-vault/kubernetes/minikube/static/vault-enterprise/Dockerfile diff --git a/provision/kubernetes/minikube/static/vault-enterprise/assets/.gitkeep b/operations/provision-vault/kubernetes/minikube/static/vault-enterprise/assets/.gitkeep similarity index 100% rename from provision/kubernetes/minikube/static/vault-enterprise/assets/.gitkeep rename to operations/provision-vault/kubernetes/minikube/static/vault-enterprise/assets/.gitkeep diff --git a/provision/kubernetes/minikube/validate.d/inspec/consul.rb b/operations/provision-vault/kubernetes/minikube/validate.d/inspec/consul.rb similarity index 100% rename from provision/kubernetes/minikube/validate.d/inspec/consul.rb rename to operations/provision-vault/kubernetes/minikube/validate.d/inspec/consul.rb diff --git a/provision/kubernetes/minikube/validate.d/inspec/vault.rb b/operations/provision-vault/kubernetes/minikube/validate.d/inspec/vault.rb similarity index 100% rename from provision/kubernetes/minikube/validate.d/inspec/vault.rb rename to operations/provision-vault/kubernetes/minikube/validate.d/inspec/vault.rb diff --git a/provision/kubernetes/minikube/validate.sh b/operations/provision-vault/kubernetes/minikube/validate.sh similarity index 100% rename from provision/kubernetes/minikube/validate.sh rename to operations/provision-vault/kubernetes/minikube/validate.sh diff --git a/provision/kubernetes/minikube/vault.yml b/operations/provision-vault/kubernetes/minikube/vault.yml similarity index 100% rename from provision/kubernetes/minikube/vault.yml rename to operations/provision-vault/kubernetes/minikube/vault.yml diff --git a/operations/provision-vault/quick-start/terraform-aws/README.md b/operations/provision-vault/quick-start/terraform-aws/README.md new file mode 100644 index 00000000..cef3a259 --- /dev/null +++ b/operations/provision-vault/quick-start/terraform-aws/README.md @@ -0,0 +1,92 @@ +# Provision a Quick Start Vault Cluster in AWS + +The goal of this guide is to allows users to easily provision a quick start Vault & Consul cluster in just a few commands. + +## Reference Material + +- [Terraform Getting Started](https://www.terraform.io/intro/getting-started/install.html) +- [Terraform Docs](https://www.terraform.io/docs/index.html) +- [Consul Getting Started](https://www.consul.io/intro/getting-started/install.html) +- [Consul Docs](https://www.consul.io/docs/index.html) +- [Vault Getting Started](https://www.vaultproject.io/intro/getting-started/install.html) +- [Vault Docs](https://www.vaultproject.io/docs/index.html) + +## Estimated Time to Complete + +5 minutes. + +## Challenge + +There are many different ways to provision and configure an easily accessible quick start Vault & Consul cluster, making it difficult to get started. + +## Solution + +Provision a quick start Vault & Consul cluster in a private network with a bastion host. + +The AWS Quick Start Vault guide leverages the scripts in the [Guides Configuration Repo](https://github.com/hashicorp/guides-configuration) to do runtime configuration for Vault & Consul. Although using `curl bash` at runtime is _not_ best practices, this makes it quick and easy to standup a Vault & Consul cluster with no external dependencies like pre-built images. This guide will also forgo setting up TLS/encryption on Vault & Consul for the sake of simplicity. + +## Prerequisites + +- [Download Terraform](https://www.terraform.io/downloads.html) + +## Steps + +We will now provision the quick start Vault & Consul clusters. + +### Step 1: Initialize + +Initialize Terraform - download providers and modules. + +#### CLI + +[`terraform init` Command](https://www.terraform.io/docs/commands/init.html) + +##### Request + +```sh +$ terraform init +``` + +##### Response +``` +``` + +### Step 2: Plan + +Run a `terraform plan` to ensure Terraform will provision what you expect. + +#### CLI + +[`terraform plan` Command](https://www.terraform.io/docs/commands/plan.html) + +##### Request + +```sh +$ terraform plan +``` + +##### Response +``` +``` + +### Step 3: Apply + +Run a `terraform apply` to provision the HashiStack. One provisioned, view the `zREADME` instructions output from Terraform for next steps. + +#### CLI + +[`terraform apply` command](https://www.terraform.io/docs/commands/apply.html) + +##### Request + +```sh +$ terraform apply +``` + +##### Response +``` +``` + +## Next Steps + +Now that you've provisioned and configured a quick start Vault & Consul cluster, start walking through the [Consul Guides](https://www.consul.io/docs/guides/index.html). diff --git a/operations/provision-vault/quick-start/terraform-aws/main.tf b/operations/provision-vault/quick-start/terraform-aws/main.tf new file mode 100644 index 00000000..842368ca --- /dev/null +++ b/operations/provision-vault/quick-start/terraform-aws/main.tf @@ -0,0 +1,154 @@ +data "aws_ami" "base" { + most_recent = true + owners = ["${var.ami_owner}"] + + filter { + name = "name" + values = ["${var.ami_name}"] + } + + filter { + name = "root-device-type" + values = ["ebs"] + } + + filter { + name = "virtualization-type" + values = ["hvm"] + } +} + +data "template_file" "base_install" { + template = "${file("${path.module}/../../templates/install-base.sh.tpl")}" +} + +data "template_file" "consul_install" { + template = "${file("${path.module}/../../templates/install-consul-systemd.sh.tpl")}" + + vars = { + consul_version = "${var.consul_version}" + consul_url = "${var.consul_url}" + name = "${var.name}" + local_ip_url = "${var.local_ip_url}" + consul_override = false + consul_config = "" + } +} + +data "template_file" "vault_install" { + template = "${file("${path.module}/../../templates/install-vault-systemd.sh.tpl")}" + + vars = { + vault_version = "${var.vault_version}" + vault_url = "${var.vault_url}" + name = "${var.name}" + local_ip_url = "${var.local_ip_url}" + vault_override = false + vault_config = "" + } +} + +data "template_file" "bastion_quick_start" { + template = "${file("${path.module}/../../templates/quick-start-bastion-systemd.sh.tpl")}" + + vars = { + name = "${var.name}" + provider = "${var.provider}" + local_ip_url = "${var.local_ip_url}" + consul_override = "${var.consul_client_config_override != "" ? true : false}" + consul_config = "${var.consul_client_config_override}" + } +} + +module "network_aws" { + source = "github.com/hashicorp-modules/network-aws" + + name = "${var.name}" + vpc_cidr = "${var.vpc_cidr}" + vpc_cidrs_public = "${var.vpc_cidrs_public}" + vpc_cidrs_private = "${var.vpc_cidrs_private}" + nat_count = "${var.nat_count}" + bastion_count = "${var.bastion_servers}" + instance_type = "${var.bastion_instance}" + os = "${replace(lower(var.ami_name), "ubuntu", "") != lower(var.ami_name) ? "Ubuntu" : replace(lower(var.ami_name), "rhel", "") != lower(var.ami_name) ? "RHEL" : "unknown"}" + image_id = "${var.bastion_image_id != "" ? var.bastion_image_id : data.aws_ami.base.id}" + tags = "${var.network_tags}" + user_data = < /var/tmp/consul-token -CONSULACL - -Vagrant.configure("2") do |config| - # use vagrant insecure public key - # important: comment this out to restrict access - config.ssh.insert_key = false - - # use base_box set at the top of this file - config.vm.box = base_box - config.vm.box_version = base_box_version - config.vm.hostname = "hashistack" - config.vm.network :forwarded_port, guest: 8200, host: 8200, auto_correct: true - config.vm.provision "shell", inline: $script, privileged: false - config.vm.provision "shell", inline: $consulacl, privileged: false - - # copy the consul and nomad directories to the vm - config.vm.provision "file", source: "../../../shared", destination: "/tmp" - - # bootstrap the vm - config.vm.provision "shell", path: "../../../shared/scripts/base.sh" - config.vm.provision "shell", path: "../../../shared/scripts/base-aws.sh" - config.vm.provision "shell", path: "../../../shared/scripts/setup-user.sh", args: "consul" - config.vm.provision "shell", path: "../../../shared/scripts/setup-user.sh", args: "nomad" - config.vm.provision "shell", path: "../../../shared/scripts/setup-user.sh", args: "vault" - - # install consul - config.vm.provision "shell", path: "../../../shared/consul/scripts/install-consul.sh", - env: { "VERSION" => consul_version, "URL" => consul_download_url,"SERVER" => consul_server } - config.vm.provision "shell", path: "../../../shared/consul/scripts/install-consul-systemd.sh" - - # install vault - config.vm.provision "shell", path: "../../../shared/vault/scripts/install-vault.sh", - env: { "VERSION" => vault_version,"URL" => vault_download_url } - config.vm.provision "shell", inline: "sudo mv /etc/vault.d/vault-ui.hcl.example /etc/vault.d/vault-ui.hcl" - config.vm.provision "shell", path: "../../../shared/vault/scripts/install-vault-systemd.sh" - - # install nomad - config.vm.provision "shell", path: "../../../shared/nomad/scripts/install-nomad.sh", - env: { "VERSION" => nomad_version,"URL" => nomad_download_url } - config.vm.provision "shell", path: "../../../shared/nomad/scripts/install-nomad-systemd.sh" - - # Increase memory for Parallels Desktop - config.vm.provider "parallels" do |p, o| - p.memory = "1024" - end - - # Increase memory for Virtualbox - config.vm.provider "virtualbox" do |vb| - vb.memory = "1024" - end - - # Increase memory for VMware - ["vmware_fusion", "vmware_workstation"].each do |p| - config.vm.provider p do |v| - v.vmx["memsize"] = "1024" - end - end -end diff --git a/secrets/README.md b/secrets/README.md deleted file mode 100644 index 2dde4235..00000000 --- a/secrets/README.md +++ /dev/null @@ -1 +0,0 @@ -Guides related to secrets management with Vault stored here. Static secrets, dynamic secrets, consul-template and so forth. \ No newline at end of file diff --git a/secrets/database_mysql/README.md b/secrets/database-mysql/README.md similarity index 98% rename from secrets/database_mysql/README.md rename to secrets/database-mysql/README.md index 678de060..72a692a6 100644 --- a/secrets/database_mysql/README.md +++ b/secrets/database-mysql/README.md @@ -1,2 +1 @@ # Generating dynamic MySQL credentials using Vault Database backend - diff --git a/secrets/database_mysql/vagrant/Vagrantfile b/secrets/database-mysql/vagrant-local/Vagrantfile similarity index 57% rename from secrets/database_mysql/vagrant/Vagrantfile rename to secrets/database-mysql/vagrant-local/Vagrantfile index c1e57bd9..764aa884 100644 --- a/secrets/database_mysql/vagrant/Vagrantfile +++ b/secrets/database-mysql/vagrant-local/Vagrantfile @@ -2,9 +2,13 @@ # -*- mode: ruby -*- # vi: set ft=ruby : -# vault variable defaults +# Vault variable defaults vault_version = ENV['VAULT_VERSION'] || "0.9.0" -vault_download_url = ENV['VAULT_ENT_URL'] || "https://releases.hashicorp.com/vault/#{vault_version}/vault_#{vault_version}_linux_amd64.zip" +vault_ent_url = ENV['VAULT_ENT_URL'] +vault_group = "vault" +vault_user = "vault" +vault_comment = "Vault" +vault_home = "/srv/vault" $vault_env = < /etc/profile.d/vault.sh @@ -38,14 +42,24 @@ Vagrant.configure("2") do |config| vault.vm.box = "bento/centos-7.3" vault.vm.box_version = "2.3.8" vault.vm.hostname = "vault" - vault.vm.provision "shell", path: "../../../shared/scripts/base.sh" - vault.vm.provision "shell", path: "../../../shared/scripts/setup-user.sh", args: "vault" - vault.vm.provision "shell", path: "../../../shared/vault/scripts/install-vault.sh", - env: { "VERSION" => vault_version,"URL" => vault_download_url } + vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/base.sh | bash" + vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/shared/scripts/setup-user.sh | bash", + env: { + "GROUP" => vault_group, + "USER" => vault_user, + "COMMENT" => vault_comment, + "HOME" => vault_home, + } + vault.vm.provision "shell", inline: "curl https://raw.githubusercontent.com/hashicorp/guides-configuration/master/vault/scripts/install-vault.sh | bash", + env: { + "VERSION" => vault_version, + "URL" => vault_ent_url, + "USER" => vault_user, + "GROUP" => vault_group, + } vault.vm.provision "shell", inline: $vault_env vault.vm.provision "shell", inline: $vault_run vault.vm.provision "shell", inline: $hosts_file end end - diff --git a/secrets/database_mysql/vagrant/database_mysql_setup.sh b/secrets/database-mysql/vagrant-local/database-mysql-setup.sh similarity index 93% rename from secrets/database_mysql/vagrant/database_mysql_setup.sh rename to secrets/database-mysql/vagrant-local/database-mysql-setup.sh index b05ff73f..847334ee 100644 --- a/secrets/database_mysql/vagrant/database_mysql_setup.sh +++ b/secrets/database-mysql/vagrant-local/database-mysql-setup.sh @@ -6,7 +6,7 @@ sudo systemctl start mariadb mysqladmin -u root password R00tPassword mysql -u root -p'R00tPassword' << EOF -GRANT ALL PRIVILEGES ON *.* TO 'vaultadmin'@'127.0.0.1' IDENTIFIED BY 'vaultadminpassword' WITH GRANT OPTION; +GRANT ALL PRIVILEGES ON *.* TO 'vaultadmin'@'127.0.0.1' IDENTIFIED BY 'vaultadminpassword' WITH GRANT OPTION; FLUSH PRIVILEGES; EOF # Authenticate to Vault @@ -15,7 +15,7 @@ vault auth password # Mount database backend vault mount database -# Configure MySQL connection +# Configure MySQL connection vault write database/config/mysql \ plugin_name=mysql-legacy-database-plugin \ connection_url="vaultadmin:vaultadminpassword@tcp(127.0.0.1:3306)/" \ diff --git a/shared/consul-template/scripts/install-consul-template.sh b/shared/consul-template/scripts/install-consul-template.sh deleted file mode 100644 index e6f0e7bb..00000000 --- a/shared/consul-template/scripts/install-consul-template.sh +++ /dev/null @@ -1,36 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - FILENAME="install-consul-template.sh" - echo "$DT $FILENAME: $1" -} - -logger "Running" - -CONSUL_TEMPLATE_VERSION=${VERSION:-"0.19.4"} -CONSUL_TEMPLATE_ZIP="consul-template_${CONSUL_TEMPLATE_VERSION}_linux_amd64.zip" -CONSUL_TEMPLATE_URL=${URL:-"https://releases.hashicorp.com/consul-template/${CONSUL_TEMPLATE_VERSION}/${CONSUL_TEMPLATE_ZIP}"} -CONSUL_TEMPLATE_USER=${USER:-"consul-template"} -CONSUL_TEMPLATE_GROUP=${GROUP:-"consul-template"} -CONFIG_DIR="/etc/consul-template.d" -DATA_DIR="/opt/consul-template/data" -DOWNLOAD_DIR="/tmp" - -logger "Downloading consul-template ${CONSUL_TEMPLATE_VERSION}" -curl --silent --output ${DOWNLOAD_DIR}/${CONSUL_TEMPLATE_ZIP} ${CONSUL_TEMPLATE_URL} - -logger "Installing consul-template" -sudo unzip -o ${DOWNLOAD_DIR}/${CONSUL_TEMPLATE_ZIP} -d /usr/local/bin/ -sudo chmod 0755 /usr/local/bin/consul-template -sudo chown ${CONSUL_TEMPLATE_USER}:${CONSUL_TEMPLATE_GROUP} /usr/local/bin/consul-template - -logger "/usr/local/bin/consul-template --version: $(/usr/local/bin/consul-template --version)" - -logger "Configuring consul-template" -sudo mkdir -pm 0755 ${CONFIG_DIR} ${DATA_DIR} -sudo chown -R ${CONSUL_TEMPLATE_USER}:${CONSUL_TEMPLATE_GROUP} ${CONFIG_DIR} ${DATA_DIR} -sudo chmod -R 0644 ${CONFIG_DIR}/* - -logger "Complete" diff --git a/shared/consul/config-hcl/consul-default.hcl b/shared/consul/config-hcl/consul-default.hcl deleted file mode 100644 index 78bb4154..00000000 --- a/shared/consul/config-hcl/consul-default.hcl +++ /dev/null @@ -1,6 +0,0 @@ -advertise_addr = "127.0.0.1" -data_dir = "/opt/consul/data" -client_addr = "0.0.0.0" -log_level = "INFO" -ui = true - diff --git a/shared/consul/config-hcl/consul-server.hcl b/shared/consul/config-hcl/consul-server.hcl deleted file mode 100644 index 39b3f998..00000000 --- a/shared/consul/config-hcl/consul-server.hcl +++ /dev/null @@ -1,3 +0,0 @@ -server = true -bootstrap_expect = 1 -leave_on_terminate = true diff --git a/shared/consul/config/consul-default.json b/shared/consul/config/consul-default.json deleted file mode 100644 index 49ffc4d6..00000000 --- a/shared/consul/config/consul-default.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "advertise_addr": "127.0.0.1", - "data_dir": "/opt/consul/data", - "client_addr": "0.0.0.0", - "log_level": "INFO", - "ui": true -} diff --git a/shared/consul/config/consul-server.json b/shared/consul/config/consul-server.json deleted file mode 100644 index 4231e551..00000000 --- a/shared/consul/config/consul-server.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "server": true, - "bootstrap_expect": 1, - "leave_on_terminate": true -} diff --git a/shared/consul/config/consul-snapshot-aws-s3.json.example b/shared/consul/config/consul-snapshot-aws-s3.json.example deleted file mode 100644 index 73e80dac..00000000 --- a/shared/consul/config/consul-snapshot-aws-s3.json.example +++ /dev/null @@ -1,9 +0,0 @@ -{ - "snapshot_agent": { - "aws_storage": { - "s3_region": "{{ s3_region }}", - "s3_bucket": "{{ s3_bucket }}", - "s3_key_prefix": "consul-snapshot" - } - } -} diff --git a/shared/consul/config/consul-snapshot-default.json.example b/shared/consul/config/consul-snapshot-default.json.example deleted file mode 100644 index 44a558b0..00000000 --- a/shared/consul/config/consul-snapshot-default.json.example +++ /dev/null @@ -1,19 +0,0 @@ -{ - "snapshot_agent": { - "snapshot": { - "interval": "30m", - "retain": 15, - "stale": false, - "service": "consul-snapshot", - "deregister_after": "72h", - "lock_key": "consul-snapshot/lock", - "max_failures": 3 - }, - "local_storage": { - "path": "/opt/consul-snapshot/data" - }, - "log": { - "level": "INFO" - } - } -} diff --git a/shared/consul/init/systemd/consul-online.service b/shared/consul/init/systemd/consul-online.service deleted file mode 100644 index 7fb10a88..00000000 --- a/shared/consul/init/systemd/consul-online.service +++ /dev/null @@ -1,13 +0,0 @@ -[Unit] -Description=Consul Online -Requires=consul.service -After=consul.service - -[Service] -Type=oneshot -ExecStart=/usr/bin/consul-online.sh -User=consul -Group=consul - -[Install] -WantedBy=consul-online.target multi-user.target diff --git a/shared/consul/init/systemd/consul-online.sh b/shared/consul/init/systemd/consul-online.sh deleted file mode 100755 index 11aec0e3..00000000 --- a/shared/consul/init/systemd/consul-online.sh +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env bash - -set -e -set -o pipefail - -CONSUL_ADDRESS=${1:-"127.0.0.1:8500"} - -# waitForConsulToBeAvailable loops until the local Consul agent returns a 200 -# response at the /v1/operator/raft/configuration endpoint. -# -# Parameters: -# None -function waitForConsulToBeAvailable() { - local consul_addr=$1 - local consul_leader_http_code - - consul_leader_http_code=$(curl --silent --output /dev/null --write-out "%{http_code}" "${consul_addr}/v1/operator/raft/configuration") || consul_leader_http_code="" - - while [ "x${consul_leader_http_code}" != "x200" ] ; do - echo "Waiting for Consul to get a leader..." - sleep 5 - consul_leader_http_code=$(curl --silent --output /dev/null --write-out "%{http_code}" "${consul_addr}/v1/operator/raft/configuration") || consul_leader_http_code="" - done -} - -waitForConsulToBeAvailable "${CONSUL_ADDRESS}" diff --git a/shared/consul/init/systemd/consul-online.target b/shared/consul/init/systemd/consul-online.target deleted file mode 100644 index 4c8f91e4..00000000 --- a/shared/consul/init/systemd/consul-online.target +++ /dev/null @@ -1,3 +0,0 @@ -[Unit] -Description=Consul Online -RefuseManualStart=true diff --git a/shared/consul/init/systemd/consul-snapshot.service b/shared/consul/init/systemd/consul-snapshot.service deleted file mode 100644 index da97c0f9..00000000 --- a/shared/consul/init/systemd/consul-snapshot.service +++ /dev/null @@ -1,16 +0,0 @@ -[Unit] -Description=Consul Snapshot Agent -Requires=consul-online.target -After=consul-online.target - -[Service] -Restart=on-failure -PermissionsStartOnly=true -ExecStart=/usr/bin/consul snapshot agent -config-dir /etc/consul-snapshot.d -ExecReload=/bin/kill -HUP $MAINPID -KillSignal=SIGTERM -User=consulsnapshot -Group=consulsnapshot - -[Install] -WantedBy=multi-user.target diff --git a/shared/consul/init/systemd/consul.service b/shared/consul/init/systemd/consul.service deleted file mode 100644 index 8aa59634..00000000 --- a/shared/consul/init/systemd/consul.service +++ /dev/null @@ -1,15 +0,0 @@ -[Unit] -Description=Consul Agent -Requires=network-online.target -After=network-online.target - -[Service] -Restart=on-failure -ExecStart=/usr/local/bin/consul agent -config-dir /etc/consul.d -ExecReload=/bin/kill -HUP $MAINPID -KillSignal=SIGTERM -User=consul -Group=consul - -[Install] -WantedBy=multi-user.target diff --git a/shared/consul/scripts/install-consul-snapshot.sh b/shared/consul/scripts/install-consul-snapshot.sh deleted file mode 100644 index def9cd2b..00000000 --- a/shared/consul/scripts/install-consul-snapshot.sh +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - FILENAME="install-consul-snapshot.sh" - echo "$DT $FILENAME: $1" -} - -CONSUL_SNAPSHOT_USER=${USER:-} -CONSUL_SNAPSHOT_GROUP=${GROUP:-} -CONSUL_SNAPSHOT_AWS_S3=${AWS_S3:-} -cd /tmp - -logger "Setup Consul Snapshot default configuration and data directories" -sudo mkdir -pm 0600 /etc/consul-snapshot.d /opt/consul-snapshot/data -sudo cp /tmp/consul/config/consul-snapshot-default.json /etc/consul-snapshot.d/consul-snapshot-default.json -sudo chmod -R 0755 /etc/consul-snapshot.d -sudo chown -R ${CONSUL_SNAPSHOT_USER}.${CONSUL_SNAPSHOT_GROUP} /etc/consul-snapshot.d /opt/consul-snapshot - -if [[ ! -z ${CONSUL_SNAPSHOT_AWS_S3} ]]; then - logger "Setup Consul Snapshot AWS S3 configuration" - sudo cp /tmp/consul/config/consul-snapshot-aws-s3.json /etc/consul-snapshot.d/consul-snapshot-aws-s3.json -fi - -logger "Complete" diff --git a/shared/consul/scripts/install-consul-systemd.sh b/shared/consul/scripts/install-consul-systemd.sh deleted file mode 100644 index dc7d677f..00000000 --- a/shared/consul/scripts/install-consul-systemd.sh +++ /dev/null @@ -1,33 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -if [[ ! -z ${YUM} ]]; then - SYSTEMD_DIR="/etc/systemd/system" - logger "Installing consul systemd service for RHEL/CentOS" - sudo cp /tmp/consul/init/systemd/consul.service ${SYSTEMD_DIR} - sudo chmod 0664 ${SYSTEMD_DIR}/consul.service -elif [[ ! -z ${APT_GET} ]]; then - SYSTEMD_DIR="/lib/systemd/system" - logger "Installing consul systemd service for Debian/Ubuntu" - sudo cp /tmp/consul/init/systemd/consul.service ${SYSTEMD_DIR} - sudo chmod 0664 ${SYSTEMD_DIR}/consul.service -else - logger "Service not installed due to OS detection failure" - exit 1; -fi - -sudo systemctl enable consul -sudo systemctl start consul - -logger "Complete" diff --git a/shared/consul/scripts/install-consul.sh b/shared/consul/scripts/install-consul.sh deleted file mode 100644 index 5c5db902..00000000 --- a/shared/consul/scripts/install-consul.sh +++ /dev/null @@ -1,53 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -CONSUL_VERSION="${VERSION}" -CONSUL_ZIP="consul_${CONSUL_VERSION}_linux_amd64.zip" -CONSUL_URL=${URL:-"https://releases.hashicorp.com/consul/${CONSUL_VERSION}/${CONSUL_ZIP}"} - -logger "Downloading consul ${CONSUL_VERSION}" -curl --silent --output /tmp/${CONSUL_ZIP} ${CONSUL_URL} - -logger "Installing consul" -sudo unzip -o /tmp/${CONSUL_ZIP} -d /usr/local/bin/ -sudo chmod 0755 /usr/local/bin/consul -sudo chown consul:consul /usr/local/bin/consul -sudo mkdir -pm 0755 /etc/consul.d -sudo mkdir -pm 0755 /opt/consul/data - -logger "/usr/local/bin/consul --version: $(/usr/local/bin/consul --version)" - -logger "Configuring consul ${CONSUL_VERSION}" -sudo cp /tmp/consul/config-hcl/* /etc/consul.d/ -sudo chown -R consul:consul /etc/consul.d /opt/consul -sudo chmod -R 0644 /etc/consul.d/* - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -if [[ ! -z ${YUM} ]]; then - logger "Installing dnsmasq" - sudo yum install -q -y dnsmasq -elif [[ ! -z ${APT_GET} ]]; then - logger "Installing dnsmasq" - sudo apt-get -qq -y update - sudo apt-get install -qq -y dnsmasq-base dnsmasq -else - logger "Dnsmasq not installed due to OS detection failure" - exit 1; -fi - -logger "Configuring dnsmasq to forward .consul requests to consul port 8600" -sudo sh -c 'echo "server=/consul/127.0.0.1#8600" >> /etc/dnsmasq.d/consul' -sudo systemctl enable dnsmasq -sudo systemctl restart dnsmasq - -logger "Complete" diff --git a/shared/envconsul/scripts/install-envconsul.sh b/shared/envconsul/scripts/install-envconsul.sh deleted file mode 100644 index 7c6aba78..00000000 --- a/shared/envconsul/scripts/install-envconsul.sh +++ /dev/null @@ -1,36 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - FILENAME="install-envconsul.sh" - echo "$DT $FILENAME: $1" -} - -logger "Running" - -ENVCONSUL_VERSION=${VERSION:-"0.7.2"} -ENVCONSUL_ZIP="envconsul_${ENVCONSUL_VERSION}_linux_amd64.zip" -ENVCONSUL_URL=${URL:-"https://releases.hashicorp.com/envconsul/${ENVCONSUL_VERSION}/${ENVCONSUL_ZIP}"} -ENVCONSUL_USER=${USER:-"envconsul"} -ENVCONSUL_GROUP=${GROUP:-"envconsul"} -CONFIG_DIR="/etc/envconsul.d" -DATA_DIR="/opt/envconsul/data" -DOWNLOAD_DIR="/tmp" - -logger "Downloading envconsul ${ENVCONSUL_VERSION}" -curl --silent --output ${DOWNLOAD_DIR}/${ENVCONSUL_ZIP} ${ENVCONSUL_URL} - -logger "Installing envconsul" -sudo unzip -o ${DOWNLOAD_DIR}/${ENVCONSUL_ZIP} -d /usr/local/bin/ -sudo chmod 0755 /usr/local/bin/envconsul -sudo chown ${ENVCONSUL_USER}:${ENVCONSUL_GROUP} /usr/local/bin/envconsul - -logger "/usr/local/bin/envconsul --version: $(/usr/local/bin/envconsul --version)" - -logger "Configuring envconsul" -sudo mkdir -pm 0755 ${CONFIG_DIR} ${DATA_DIR} -sudo chown -R ${ENVCONSUL_USER}:${ENVCONSUL_GROUP} ${CONFIG_DIR} ${DATA_DIR} -sudo chmod -R 0644 ${CONFIG_DIR}/* - -logger "Complete" diff --git a/shared/nomad/config/nomad-client.hcl b/shared/nomad/config/nomad-client.hcl deleted file mode 100644 index 6ea3c550..00000000 --- a/shared/nomad/config/nomad-client.hcl +++ /dev/null @@ -1,9 +0,0 @@ -client { - enabled = true - client_max_port = 15000 - - options { - "docker.cleanup.image" = "0" - "driver.raw_exec.enable" = "1" - } -} diff --git a/shared/nomad/config/nomad-consul.hcl b/shared/nomad/config/nomad-consul.hcl deleted file mode 100644 index 3fd46f2e..00000000 --- a/shared/nomad/config/nomad-consul.hcl +++ /dev/null @@ -1,10 +0,0 @@ -consul { - address = "127.0.0.1:8500" - auto_advertise = true - - client_service_name = "nomad-client" - client_auto_join = true - - server_service_name = "nomad-server" - server_auto_join = true -} diff --git a/shared/nomad/config/nomad-default.hcl b/shared/nomad/config/nomad-default.hcl deleted file mode 100644 index 6022919b..00000000 --- a/shared/nomad/config/nomad-default.hcl +++ /dev/null @@ -1,3 +0,0 @@ -data_dir = "/opt/nomad/data" -log_level = "INFO" -enable_debug = true diff --git a/shared/nomad/config/nomad-server.hcl b/shared/nomad/config/nomad-server.hcl deleted file mode 100644 index ac8b3fea..00000000 --- a/shared/nomad/config/nomad-server.hcl +++ /dev/null @@ -1,5 +0,0 @@ -server { - enabled = true - bootstrap_expect = 1 - heartbeat_grace = "30s" -} diff --git a/shared/nomad/init/systemd/nomad-force-leave-missing-peers-ec2.service b/shared/nomad/init/systemd/nomad-force-leave-missing-peers-ec2.service deleted file mode 100644 index 5f2c2089..00000000 --- a/shared/nomad/init/systemd/nomad-force-leave-missing-peers-ec2.service +++ /dev/null @@ -1,14 +0,0 @@ -[Unit] -Description=Force Leave Missing Nomad Peers -Requires=network-online.target nomad-online.target -After=network-online.target nomad-online.target - -[Service] -Type=oneshot -RemainAfterExit=true -ExecStart=/usr/bin/nomad-force-leave-missing-peers-aws-ec2.sh -User=nomad -Group=nomad - -[Install] -WantedBy=multi-user.target diff --git a/shared/nomad/init/systemd/nomad-force-leave-missing-peers-ec2.sh b/shared/nomad/init/systemd/nomad-force-leave-missing-peers-ec2.sh deleted file mode 100755 index 98c30b07..00000000 --- a/shared/nomad/init/systemd/nomad-force-leave-missing-peers-ec2.sh +++ /dev/null @@ -1,162 +0,0 @@ -#!/usr/bin/env bash - -set -e - -# getMyRegion returns the region in which the current instance is running, -# based on the availability zone read from the EC2 metadata service. -# -# Parameters: -# None. -function getMyRegion() { - local metadata_base_url="http://169.254.169.254/latest/meta-data" - local this_instance_az - - this_instance_az=$(curl --silent --location ${metadata_base_url}/placement/availability-zone) - - #shellcheck disable=SC2001 - echo "${this_instance_az}" | sed 's/.$//' -} - -# getMyIPAddress returns the private IP address of the current instance, -# from the EC2 metadata service (rather than parsing the output of ifconfig -# or similar utilities). -# -# Parameters: -# None. -function getMyIPAddress() { - local metadata_base_url="http://169.254.169.254/latest/meta-data" - - curl --silent --location "${metadata_base_url}/local-ipv4" -} - -# getMyASGName returns the name of the autoscaling group of which the current -# instance is running, based on the aws:autoscaling:groupName tag which is -# populated by the autoscaling control system. -# -# Parameters: -# $1: the name of the region in which the instance is running. -function getMyASGName() { - local region=$1 - local metadata_base_url="http://169.254.169.254/latest/meta-data" - - local this_instance_id - - this_instance_id=$(curl --silent --location ${metadata_base_url}/instance-id) - - aws ec2 describe-tags \ - --region "${region}" \ - --filters "Name=resource-type,Values=instance" \ - "Name=resource-id,Values=${this_instance_id}" \ - "Name=key,Values=aws:autoscaling:groupName" \ - --query "Tags[0].Value" \ - --output=text -} - -# getInstanceAddressesInASG returns a list of private IP addresses for -# instances in a running or pending state which are members of the given -# autoscaling group, sorted in ascending order of launch time (oldest first). -# -# Parameters: -# $1: the name of the region in which the ASG exists -# $2: the name of the autoscaling group -function getRunningInstanceAddressesInASG() { - local region=$1 - local asg_name=$2 - local metadata_base_url="http://169.254.169.254/latest/meta-data" - - local instances_in_asg - local running_instances - - instances_in_asg=$(aws autoscaling describe-auto-scaling-groups \ - --region "${region}" \ - --auto-scaling-group-names="${asg_name}" \ - --query "AutoScalingGroups[0].Instances[*].{InstanceId:InstanceId}" \ - --output text) - - #shellcheck disable=SC2086 - running_instances=$(aws ec2 describe-instance-status \ - --region "${region}" \ - --instance-ids ${instances_in_asg} \ - --filter "Name=instance-state-name,Values=pending,running" \ - --query "InstanceStatuses[*].InstanceId" \ - --output text) - - #shellcheck disable=SC2086 - aws ec2 describe-instances \ - --region "${region}" \ - --instance-ids ${running_instances} \ - --query "Reservations[*].Instances[*].{LaunchTime:LaunchTime,PrivateIpAddress:PrivateIpAddress}" \ - --output text | sort -s -n -k 1,1 | cut -f 2 -s -} - -function findOtherNomadNode() { - local this_ip=$1 - - curl --silent "http://127.0.0.1:4646/v1/agent/members" \ - | jq -M -r '.Members[] | .Addr' \ - | cut -d ':' -f1 \ - | grep -v "${this_ip}" \ - | head -n 1 -} - -# getNomadRaftPeers returns a list of the IP addresses of the Nomad servers -# according to the /v1/agent/members endpoint. We always query the local Nomad -# agent to find who to talk to. -function getNomadRaftPeers() { - local to_ask=$1 - - curl --silent "http://${to_ask}:4646/v1/agent/members" \ - | jq -M -r '.Members[] | .Addr' \ - | cut -d ':' -f1 -} - -# forceLeaveRaftPeer force leaves the given node from the Nomad cluster. -# -# Parameters: -# $1: The address of the node to force leave -function forceLeaveRaftPeer() { - local node_address_to_leave=$1 - - local node_id_to_leave - - node_id_to_leave=$(curl --silent "http://${to_talk_to}:4646/v1/agent/members" \ - | jq -M -r ".Members[] | select(.Addr == \"${node_address_to_leave}\") | .Name") - - if [ ! -z "${node_id_to_leave}" ] ; then - /usr/bin/nomad server-force-leave ${node_id_to_leave} - fi -} - -# forceLeaveOldServers compares the list of current Nomad raft peers obtained -# via the /v1/agent/members endpoint with the list of instances which -# are running or pending in the autoscaling group, and force leaves any servers -# which are not running or pending. We use the local Nomad Agent for queries -# and to carry out the force-leave operation. -# -# Parameters: -# None -function forceLeaveOldServers() { - local this_instance_region - local this_asg_name - local this_ip_address - local to_talk_to - - this_instance_region=$(getMyRegion) - this_ip_address=$(getMyIPAddress) - this_asg_name=$(getMyASGName "${this_instance_region}") - - to_talk_to=$(findOtherNomadNode "${this_ip_address}") - nomad_raft_peers=$(getNomadRaftPeers "${to_talk_to}") - instances_in_asg=$(getRunningInstanceAddressesInASG "${this_instance_region}" "${this_asg_name}") - -echo "here" - - for peer in ${nomad_raft_peers}; do - if [ -z "$(echo "${instances_in_asg}" | grep "${peer}")" ] ; then - echo "Force leaving ${peer} from Nomad..." - forceLeaveRaftPeer "${peer}" - fi - done -} - -forceLeaveOldServers diff --git a/shared/nomad/init/systemd/nomad-online.service b/shared/nomad/init/systemd/nomad-online.service deleted file mode 100644 index d3e0a03e..00000000 --- a/shared/nomad/init/systemd/nomad-online.service +++ /dev/null @@ -1,13 +0,0 @@ -[Unit] -Description=Nomad Online -Requires=nomad.service -After=nomad.service - -[Service] -Type=oneshot -ExecStart=/usr/bin/nomad-online.sh -User=nomad -Group=nomad - -[Install] -WantedBy=nomad-online.target multi-user.target diff --git a/shared/nomad/init/systemd/nomad-online.sh b/shared/nomad/init/systemd/nomad-online.sh deleted file mode 100755 index 28bb6ae7..00000000 --- a/shared/nomad/init/systemd/nomad-online.sh +++ /dev/null @@ -1,22 +0,0 @@ -#!/usr/bin/env bash - -NOMAD_ADDRESS=${1:-"127.0.0.1:4646"} - -# waitForNomadToBeAvailable loops until the local Nomad agent returns a 200 -# response at the /v1/status/leader endpoint. -# -# Parameters: -# None -function waitForNomadToBeAvailable() { - local nomad_addr=$1 - local nomad_leader_http_code - - nomad_leader_http_code=$(curl --silent --output /dev/null --write-out "%{http_code}" "${nomad_addr}/v1/status/leader") || nomad_leader_http_code="" - - while [ "x${nomad_leader_http_code}" != "x200" ] ; do - sleep 5 - nomad_leader_http_code=$(curl --silent --output /dev/null --write-out "%{http_code}" "${nomad_addr}/v1/status/leader") || nomad_leader_http_code="" - done -} - -waitForNomadToBeAvailable "${NOMAD_ADDRESS}" diff --git a/shared/nomad/init/systemd/nomad-online.target b/shared/nomad/init/systemd/nomad-online.target deleted file mode 100644 index 4caa2a56..00000000 --- a/shared/nomad/init/systemd/nomad-online.target +++ /dev/null @@ -1,3 +0,0 @@ -[Unit] -Description=Nomad Online -RefuseManualStart=true diff --git a/shared/nomad/init/systemd/nomad-vault.service b/shared/nomad/init/systemd/nomad-vault.service deleted file mode 100644 index e0916f7d..00000000 --- a/shared/nomad/init/systemd/nomad-vault.service +++ /dev/null @@ -1,17 +0,0 @@ -[Unit] -Description=Nomad Agent -Requires=consul-online.target vault-token-ready.target -After=consul-online.target vault-token-ready.target - -[Service] -EnvironmentFile=/secrets/nomad-server-token -PermissionsStartOnly=true -Restart=on-failure -ExecStart=/usr/bin/nomad agent -config /etc/nomad -ExecReload=/bin/kill -HUP $MAINPID -KillSignal=SIGTERM -User=root -Group=root - -[Install] -WantedBy=multi-user.target diff --git a/shared/nomad/init/systemd/nomad.service b/shared/nomad/init/systemd/nomad.service deleted file mode 100644 index 70b0eb0e..00000000 --- a/shared/nomad/init/systemd/nomad.service +++ /dev/null @@ -1,15 +0,0 @@ -[Unit] -Description=Nomad Agent -Requires=consul-online.target -After=consul-online.target - -[Service] -Restart=on-failure -ExecStart=/usr/local/bin/nomad agent -config /etc/nomad.d -ExecReload=/bin/kill -HUP $MAINPID -KillSignal=SIGTERM -User=root -Group=root - -[Install] -WantedBy=multi-user.target diff --git a/shared/nomad/init/systemd/vault-token-ready.service b/shared/nomad/init/systemd/vault-token-ready.service deleted file mode 100644 index 2e8cbf7e..00000000 --- a/shared/nomad/init/systemd/vault-token-ready.service +++ /dev/null @@ -1,11 +0,0 @@ -[Unit] -Description=Vault Introduction Complete -Requires=vault-secure-intro.service -After=vault-secure-intro.service - -[Service] -Type=oneshot -ExecStart=/usr/bin/vault-token-ready.sh - -[Install] -WantedBy=vault-ready.target multi-user.target diff --git a/shared/nomad/init/systemd/vault-token-ready.sh b/shared/nomad/init/systemd/vault-token-ready.sh deleted file mode 100755 index b365ea3a..00000000 --- a/shared/nomad/init/systemd/vault-token-ready.sh +++ /dev/null @@ -1,17 +0,0 @@ -#!/usr/bin/env bash - -set -e -set -o pipefail - -function waitForVaultToken() { - local path=$1 - - while [ ! -s "${path}" ] ; do - echo "Waiting for file..." - sleep 1 - done - - echo "File found." -} - -waitForVaultToken "/secrets/nomad-server-token" diff --git a/shared/nomad/init/systemd/vault-token-ready.target b/shared/nomad/init/systemd/vault-token-ready.target deleted file mode 100644 index 3d8cd444..00000000 --- a/shared/nomad/init/systemd/vault-token-ready.target +++ /dev/null @@ -1,3 +0,0 @@ -[Unit] -Description=Vault Introduction Complete -RefuseManualStart=true diff --git a/shared/nomad/scripts/install-docker.sh b/shared/nomad/scripts/install-docker.sh deleted file mode 100644 index a545973a..00000000 --- a/shared/nomad/scripts/install-docker.sh +++ /dev/null @@ -1,28 +0,0 @@ -#!/bin/bash -set -x - -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -logger "Running" - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -if [[ ! -z ${YUM} ]]; then - echo "Installing Docker with RHEL Workaround" - sudo yum-config-manager -y --add-repo https://download.docker.com/linux/centos/docker-ce.repo - sudo yum install -y docker-ce -elif [[ ! -z ${APT_GET} ]]; then - echo "Installing Docker" - curl -sSL https://get.docker.com/ | sudo sh -else - logger "Prerequisites not installed due to OS detection failure" - exit 1; -fi - -sudo sh -c "echo \"DOCKER_OPTS='--dns 127.0.0.1 --dns 8.8.8.8 --dns-search service.consul'\" >> /etc/default/docker" - -logger "Complete" diff --git a/shared/nomad/scripts/install-nomad-systemd.sh b/shared/nomad/scripts/install-nomad-systemd.sh deleted file mode 100644 index 1e9ea4f3..00000000 --- a/shared/nomad/scripts/install-nomad-systemd.sh +++ /dev/null @@ -1,42 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -if [[ ! -z ${YUM} ]]; then - SYSTEMD_DIR="/etc/systemd/system" - logger "Installing systemd services for RHEL/CentOS" - sudo cp /tmp/nomad/init/systemd/nomad.service ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.service ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.target ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.sh /usr/bin/consul-online.sh - sudo chmod 0664 ${SYSTEMD_DIR}/{nomad*,consul*} -elif [[ ! -z ${APT_GET} ]]; then - SYSTEMD_DIR="/lib/systemd/system" - logger "Installing systemd services for Debian/Ubuntu" - sudo cp /tmp/nomad/init/systemd/nomad.service ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.service ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.target ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.sh /usr/bin/consul-online.sh - sudo chmod 0664 ${SYSTEMD_DIR}/{nomad*,consul*} -else - logger "Service not installed due to OS detection failure" - exit 1; -fi - -sudo systemctl enable consul -sudo systemctl start consul - -sudo systemctl enable nomad -sudo systemctl start nomad - -logger "Complete" diff --git a/shared/nomad/scripts/install-nomad.sh b/shared/nomad/scripts/install-nomad.sh deleted file mode 100644 index 11b0abd6..00000000 --- a/shared/nomad/scripts/install-nomad.sh +++ /dev/null @@ -1,32 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -NOMAD_VERSION="${VERSION}" -NOMAD_ZIP="nomad_${NOMAD_VERSION}_linux_amd64.zip" -NOMAD_URL=${URL:-"https://releases.hashicorp.com/nomad/${NOMAD_VERSION}/${NOMAD_ZIP}"} - -logger "Downloading nomad ${NOMAD_VERSION}" -curl --silent --output /tmp/${NOMAD_ZIP} ${NOMAD_URL} - -logger "Installing nomad" -sudo unzip -o /tmp/${NOMAD_ZIP} -d /usr/local/bin/ -sudo chmod 0755 /usr/local/bin/nomad -sudo chown root:root /usr/local/bin/nomad -sudo mkdir -pm 0755 /etc/nomad.d -sudo mkdir -pm 0755 /opt/nomad/data - -logger "/usr/local/bin/nomad --version: $(/usr/local/bin/nomad --version)" - -logger "Configuring nomad ${NOMAD_VERSION}" -sudo cp /tmp/nomad/config/* /etc/nomad.d/ -sudo chown -R root:root /etc/nomad.d /opt/nomad -sudo chmod -R 0644 /etc/nomad.d/* - -logger "Complete" diff --git a/shared/nomad/scripts/install-oracle-jdk.sh b/shared/nomad/scripts/install-oracle-jdk.sh deleted file mode 100644 index fe791e06..00000000 --- a/shared/nomad/scripts/install-oracle-jdk.sh +++ /dev/null @@ -1,52 +0,0 @@ -#!/bin/bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -download_jdk() { - local JDK_VERSION="$1" - local EXT="$2" - readonly URL="http://www.oracle.com" - readonly JDK_DOWNLOAD_URL1="${URL}/technetwork/java/javase/downloads/index.html" - readonly JDK_DOWNLOAD_URL2=$(curl -s ${JDK_DOWNLOAD_URL1} | egrep -o "\/technetwork\/java/\javase\/downloads\/jdk${JDK_VERSION}-downloads-.+?\.html" | head -1 | cut -d '"' -f 1) - [[ -z "${JDK_DOWNLOAD_URL2}" ]] && logger "Could not get jdk download url - ${JDK_DOWNLOAD_URL1}" && exit 1 - readonly JDK_DOWNLOAD_URL3="${URL}${JDK_DOWNLOAD_URL2}" - readonly JDK_DOWNLOAD_URL4=$(curl -s ${JDK_DOWNLOAD_URL3} | egrep -o "http\:\/\/download.oracle\.com\/otn-pub\/java\/jdk\/[7-8]u[0-9]+\-(.*)+\/jdk-[7-8]u[0-9]+(.*)linux-x64.${EXT}") - for DL_URL in "${JDK_DOWNLOAD_URL4[@]}"; do - wget --no-cookies --no-check-certificate --header "Cookie: oraclelicense=accept-securebackup-cookie" -N ${DL_URL} - done -} - -logger "Running" - -logger "Installing Oracle JDK" - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -if [[ ! -z ${YUM} ]]; then - logger "RHEL/CentOS system detected" - download_jdk 8 rpm - sudo rpm -Uvh jdk-*-linux-x64.rpm -elif [[ ! -z ${APT_GET} ]]; then - logger "Debian/Ubuntu system detected" - download_jdk 8 tar.gz - sudo mkdir -p /opt/jdk - sudo tar xf jdk-*-linux-x64.tar.gz -C /opt/jdk - JDK_VERSION="$(ls /opt/jdk/)" - sudo update-alternatives --install /usr/bin/java java /opt/jdk/${JDK_VERSION}/bin/java 2000 - sudo update-alternatives --install /usr/bin/javac javac /opt/jdk/${JDK_VERSION}/bin/javac 2000 - sudo update-alternatives --install /usr/bin/jar jar /opt/jdk/${JDK_VERSION}/bin/jar 2000 - update-alternatives --display java - update-alternatives --display javac - update-alternatives --display jar -else - logger "Oracle JDK not installed due to OS detection failure" - exit 1; -fi - -logger "Complete" diff --git a/shared/scripts/base-aws.sh b/shared/scripts/base-aws.sh deleted file mode 100644 index 792b5f74..00000000 --- a/shared/scripts/base-aws.sh +++ /dev/null @@ -1,31 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -if [[ ! -z ${YUM} ]]; then - logger "RHEL/CentOS system detected" - logger "Performing updates and installing prerequisites" - curl --silent -O https://bootstrap.pypa.io/get-pip.py - sudo python get-pip.py - sudo pip install awscli -elif [[ ! -z ${APT_GET} ]]; then - logger "Debian/Ubuntu system detected" - logger "Performing updates and installing prerequisites" - sudo apt-get -qq -y update - sudo apt-get install -qq -y awscli -else - logger "Prerequisites not installed due to OS detection failure" - exit 1; -fi - -logger "Complete" diff --git a/shared/scripts/base-azure.sh b/shared/scripts/base-azure.sh deleted file mode 100644 index e958632f..00000000 --- a/shared/scripts/base-azure.sh +++ /dev/null @@ -1,32 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -if [[ ! -z ${YUM} ]]; then - logger "RHEL/CentOS system detected" - logger "Performing updates and installing prerequisites" - sudo yum check-update - sudo yum install -q -y gcc libffi-devel python-devel openssl-devel python-pip - sudo pip install azure-cli -elif [[ ! -z ${APT_GET} ]]; then - logger "Debian/Ubuntu system detected" - logger "Performing updates and installing prerequisites" - sudo apt-get -qq -y update - sudo apt-get install -qq -y libssl-dev libffi-dev python-dev build-essential python-pip - sudo pip install azure-cli -else - logger "Prerequisites not installed due to OS detection failure" - exit 1; -fi - -logger "Complete" diff --git a/shared/scripts/base.sh b/shared/scripts/base.sh deleted file mode 100644 index 8a86cdfa..00000000 --- a/shared/scripts/base.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -logger "Installing jq" -sudo curl --silent -Lo /bin/jq https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64 -sudo chmod +x /bin/jq - -logger "Setting timezone to UTC" -sudo timedatectl set-timezone UTC - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -if [[ ! -z ${YUM} ]]; then - logger "RHEL/CentOS system detected" - logger "Performing updates and installing prerequisites" - sudo yum-config-manager --enable rhui-REGION-rhel-server-releases-optional - sudo yum-config-manager --enable rhui-REGION-rhel-server-supplementary - sudo yum-config-manager --enable rhui-REGION-rhel-server-extras - sudo yum -y check-update - sudo yum install -q -y wget unzip bind-utils ruby rubygems ntp - sudo systemctl start ntpd.service - sudo systemctl enable ntpd.service -elif [[ ! -z ${APT_GET} ]]; then - logger "Debian/Ubuntu system detected" - logger "Performing updates and installing prerequisites" - sudo apt-get -qq -y update - sudo apt-get install -qq -y wget unzip dnsutils ruby rubygems ntp - sudo systemctl start ntp.service - sudo systemctl enable ntp.service - logger "Disable reverse dns lookup in SSH" - sudo sh -c 'echo "\nUseDNS no" >> /etc/ssh/sshd_config' - sudo service ssh restart -else - logger "Prerequisites not installed due to OS detection failure" - exit 1; -fi - -logger "Complete" diff --git a/shared/scripts/cleanup-aws.sh b/shared/scripts/cleanup-aws.sh deleted file mode 100644 index 8278c872..00000000 --- a/shared/scripts/cleanup-aws.sh +++ /dev/null @@ -1,15 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -logger "Cleanup AWS install artifacts" -sudo rm -rf /var/lib/cloud/instances/* -sudo rm -f /root/.ssh/authorized_keys - -logger "Complete" diff --git a/shared/scripts/cleanup.sh b/shared/scripts/cleanup.sh deleted file mode 100644 index fd9095ac..00000000 --- a/shared/scripts/cleanup.sh +++ /dev/null @@ -1,36 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -logger "Reset HashiCorp services" -[[ -f "/usr/local/bin/consul" ]] && sudo systemctl stop consul && sudo rm -rf /opt/consul/data/* -[[ -f "/usr/local/bin/nomad" ]] && sudo systemctl stop nomad && sudo rm -rf /opt/nomad/data/* -[[ -f "/usr/local/bin/vault" ]] && sudo systemctl stop vault && sudo rm -rf /opt/vault/data/* - -logger "Cleanup install artifacts" -sudo rm -rf /tmp/* - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -if [[ ! -z ${YUM} ]]; then - logger "RHEL/CentOS system detected" - logger "Performing cleanup" - history -cw -elif [[ ! -z ${APT_GET} ]]; then - logger "Debian/Ubuntu system detected" - logger "Performing cleanup" - history -c -else - logger "Cleanup aborted due to OS detection failure" - exit 1; -fi - -logger "Complete" diff --git a/shared/scripts/disable-firewall.sh b/shared/scripts/disable-firewall.sh deleted file mode 100644 index 4edd8c3f..00000000 --- a/shared/scripts/disable-firewall.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -if [[ ! -z ${YUM} ]]; then - logger "RHEL/CentOS system detected" - logger "Disabling firewall" - sudo systemctl stop firewalld.service - sudo systemctl disable firewalld.service -elif [[ ! -z ${APT_GET} ]]; then - logger "Debian/Ubuntu system detected" - logger "Disabling firewall" - sudo ufw disable -else - logger "Prerequisites not installed due to OS detection failure" - exit 1; -fi - -logger "Complete" diff --git a/shared/scripts/setup-testing.sh b/shared/scripts/setup-testing.sh deleted file mode 100644 index 3bfae1ee..00000000 --- a/shared/scripts/setup-testing.sh +++ /dev/null @@ -1,13 +0,0 @@ -#!/usr/bin/env bash - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -sudo gem install bundler --no-ri --no-rdoc -sudo /usr/local/bin/bundle install --system - -logger "Complete" diff --git a/shared/scripts/setup-user.sh b/shared/scripts/setup-user.sh deleted file mode 100644 index 8a3ce537..00000000 --- a/shared/scripts/setup-user.sh +++ /dev/null @@ -1,68 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -USER="${1}" -COMMENT="Hashicorp ${1} user" -GROUP="${1}" -HOME="/srv/${1}" - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -user_rhel() { - # RHEL user setup - sudo /usr/sbin/groupadd --force --system ${GROUP} - - if ! getent passwd ${USER} >/dev/null ; then - sudo /usr/sbin/adduser \ - --system \ - --gid ${GROUP} \ - --home ${HOME} \ - --no-create-home \ - --comment "${COMMENT}" \ - --shell /bin/false \ - ${USER} >/dev/null - fi -} - -user_ubuntu() { - # UBUNTU user setup - if ! getent group ${GROUP} >/dev/null - then - sudo addgroup --system ${GROUP} >/dev/null - fi - - if ! getent passwd ${USER} >/dev/null - then - sudo adduser \ - --system \ - --disabled-login \ - --ingroup ${GROUP} \ - --home ${HOME} \ - --no-create-home \ - --gecos "${COMMENT}" \ - --shell /bin/false \ - ${USER} >/dev/null - fi -} - -if [[ ! -z ${YUM} ]]; then - logger "Setting up user ${USER} for RHEL/CentOS" - user_rhel -elif [[ ! -z ${APT_GET} ]]; then - logger "Setting up user ${USER} for Debian/Ubuntu" - user_ubuntu -else - logger "${USER} user not created due to OS detection failure" - exit 1; -fi - -logger "Complete" diff --git a/shared/vault/config/vault-consul.hcl b/shared/vault/config/vault-consul.hcl deleted file mode 100644 index fd23c13a..00000000 --- a/shared/vault/config/vault-consul.hcl +++ /dev/null @@ -1,4 +0,0 @@ -backend "consul" { - address = "127.0.0.1:8500" - path = "vault/" -} diff --git a/shared/vault/config/vault-no-tls.hcl b/shared/vault/config/vault-no-tls.hcl deleted file mode 100644 index b609782c..00000000 --- a/shared/vault/config/vault-no-tls.hcl +++ /dev/null @@ -1,5 +0,0 @@ -listener "tcp" { - address = "0.0.0.0:8200" - tls_disable = 1 -} -ui=true diff --git a/shared/vault/config/vault-tls.hcl.example b/shared/vault/config/vault-tls.hcl.example deleted file mode 100644 index 6020160b..00000000 --- a/shared/vault/config/vault-tls.hcl.example +++ /dev/null @@ -1,6 +0,0 @@ -listener "tcp" { - address = "0.0.0.0:8200" - tls_disable = 0 - tls_cert_file = "/etc/ssl/vault/vault.crt" - tls_key_file = "/etc/ssl/vault/vault.key" -} diff --git a/shared/vault/config/vault-ui.hcl.example b/shared/vault/config/vault-ui.hcl.example deleted file mode 100644 index 334c13ad..00000000 --- a/shared/vault/config/vault-ui.hcl.example +++ /dev/null @@ -1 +0,0 @@ -ui = true \ No newline at end of file diff --git a/shared/vault/init/systemd/vault-secure-intro.service b/shared/vault/init/systemd/vault-secure-intro.service deleted file mode 100644 index b60c68b8..00000000 --- a/shared/vault/init/systemd/vault-secure-intro.service +++ /dev/null @@ -1,15 +0,0 @@ -[Unit] -Description=Vault Secure Introduction -Requires=network-online.target -After=network-online.target - -[Service] -Restart=on-failure -ExecStart=/usr/bin/vault-secure-intro -config /etc/vault-si/vault-si.hcl -ExecReload=/bin/kill -HUP $MAINPID -KillSignal=SIGTERM -User=vault-si -Group=vault-si - -[Install] -WantedBy=multi-user.target diff --git a/shared/vault/init/systemd/vault.service b/shared/vault/init/systemd/vault.service deleted file mode 100644 index ee2cd48f..00000000 --- a/shared/vault/init/systemd/vault.service +++ /dev/null @@ -1,17 +0,0 @@ -[Unit] -Description=Vault Agent -Requires=consul-online.target -After=consul-online.target - -[Service] -Restart=on-failure -PermissionsStartOnly=true -ExecStartPre=/sbin/setcap 'cap_ipc_lock=+ep' /usr/local/bin/vault -ExecStart=/usr/local/bin/vault server -config /etc/vault.d -ExecReload=/bin/kill -HUP $MAINPID -KillSignal=SIGTERM -User=vault -Group=vault - -[Install] -WantedBy=multi-user.target diff --git a/shared/vault/scripts/install-vault-systemd.sh b/shared/vault/scripts/install-vault-systemd.sh deleted file mode 100644 index f7c2fd6f..00000000 --- a/shared/vault/scripts/install-vault-systemd.sh +++ /dev/null @@ -1,42 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -# Detect package management system. -YUM=$(which yum 2>/dev/null) -APT_GET=$(which apt-get 2>/dev/null) - -if [[ ! -z ${YUM} ]]; then - SYSTEMD_DIR="/etc/systemd/system" - logger "Installing systemd services for RHEL/CentOS" - sudo cp /tmp/vault/init/systemd/vault.service ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.service ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.target ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.sh /usr/bin/consul-online.sh - sudo chmod 0664 ${SYSTEMD_DIR}/{vault*,consul*} -elif [[ ! -z ${APT_GET} ]]; then - SYSTEMD_DIR="/lib/systemd/system" - logger "Installing systemd services for Debian/Ubuntu" - sudo cp /tmp/vault/init/systemd/vault.service ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.service ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.target ${SYSTEMD_DIR} - sudo cp /tmp/consul/init/systemd/consul-online.sh /usr/bin/consul-online.sh - sudo chmod 0664 ${SYSTEMD_DIR}/{vault*,consul*} -else - logger "Service not installed due to OS detection failure" - exit 1; -fi - -sudo systemctl enable consul -sudo systemctl start consul - -sudo systemctl enable vault -sudo systemctl start vault - -logger "Complete" diff --git a/shared/vault/scripts/install-vault.sh b/shared/vault/scripts/install-vault.sh deleted file mode 100644 index d407a70f..00000000 --- a/shared/vault/scripts/install-vault.sh +++ /dev/null @@ -1,37 +0,0 @@ -#!/usr/bin/env bash -set -x - -logger() { - DT=$(date '+%Y/%m/%d %H:%M:%S') - echo "$DT $0: $1" -} - -logger "Running" - -VAULT_VERSION="${VERSION}" -VAULT_ZIP="vault_${VAULT_VERSION}_linux_amd64.zip" -VAULT_URL=${URL:-"https://releases.hashicorp.com/vault/${VAULT_VERSION}/${VAULT_ZIP}"} - -logger "Downloading vault ${VAULT_VERSION}" -curl --silent --output /tmp/${VAULT_ZIP} ${VAULT_URL} - -logger "Installing vault" -sudo unzip -o /tmp/${VAULT_ZIP} -d /usr/local/bin/ -sudo chmod 0755 /usr/local/bin/vault -sudo chown vault:vault /usr/local/bin/vault -sudo mkdir -pm 0755 /etc/vault.d -sudo mkdir -pm 0755 /etc/ssl/vault - -logger "/usr/local/bin/vault --version: $(/usr/local/bin/vault --version)" - -logger "Configuring vault ${VAULT_VERSION}" -sudo cp /tmp/vault/config/* /etc/vault.d -sudo chown -R vault:vault /etc/vault.d /etc/ssl/vault -sudo chmod -R 0644 /etc/vault.d/* -echo "export VAULT_ADDR=http://127.0.0.1:8200" | sudo tee /etc/profile.d/vault.sh - - -logger "Granting mlock syscall to vault binary" -sudo setcap cap_ipc_lock=+ep /usr/local/bin/vault - -logger "Complete"