Skip to content

Foster Family
Compare
Choose a tag to compare
@jstange jstange released this 12 Nov 04:38
· 1121 commits to master since this release
v3.0.0
9b21149
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Microsoft Azure Support

Azure is now a supported cloud layer, currently supporting the VPC, FirewallRule, Server, ContainerCluster at beta quality. Azure is a valid target environment for Mu Masters.

To add credentials from an Azure subscription to a Mu environment, use mu-configure. See also the documentation for adding Azure credentials.

Ansible Groomer

Ansible is now a supported groomer layer. Setting groomer: Ansible in a Server or ServerPool will enable it for a node. Ansible is the default groomer for gem-only installations.

Mu deploys will automatically generate playbooks from the run_list parameter of appropriate resources, which will be invoked on groom (e.g. by mu-node-manage -m groom). Mu will look in the roles/ subdirectory of platform repositories for Ansible roles; if no matches are found, it can also query Ansible Galaxy for matching roles.

Example from the Mu demo platform repository:

---
appname: janusgraph
cloud: Google
servers:
- name: dbstore
  groomer: Ansible
  associate_public_ip: true
  platform: debian9
  size: n1-standard-2
  run_list:
  - janusgraph
  storage:
  - size: 64 # swap
    device: /dev/sdb
  - size: 512 # /data
    device: /dev/sdc

...the janusgraph role can be found here. To try out this demonstration, make sure that https://github.com/cloudamatic/mu_demo_platform is listed in your Additional Repositories with mu-configure.

Adoption

A new utility called mu-adopt is now available for alpha-testing in Google Cloud. This functionality can crawl existing cloud environments, identify resources that can be managed by Mu, and import them into Mu Baskets of Kittens and/or virtual deployments. This allows, effectively, for Mu to take over management of resources that were created manually, with other tools, or by other Mu installations.

$ mu-adopt --types vpcs
Oct 29 15:15:03 - adoption - Scraping Google/egtlabs for vpcs
Oct 29 15:15:26 - adoption - Found 5 raw vpcs in Google
Oct 29 15:15:26 - adoption - Scraping complete
Oct 29 15:15:26 - mu-adopt - Generating baskets
Oct 29 15:15:31 - adoption - Minimizing footprint of 4 found resources
Oct 29 15:15:32 - mommacat - Creating deploy secret for MUNETWORK-ADOPT-2019102915-QP
Oct 29 15:15:32 - adoption - Committing adopted deployment to /opt/mu/var/deployments/MUNETWORK-ADOPT-2019102915-QP
Oct 29 15:15:32 - mu-adopt - Writing to munetwork.yaml

$ cat munetwork.yaml
---
appname: munetwork
vpcs:
- project: egt-labs-admin
  name: k8s-dev-2019091813-si-googlevpc-internet
  create_standard_subnets: false
  auto_create_subnetworks: true
  peers:
  - vpc:
      type: vpcs
      name: k8s-dev-2019091813-si-googlevpc-private
      habitat:
        id: egt-labs-admin
        type: habitats
- project: egt-labs-admin
  name: k8s-dev-2019091813-si-googlevpc-private
  create_standard_subnets: false
  auto_create_subnetworks: true
  peers:
  - vpc:
      type: vpcs
      name: k8s-dev-2019091813-si-googlevpc-internet
      habitat:
        id: egt-labs-admin
        type: habitats
- project: egt-labs-admin
  name: k8s-dev-2019091814-ou-googlevpc-internet
  create_standard_subnets: false
  auto_create_subnetworks: true
  peers:
  - vpc:
      type: vpcs
      name: k8s-dev-2019091814-ou-googlevpc-private
      habitat:
        id: egt-labs-admin
        type: habitats
- project: egt-labs-admin
  name: k8s-dev-2019091814-ou-googlevpc-private
  create_standard_subnets: false
  auto_create_subnetworks: true
cloud: Google
credentials: egtlabs

mu-adopt also has a --diff mode which will search for an existing adopted deployment, and report changes to resources under that deployment's purview for use in auditing.

$ mu-adopt --types users groups roles --diff
Oct 29 17:16:46 - adoption - Scraping Google/egtlabs for users
Oct 29 17:16:54 - adoption - Found 17 raw users in Google
Oct 29 17:16:54 - adoption - Scraping Google/egtlabs for groups
Oct 29 17:16:57 - adoption - Found 108 raw groups in Google
Oct 29 17:16:57 - adoption - Scraping Google/egtlabs for roles
Oct 29 17:17:26 - adoption - Found 29 raw roles in Google
groups =>
  gcp-admin =>
    members =>
      + [email protected]

Amazon Linux 2 supported for Mu Masters

Full Mu Master installs are now supported on Amazon Linux 2, in addition to CentOS/RHEL 6 and 7, on AWS.

Improved ContainerCluster support

Google - GKE support is now more robust, with largely complete API coverage

AWS - More resilient coverage for EKS, broader support for Fargate, and LoadBalancer integration for ECS

Azure - AKS support added

Example syntax for Fargate/ECS with LoadBalancer integration:

---
appname: fargate
cloud: AWS
vpcs:
- name: vpc
loadbalancers:
- name: ecs-lb
  vpc:
    vpc_name: vpc
    subnet_pref: all_public
  listeners:
  - lb-port: 80
    lb-protocol: HTTP
    instance-port: 80
    instance-protocol: HTTP
    targetgroup: httpd
  targetgroups:
  - name: httpd
    port: 80
    proto: HTTP
    target_type: ip
  ingress_rules:
  - port: 80
    hosts:
    - 0.0.0.0/0
container_clusters:
- name: containers
  flavor: Fargate
  instance_type: t2.medium
  ingress_rules:
  - port: 80
    hosts:
    - 0.0.0.0/0
  dependencies:
  - name: ecs-lb
    type: loadbalancer
  containers:
  - name: reverse-proxy
    image: ryantiger658/httpd-demo:latest
    instance_count: 4
    port_mappings:
    - container_port: 80
      host_port: 80
    essential: true
    loadbalancers:
    - name: ecs-lb
      container_port: 80

Google Role/User/Group coverage improvements

The Google User, Role, and Group resources have been enhanced significantly. They should now operate in GSuite domains seamlessly, as well as "regular" Google accounts and service accounts. Other resources which require User or Role objects now do so by creating first-class sibling objects.

See the mu-configure documentation for details on configuring Mu for use with a GSuite domain.

Quality of life enhancements

  • Automated the generation of our custom base images for AWS
  • A new Basket of Kittens is available in install/mu-master.yaml, which can be used to deploy a full Mu Master into a specific cloud. This can be run from a gem-only installation.
  • Ansible groomer now presents more metadata to client nodes
  • MU.availableClouds method is available for use in ERB blocks in BoKs to list cloud platforms for which Mu has working credentials
  • When auto-generating VPCs with private subnets, Mu will attempt to allocate an IP block which can be peered with Mu's own VPC (if applicable). Failing this, it will instead attempt to build a bastion host.
  • Automatically-generated NATs/Bastions use the mu-nat Ansible role, and so are no longer dependent on a Chef installation.
  • If Slack notifications are configured, messages are now sent for node grooms, groom failures, and node retirements
  • All parameters in BoKs should now be properly bound to local variables visible to ERB meta-code
  • Improved config-time validation of some AWS RDS parameters
  • major refactoring of repetitive cloud layer resource implementation code into MU::Cloud
  • internal calls to MU::Cloud child objects no longer have to know whether they're calling a wrapper object or the underlying resource object

Bugfixes

  • We should be able to scale effectively now for extremely large deploys (hundreds of resources)
  • Speed improvements for MU::Config parsing
  • Fixes for edge cases in dependency resolution
  • mu-ssh should now behave correctly in gem installs
  • Workarounds for Mu Master environment issues on CentOS 6
  • mu-upload-chef-artifacts should be better at helping Berkshelf when the latter gets confused
  • Generally improved multicloud support
  • Google layer should seamlessly paginate long responses from most APIs
  • scrub_mu_isms honored somewhat more often/correctly
  • mu-gen-docs should be less fragile when generating documentation for cloud layers to which it has no credentials
  • AWS VPC cleanup should be better at handling rogue manually-created resources that prevent deletion
  • Bump versions of externally-sourced gems with security advisories
  • Automatic installation/discovery of kubectl for use with ContainerCluster resources is no longer AWS-specific
  • Fixed some API bitrot for ElastiCache and ElasticSearch
Assets 2
Loading