Skip to content

Directory Services Release
Compare
Choose a tag to compare
@rpattcorner rpattcorner released this 09 Dec 21:13
· 4052 commits to master since this release
v1.1
1075539

/opt/mu/etc/mu.yaml

We've needed a more sophisticated config for masters for a while now, so here one is. This is intended to eventually replace mu.rc. For now they exist in parallel, pending @zr2d2's installer and utility work. The new LDAP functionality and mu-user-manage obey this file.

The first time one of them is invoked, it will generate a sample version based on your current mu.rc. Here's FEMA's, minus the LDAP section (more on that later):

---
  mu_admin_email: [email protected]
  jenkins_admin_email: [email protected]
  public_address: mu.femadata.com
  hostname: femadata-mu
  allow_invade_foreign_vpcs: true
  installdir: /opt/mu
  datadir: /opt/mu/var
  ssl:
    cert: /opt/mu/var/ssl/fema.crt
    key: /opt/mu/var/ssl/fema.key
    chain: /opt/mu/var/ssl/godaddy_chain.crt
  scratchpad:
    template_path: /opt/mu/var/fema_platform/utils/includes/scratchpad.html
    max_age: 604800
  mu_repo: cloudamatic/mu.git
  repos:
  - eGT-Labs/fema_platform.git
  aws:
    account_number: 129261966611
    region: us-east-1
    log_bucket_name: mu-logs-production

Scratchpad

We needed a native one-time secret mechanism for sharing passwords and the like, so I made one up. It simply sticks a string in a vault with a random 32-character item name. Hitting a particular URL (/scratchpad/itemname on MommaCat) will display the secret and delete the item.

image

You can call the creation routine stuff from Ruby with MU::Master.storeScratchPadSecret(text), which returns the item name. If you're in the mu-master cookbook you can call it from Chef too, which I wager we'll find useful. Ditto the other end of it with MU::Master.fetchScratchPadSecret(itemname), if you need to grab a secret directly.

Self-signed SSL certificates now behave better

We were doing it wrong, leading to cranky browsers and utilities. Updating an existing Mu server will regenerate your internal certificates with correct ones. That will impact your existing nodes, but a knife ssl fetch should calm them down.

Cleaner URLs

Scratchpad made one too many random things listening on weird ports. I gave Apache control of port 443 and set it up as a reverse proxy as follows:

Unified authentication through LDAP

Mu users are now stored in a directory service. By default it's a bundled setup of 389 DS (http://directory.fedoraproject.org/), which is basically OpenLDAP with some tooling and schemas built around it. The software supports replication, which I bet we're going to want someday.

Nagios, Jenkins, and system users (your non-root ssh logins) all use the directory for authentication now. Chef's web UI could do it too, if we installed it.

As a corollary we get a fairly robust library under MU::Master::LDAP that can speak LDAP to this or to Active Directory. I've already built a nice FEMA tool off of it it.

This is only intended to be used for Mu's internal services, but in theory there's nothing stopping from using it as a general directory and authentication mechanism for your infrastructure. I don't know that we should recommend that, though.

Big deal: We can also slave to an Active Directory infrastructure instead of our bundled directory, which is how we're rolling in FEMA.

The default configuration of the ldap stanza of mu.yaml is going to be right for nearly everyone. If you want to customize, e.g. to dangle yourself off of an existing AD tree, it looks a bit like this:

  ldap:
    type: Active Directory
    base_dn: OU=FEMAData,DC=ad,DC=femadata,DC=com
    user_ou: OU=Users,OU=FEMAData,DC=ad,DC=femadata,DC=com
    bind_creds:
      vault: active_directory
      item: mu_svc_acct
      username_field: dn
      password_field: password
    join_creds:
      vault: active_directory
      item: join_domain
      username_field: username
      password_field: password
    domain_name: ad.femadata.com
    domain_netbios_name: femadata
    user_group_dn: CN=Mu-Users,OU=Groups,OU=Management,OU=FEMAData,DC=ad,DC=femadata,DC=com
    user_group_name: mu-users
    admin_group_dn: CN=Mu,OU=Groups,OU=Management,OU=FEMAData,DC=ad,DC=femadata,DC=com
    admin_group_name: mu
    dcs:
    - dc1.ad.femadata.com
    - dc2.ad.femadata.com

...obviously, all of the AD resources referenced there must exist and be properly configured, as must the vaults that store the sensitive credentials.

Note that if your bind user doesn't have write privileges to the relevant bits of AD, you won't be able to do much with Mu's tools to manage users... but things will still work in a read-only fashion. The assumption is that you're doing it through AD's management tools. This is a legit use case.

When using the inbuilt 389 DS, we're using SSSD instead of Winbind to integrate with PAM and friends. This is apparently the new hotness. We may want to consider porting that to the Linux AD recipes someday.

@amirahav's new AWS functionality

CloudWatch/CloudWatch Logs

We can now create CloudWatch alarms. Alarms can be created within each resource that supports it (eg. Databases, Servers, Server Pools, Load Balancers) or also as first class citizens in the BoK language itself. This should give you a better picture of the status of various resources. For example you can now get notified when the number of healthy instances attached to a load balancer is below a certain number, or configure AWS to "recover" you instance when it fails the System Status Check

CloudTrail monitoring was also added to CloudWatch directly. Assuming CloudTrial is enabled in your account you can enable CloudWatch integration by setting enable_cloudtrail_logging: true in the BoK. See example at demo/cloudwatch-logs-cloudtrail.yaml

ElasticCache

You can now create cache clusters and can choose between memcached and redis. See example at demo/cache_cluster.yaml

The ElasticCache API is pretty awful. To make sure cleanup doesn't hold up other deployments/cleanups we are loading an existing deployment instead of searching for resources directly. This can create a situation where not all resources are deleted.

RDS

You can now create database clusters using the Aurora engine. We will manufacture database instances for you at the configuration level based on the number of nodes specified in the cluster_node_count attribute (this resembles how Read Replicas are generated) . See example at demo/db_cluster.yaml

Databases can now be created based on a point in time state of an existing database instance. Support for point in time creation style varies between database engine types

Password management now works the same way it does on other resources that use passwords - you can choose between a randomly generated password or a password that is stored in a vault. The option to store a clear text password in the node structure was also removed. You can retrieve the password by using a vault only. These means that both password and unecrypted_master_password that where deprecated in Internal Wrangler have been removed.

Other changes:

  • Tags are now propagated to snapshots. This is not exposes at the BoK language level
  • Cleanup - Will now search for other RDS resources without first looking for a database instance
  • flattening of some BoK configuration attributes the were added in Internal Wrangler
  • EC2-Classic support was mostly removed. Cleanup should still work, but deployment is not supported anymore
  • Deployment into the default VPC should work better now, however the database instance will be set to be publicly accessible regardless of what publicly_accessible is set to.

FlowLogs

You can now enable traffic logging on a VPC and/or subnets by setting enable_traffic_logging: true on the VPC or subnets. If you enable it at the VPC level it will apply to all subnets and network interfaces. If you only want to enable logging on specific subnets, set this at the subnet level.

AutoScaling

With the addition of ClouWatch you can now create fully functioning scaling policies. You can configure multiple ClouWatch alarms to point at the same scaling policy to help you scale based on different criteria (eg. CPU usage and/or network traffic)

Instead of using the simple scaling policy type which adds/removes a fixed number of instances, you can now use step scaling which will add/remove capacity based on the magnitude of the breach. See example at demo/autoscale_step_scaling.yaml

SNS

This is a very rudimentary implementation of the API, and only includes the ability to create notification topics to make is possible for CloudWatch to send notification by email.

Changes to DNS

DNS registration in server/server pool now allows you to toggle between registering private or public addresses by using the optional target_type parameter in the BoK. This, in conjunction with a couple of bugfixes allows you to use either CNAME or A records correctly.

You may also append the environment name (eg. dev, prod, etc...) to the DNS name. This is useful when using environment specific certificates and still wanting to leverage the automated DNS naming to help with finding nodes using Chef.

    dns_records:
      - name: vpn
    <% unless $environment == "prod" %>
        append_environment_name: true
    <% end %>
        override_existing: true
        type: CNAME
        ttl: 300
        zone:
          name: boundlessgeo.com
      - ttl: 300
    <% unless $environment == "prod" %>
        append_environment_name: true
    <% end %>
        target_type: private
        zone:
          name: boundlessgeo.com

DNS registration can also be done now using the dnszones BoK stanza. You can now add records to an existing zone instead of only being able to do so using the inline dns_records attribute in each resource. This is helpful when replacing an existing deployment and not wanting to make DNS changes until the Chef run is complete.

    appname: test
    server_pools:
    - name: server
      -- some stuff --

    loadbalancers:
    - name: lb
      -- some stuff --

    dnszones:
    - name: boundlessgeo.com
      dependencies:
      - type: loadbalancer
        name: lb
      - type: server_pool
        name: server
      records:
      - name: test-lb
        type: R53ALIAS
        mu_type: loadbalancer
        target: lb

You also can pass a mu_name and a deploy ID to find resources if you'd like

Other changes

  • GlsuterFS now allows you configure a network timeout so failover happens quicker. This doesn't affect the mount itself, but only nodes that use the native fuse client. There might some performance penalty there, or it might just have been some AWS randomness
  • Linux domain nodes will allow you to specify an external samba config file you can include instead of modifying smb.conf directly. See the default.ad.samba_include_file attribute
  • DNS record cleanup is now done across all hosted zones, and uses paging so you shouldn't see any stale records anymore

New mu-user-manage

Mostly similar to the old one. But it's written in Ruby and deals with LDAP. It can also generate random passwords and leverage Scratchpad to share them. It leverages a lot of stuff from the new MU::Master module and its submodules, MU::Master::LDAP and MU::Master::Chef, any of which can be called from other places in Ruby and even Chef.

@zr2d2: This may serve as a model for your rewrites, or you may end up deciding you hate it. We'll get into that once you're in the weeds.

Oh, and now we can work around some of Chef's dumb problems. We circumvent the duplicate email address restriction by adding + to the username component of the address, which is effectively a noop as far as SMTP is concerned. We can also have dots in usernames now, simply by storing a dotless version in Chef and retaining a mapping.

# mu-user-manage --help
Listing users:
/opt/mu/bin/mu-user-manage

Show details for a specific user:
/opt/mu/bin/mu-user-manage <username>

Adding/modifying users:
/opt/mu/bin/mu-user-manage [-a|-r] [-e <email>] [-n '<Real Name>'] [-i|-p <password>|-g] [-o <chef_org>] [-v <chef_org>] [-m <email>] [-l <chef_user>] <username>

Deleting users:
/opt/mu/bin/mu-user-manage [-i] -d <username>
  -d, --delete                         Delete the user and all of their Chef
                                       and filesystem artifacts.
  -s, --skipupload                     Do not upload Chef artifacts to new
                                       users' orgs for them. The user's
                                       dotfiles will be configured to do so
                                       automatically on their first interactive
                                       login.
  -m, --monitoring-alerts-to=<s>       Send this user's monitoring alerts to an
                                       alternate address. Set to 'none' to
                                       disable monitoring alerts to this user.
  -n, --name=<s>                       The user's real name. Required when
                                       creating a new user.
  -e, --email=<s>                      The user's email address. Required when
                                       creating a new user.
  -a, --admin                          Flag the user as a Mu admin. They will
                                       be granted sudo access to the 'mu'
                                       (root's) Chef organization.
  -r, --revoke-admin                   Revoke the user's status as a Mu admin.
                                       Access to the 'mu' (root) Chef
                                       organization and sudoers will be
                                       removed.
  -o, --orgs=<s+>                      Add the user to the named Chef
                                       organization, in addition to their
                                       default org or orgs.
  -v, --remove-from-orgs=<s+>          Remove the user to the named Chef
                                       organization.
  -p, --password=<s>                   Set a specific password for this user.
  -g, --generate-password              Generate and set a random password for
                                       this user.
  -l, --link-to-chef-user=<s>          Link to an existing Chef user. Chef's
                                       naming restrictions sometimes
                                       necessitate having a different account
                                       name than everything else. Also useful
                                       for linking a pre-existing Chef user to
                                       the rest of a Mu account.
  -i, --interactive                    Interactive prompt to set a password.
  -c, --scratchpad, --no-scratchpad    Use Mu's Scratchpad to securely share
                                       user passwords instead of printing the
                                       password directly to the terminal.
                                       (Default: true)
  -t, --notify-user                    Share the Scratchpad link for new
                                       passwords to users via email, instead of
                                       printing to the screen.
  -h, --help                           Show this message
Assets 2
Loading