Skip to content

Parameter Types and Validation

Jump to bottom
Sam Hatfield edited this page Aug 29, 2016 · 9 revisions

Converge Parameter Validation: a functional specification

Owner: Sam Hatfield (@sehqlr)

Date/Version: 2016-08-26

Demo & First Release

Overview

The param configuration allows users to declare values that are:

  • immutable, as in, they will not be altered during converge operations
  • easily referenced in other parts of your converge configuration, using the param keyword

The syntax aims to allow users to validate and constrain their params so that bad input is not introduced into a system managed by converge. The principle ways it does this is by allowing users to define a type of input, and additional constraints, called "rules."

Param types

The type of a parameter constrains what kind of data can be put into a param. For example, if you want a param to define how many virtual machines you need, you need that param to be an integer, not a float or your cat's name.

string

This is the default type for parameters.

Functions available in rules and predicates:

Name Description Rule Example
len Counts number of chars in the string, returns an int `len
empty this is a shorthand for `len eq 0`
oneOf checks for value's membership in a list of other strings oneOf apple blackberry cherry
notIn this is a shorthand for not oneOf notIn lions tigers bears

int

This is the first type added to parameters. Like string, this type can be inferred from the default value.

Functions available in rules and predicates:

Name Description Rule Example
min is the same as param.Value >= arg min 45
max is the same as param.Value <= arg max 8
range this is shorthand for and (min x) (max y) range 4 12
isEven is the same as param.Value % 2 == 0 `len
isOdd this is shorthand for not isEven `len

rules syntax

A "rule" is a more specific constraint for a parameter. They are represented as a partial text/template expression, with the value of the parameter piped in.

Params are configured in an HCL block, with the following fields: default, type, and either rule or rules.

param "<name>" {
    # all of these fields are optional
    default = <value>,
    type = <string|int>,
    <rule|rules|ruleset> = <definition>,
}

Any or none of these:

Name Input Description Notes
default any plaintext the value of the param, if not overridden by other config If this is not included, then the user must provide a value elsewhere
type string or int what kind of values can be assigned to this param if this is not included, then it's inferred from default, or falls back on string

Only one of these:

Name Input Description Notes
rule nested HCL block a single test to validate the parameter the syntax for the rule block is described below
rules an array of predicates rule blocks condensed into a shorthand form Each string is parsed as a must rule with a standard error message
ruleset name of a ruleset block a ruleset is modularized collection of rules rulesets are defined in their own block, see below

rule

param "<name>"
    ...
    rule {
        <level> = "<predicate>"
        err = "<message>"
}

The err field is required, and one, and only one, of the options for <level> must be present. The field used defines the behavior of converge when a param does not validate.

Options for <level>:

Name Description
should if validation fails, exit with return code 1
must if validation fails, exit with return code 2

rules

param "goldilock's password" {
    type = int
    rules = [
        "len | range 8 12",
        "ne password",
    ]
}

ruleset

The argument passed to the ruleset is named by the <argument> field. You can refer to that value in the predicates with $argument.

param.ruleset "<name>" "<argument>" {
    type = "<string|int>"
    add "<name>" {
        description = "<description>"
        <level> = "<predicate>"
        err = "<message>"
    }
    add "<name>" {
        ...
    }
    ...
}

Good HCL Examples

These snippets will validate:

param "name" { }
# type is string

param "quorum" {
    type = int

    rule {
        should = "isOdd"
        err = "use an odd number of nodes for fault tolerance"
    }

    rule {
        must = "min 1"
        err = "you have to have at least one control node!"
    }
}

param "blocksize" {
    default = 128,
    # type is inferred from default
    rules = [
        "range 50 512",
    ]
}

param "password" {
    # type defaults to string
    rules = [
        "len | range 5 25",
    ]
}

param "cipher" {
    rules = [
        "oneOf Rijndael Serpent Twofish",
        "notIn DES Blowfish",
    ]
}

Bad HCL Examples

Along with params that don't pass validation, these snippets will fail with additional errors:

# this throws an error:
# "float32 is an unsupported type.
# Use one of these instead:
# {{list of supported types}}"
param "unsupported_type" {
    type = float32,
    rules = [
        "isEven",
        "range 1 34",
    ]
}

# this throws an error:
# "isPrime is an unsupported extension for type int.
# Use one of these instead:
# {{list of supported extensions}}"
param "unsupported_extension" {
    default = 7,
    type = int,
    rule {
        must = "isPrime",
        err = "primes are magical"
    }
}

Implementation Notes for later

The expression is injected into a validation template, which is then rendered with the value of the param. If the template renders a truthy string and no errors, then the param is valid. Otherwise, it is invalid, and an error is returned with the text from err. The validation template checks the type of the value first automatically. Validate takes a list of strings and injects them into this template:

{{ . | $expression }}

Because of how values are piped between expressions in go templates, function calls in expressions take the provided value as their last argument. (aside: we should have an internal function for rearranging these for other funcs.) Each validation returns an error value. If a validation returns nil, then it is valid.

Future Work

This is what the syntax will look like in future versions. Most of the work will go to adding new types and extensions for them.

Type Inferred? What it represents
decimal yes floating point numbers and percentages
url no URLs, including protocall, but not including GET args
ipv4 no IPv4 addresses
ipv6 no IPv6 addresses
CIDR no CIDR subnet notation
blocksize no storage capacity as number + unit (8MiB) or % of disk space

HCL Examples

These are valid:

param "dns_server" {
    default = 8.8.8.8
    type = ipv4
    # validation is implicit
}

param "vm_size" {
    default = 128MiB,
    type = blocksize,
    rules = [
        "range 50MiB 512MiB",
    ]
}

The same invalidations apply from the Demo. The list of valid types and extensions for those types would be added to the list output.

This is an example that assumes that there is some way to load in third-party modules

param.ruleset "gpg_key" {
    # this assumes some loading/registering process for 3rd party code
    type = custom.GpgKey
    # $param is the value passed into the ruleset
    rule "passphrase_protected" {
        must = "custom.isPassphraseProtected $param"
        msg = "gpg key is not passphrase protected"
    }
    rule "is_signed_by_Bob" {
        must = "custom.isSignedBy $param {{param `fingerprint.bob`}}"
        msg = "Bob did not sign this key!"
    }

}

param "public_key" {
    ruleset = "gpg_key {{file `key`}}"
}

Users defining their own validators and types will have to write those error messages. However, all of their messages will be prepended with "The module loaded at line X did this..."

Clone this wiki locally