| RFC | Title | Author | Status | Type |
|---|---|---|---|---|
42 |
Policyfile HTTP Resource API |
Daniel DeLeo <[email protected]> |
Accepted |
Standards Track |
Policyfiles are a new feature of Chef that allow the user to specify a
run_list and exact set of cookbooks that chef-client will use to
converge a node (host) to a desired state. In contrast with the current
behaviors of Chef Client and Server, Policyfiles provide a way to
describe and manage configuration code on a whole-node basis, reducing
Chef Client's use of shared, mutable resources hosted on the server and
providing operators easier visibility into configuration code and
changes. For further description of the Policyfile feature, read the dedicated
POLICYFILE README.
A preview implementation of the Policyfile feature is currently available; in this implementation, Policyfile documents are stored as data bag items. Although this implementation works, it does not provide the baseline quality users expect. In particular, there is no validation, no means to make any relational queries, and documents are indexed for search as data bag items.
This RFC describes a new Policyfile HTTP Resource API that remediates these issues.
As a Chef user,
I want Policyfiles to have a dedicated HTTP Resource API,
so that Policyfiles will be validated upon upload
and I can query the relationships between Policyfiles and other objects,
and Policyfiles will not be indexed as an incorrect data type.
A Policyfile lock is a JSON document that specifies a node's run list and the set of cookbooks a node will use to converge the node.
A Policy Group is a set of nodes defined by a single token. Nodes must be a member of a Policy Group in order to use the Policyfile feature, and nodes may only be a member of one policy group.
A Policyfile lock is a JSON document that has the following structure:
{
"revision_id": "edd40c30c4e0ebb3658abde4620597597d2e9c17",
"name": "some_policy_name",
"run_list": [
"recipe[policyfile_example_cookbook::default]"
],
"cookbook_locks": {
"policyfile_example_cookbook": {
"version": "1.0.0",
"identifier": "f04cc40faf628253fe7d9566d66a1733fb1afbe9"
}
}
}
The revision_id field is used to distinguish revisions of a Policyfile lock
document. It is a JSON String containing a hash of the canonicalized content of
the Policyfile lock, given in hexadecimal form. A fully compliant server MUST
independently compute the revision_id and reject a creation request if the
submitted revision_id is incorrect.
The canonicalized form of the Policyfile lock is not finalized at this time, but will be provided in a future update to this document. Note that the format will not be based on JSON, as JSON has no canonical form.
The name field describes the functional role that is fulfilled by a node that
applies this policy. It may contain alphanumeric characters, hyphens,
underscores, the dot character and the colon character. In regular expression
form: /^[\-[:alnum:]_\.\:]+$/. The name is also limited to a maximum of 255
characters. The Chef Server must reject a Policyfile lock document if the name
does not meet these criteria.
The run_list field gives the ordered list of recipes that Chef client should
apply to converge a node. Each item MUST be in fully qualified form, that is,
recipe[COOKBOOK_NAME::RECIPE_NAME]. Roles are not valid run list items in the
Policyfile lock document, and the Chef Server MUST reject run lists that
contain roles.
The cookbook_locks Field is a JSON object (Ruby Hash). The keys are cookbook
names. The Chef Server MUST validate the keys according to the same validation
rules applied to cookbook names for other APIs. The values are JSON objects
that contain information about the cookbook. These objects MUST have a
version field that gives the cookbook's version number. The server must apply
the same validation rules to this field as it applies to cookbooks uploaded to
the cookbook artifacts API, described in Chef RFC022.
They MUST also have an identifier field. This gives the cookbook's
identifier, as described in Chef RFC022. The Chef Server must validate this
field according to the same rules as applied to cookbooks uploaded to the
cookbook artifacts API. Cookbook lock objects MAY contain other fields with
arbitrary information about the cookbook. The Chef Server must accept and
persist this data. For example, ChefDK generates Policyfile lock documents that
include source URLs, version control information, and the name of the directory
where the cookbook is stored in ChefDK's cache. Cookbook lock objects MAY
contain a dotted_decimal_identifier field which contains a representation of
the identifier as a version number. This field is used when cookbooks are
stored on a server that does not support the arbitrary cookbook identifier API
described in Chef RFC022.
Example cookbook locks including optional fields:
"policyfile_demo": {
"revision_id": "edd40c30c4e0ebb3658abde4620597597d2e9c17",
"version": "0.1.0",
"identifier": "f04cc40faf628253fe7d9566d66a1733fb1afbe9",
"dotted_decimal_identifier": "67638399371010690.23642238397896298.25512023620585",
"source": "cookbooks/policyfile_demo",
"cache_key": null,
"scm_info": {
"scm": "git",
"remote": "[email protected]:danielsdeleo/policyfile-jenkins-demo.git",
"revision": "edd40c30c4e0ebb3658abde4620597597d2e9c17",
"working_tree_clean": false,
"published": false,
"synchronized_remote_branches": [
]
},
"source_options": {
"path": "cookbooks/policyfile_demo"
}
},
"apt": {
"version": "2.6.1",
"identifier": "5f7045a8aeaf6ccda3b3594258df9ee982b3a023",
"dotted_decimal_identifier": "26863567272587116.57882360917678303.174725757378595",
"cache_key": "apt-2.6.1-supermarket.chef.io",
"origin": "https://supermarket.chef.io/api/v1/cookbooks/apt/versions/2.6.1/download",
"source_options": {
"artifactserver": "https://supermarket.chef.io/api/v1/cookbooks/apt/versions/2.6.1/download",
"version": "2.6.1"
}
},Note that a future update to Chef RFC 022 will introduce a standardized format for cookbook identifiers based on the content of the cookbook and will introduce strict server-side checking of the identifier field.
Named run lists provide a replacement for the override run list feature in Chef Client, which is not compatible with Policyfiles because the run list and cookbook set must be bundled together in a Policyfile lock document.
Then named_run_lists field is a JSON object; inside this object, the keys are
the names of the named run lists and values are JSON arrays of run list items.
As with the top-level run_list field, run list items for named run lists MUST
be in fully qualified recipe form, and roles are not accepted.
A named run list name may contain alphanumeric characters, hyphens,
underscores, the dot character and the colon character. In regular expression
form: /^[\-[:alnum:]_\.\:]+$/. It must be a least one character long and
cannot exceed 255 characters.
"named_run_lists": {
"update_jenkins": [
"recipe[jenkins::master]",
"recipe[policyfile_demo::default]"
]
},A Policyfile lock may contain default and override attributes. Chef Client will treat these attributes the same as it currently treats role attributes. Default and override attributes are stored in separate top-level keys:
{
"default_attributes": {},
"override_attributes": {}
}A Policyfile lock can contain arbitrary information in other top-level fields.
The Chef Server MUST accept and preserve the data in these fields (though the
server MAY enforce a limit on overall document size). ChefDK currently uses a
solution_dependencies field to store a list of all dependencies relevant to
the cookbook set.
Container for policy groups.
Returns a list of policy groups that exist.
Authorization endpoint for the policy group :policy_group_name
Returns a JSON object showing the active policy names and revision IDs for the
policy group :policy_group_name.
Returns the policyfile lock document at the revision that is currently associated with the given policy group and policy name.
Sets the active revision of :policy_name for the policy group
:policy_group_name to the policyfile lock document in the request body. The
policy group, policy name, and policyfile lock revision will be created if any
does not exist.
Policyfile lock revisions may not be updated. When the server already has a
Policyfile lock document with the same revision_id as given in the request
body, the effect of the request is to set the current active revision of
:policy_name for the given :policy_group_name to the revision specified in
the request body and ignore the other fields in the request body.
Sets the current active revision of :policy_name for the :policy_group_name
to the value given in the POST body. The revision must already exist on the
server. For example, to set the active revision of the 'appserver' policy in
the 'qa' policy group, you would POST to
/policy_groups/qa/policies/appserver, with a POST body like the following:
{
"revision_id": "bfec256f76a52ff707ca72e71b464a41c410b229"
}Returns a list of nodes that belong to the policy group. Can be filtered for a
specific :policy_name with the policy_name query parameter, using a URL of
the form: GET /policy_groups/:policy_group_name/nodes?policy_name=:policy_name
TODO: This is where AuthZ for policies goes, does it do anything else?
Create a new Policyfile lock document. The POST body MUST be a valid Policyfile
lock document, as described above. The document can subsequently be retrieved
at the relative URL /policies/:policy_name/revisions/:revision_id
Policyfile lock revisions cannot be updated. If the server already has a
Policyfile lock document with the same revision_id as given in the body of
the request, the server MUST return 409 response.
Returns a list of the available revisions for the given policy name
Returns the policyfile lock document with the given name and revision ID.
Deletes the policyfile lock document with the given name and revision ID.
Returns the list of policy groups that are associated with the given policy name at the given revision ID.
In addition to the basic REST interface described above, the server may also provide a promotion API. The promotion API facilitates defining the deployment cycle for policies (i.e., the order in which policies move through policy groups) and allows policies to be promoted to subsequent stages of the deployment cycle with a single API call.
Returns the policy group configuration for :policy_group_name. The document
will contain the name of the next policy group in the deployment cycle. Exact
format is TBD, but it could look like:
{
"name": "development",
"next_group_name": "qa"
}Set the policy group configuration for :policy_group_name. Exact format is
TBD, but will be identical to the document format described above for GET.
Promotes all or some of the policyfile lock revisions currently active in
:policy_group_name to the next stage of the deployment cycle. POST body
format is TBD.
The user generates a Policyfile lock document locally. When using ChefDK, the
policyfile lock document is generated by gathering requirements specified in a
Ruby DSL (e.g., the Policyfile.rb) and solving cookbook dependencies, but
users may create the policyfile lock document in some other way if desired.
Before the Policyfile lock document is published to the Chef Server, all
cookbooks described in the cookbook_locks section of the document are
uploaded to the cookbook artifacts API.
The policyfile lock document may be uploaded in a single step:
PUT /policy_groups/:policy_group_name/policies/:policy_name
Or it may be uploaded in two steps:
POST /policies/:policy_name/revisions/: Upload the policy to the Chef ServerPOST /policy_groups/:policy_group_name/policies/:policy_name: make the policyfile lock document revision uploaded in the previous step the active one for the specified policy group.
Chef Client is configured with both a desired :policy_group_name and
:policy_name. Given that information, it can retrieve the Policyfile lock
document with a single request:
GET /policy_groups/:policy_group_name/policies/:policy_name: Get the policy document with the run list and cookbook set for the chef run.
The Chef Server authorizes an actor to access the Policyfile API separately for policy groups and policy names (i.e., the set of all policyfile lock documents with the same name). For operations that affect both policy groups and policy names, an actor must have all relevant permissions to perform the requested operation.
Container-level Authorization:
- An actor must have
Createpermission on thepolicy_groupscontainer to create a new group. - An actor must have
Listpermission on thepolicy_groupscontainer to list groups.
Object-level Authorization:
- An actor must have
Readpermission on a specificpolicy_groupto read the promotion configuration for that group (promotion API only) - An actor must have
Readpermission on a specificpolicy_groupto read the list of active policies and revisions. - An actor must have
Updatepermission to set the active revision for a givenpolicy_nameto a specific revision viaPOST /policy_groups/:policy_group_name/policies/:policy_name - An actor must have
Updatepermission to set the active revision for a givenpolicy_nameto a specific revision via the promotion API. - An actor must have
Updatepermission to modify the promotion policy for a givenpolicy_group. - An actor must have
Readpermission on a specificpolicy_groupandListpermission on thenodescontainer to list nodes in a policy group viaGET /policy_groups/:policy_group_name/nodes - An actor must have
Deletepermission on a specificpolicy_groupto delete it.
Authorization for policy names is similar to cookbook versions and data bags in that actions on individual objects are authorized based on permissions to the collection; individual policy file lock revisions do not have independent ACLs.
Container-level Authorization:
- An actor must have
Createpermission on thepoliciescontainer to create a newpolicy_name. This applies when creating a new policyfile lock revision viaPOST /policies/:policy_name/revisions/and no existing policyfile lock document with the same name exists. - An actor must have
Listpermission on thepoliciescontainer to list all policy names.
Object-level Authorization:
- An actor must have
Updateto create a new policyfile lock revision. This applies when creating a new revision of a policy name viaPOST /policies/:policy_name/revisions/and there is an existing policyfile lock document with the same name. - An actor must have
Readto list the available revisions for a givenpolicy_nameviaGET /policies/:policy_name/revisions/. - An actor must have
Readto retrieve a specific revision of a givenpolicy_nameviaGET /policies/:policy_name/revisions/:revision_id. - An actor must have
Deleteto delete a specific revision of a givenpolicy_name.
For convenience and to reduce HTTP round-trips, some APIs update and/or disclose information about different object types. An actor is permitted to perform the requested action only if authorized for all of the requested actions individually,
- An actor must have
Readpermission on a specificpolicy_groupand also haveReadon a specificpolicy_nameto retrieve the active policy lock document revision for the given group and name. - To set the current active policy lock revision via
PUT /policy_groups/:policy_group_name/policies/:policy_name, an actor must have eitherUpdatepermission on the givenpolicy_groupif it exists, orCreateon thepolicy_groupscontainer to create the desired group, AND eitherUpdatepermission on the givenpolicy_nameif it exists orCreateon thepoliciescontainter to create the the desired policy if it doesn't exist. The same restrictions apply to the promotion API. - An actor must have
Readpermission on a sepcificpolicy_nameandListpermission on thepolicy_groupscontainer to list the groups associated with a given policy at a given revision viaGET /policies/:policy_name/revisions/:revision_id/policy_groups
The means by which a node is associated to a Policyfile lock is designed such that it is possible to have more than one revision of a Policyfile lock active in a Chef organization at one time. That is, it must be possible to update the Policyfile lock for only some nodes with a given functional role. For example, a configuration code change may deploy a newer version of some important software that must be tested in a non-production environment before production deployment, or may cause a service restart that temporarily reduces capacity.
It is also desirable that the Policyfile lock document itself not contain any information about which nodes it is applied to, as this simplifies the overall design and makes it easy to create tooling to manage promotion of a Policyfile lock through the phases of a deployment cycle.
To accommodate these two constraints, Policyfile locks are associated to a node via both the policy name and policy group. The policy group models the phases of a deployment cycle, such as "development", "staging" and "production." Each policy group can have a different revision of a Policyfile lock, which enables configuration code to be updated in one policy group without any effect on the configuration code in other policy groups.
This is how chef-client retrieves the policy it will apply for the current
chef-client run. It is also possible to compare (diff) the active policy for
two different policy groups by retrieving the active policy for each group and
computing the difference.
This enables a GUI to provide a list of groups for navigational purposes and for list-based input (such as selecting the policy group for a new node object).
This enables a GUI to provide a list of policy names for navigational purposes and for list-based input.
When the node document is updated to include the policy group and policy name (a future change outside the scope of this RFC), this information will be accessible via search; however, the information can be easily retrieved via a relational database query for use in a web-based UI.
Although the design of Policyfile and related APIs greatly simplifies the task of finding and deleting unused cookbook versions (cookbook artifacts in this case), no such feature is proposed at this time.
The authorization behaviors for Policyfiles are designed to allow the user to limit access by either deployment phase or host's functional role.
In the first case, it may be desirable, for example, to limit access to production policy groups to a subset of individuals or perhaps a Ci system while still giving users access to update dev environments or create one-off dev environments for testing.
In the second case, a business may have some types of systems where access must be restricted to a smaller set of individuals, perhaps because those machines store sensitive data or require expertise from other teams to vet changes (e.g., a DBA must review changes to database nodes).
This work is in the public domain. In jurisdictions that do not allow for this, this work is available under CC0. To the extent possible under law, the person who associated CC0 with this work has waived all copyright and related or neighboring rights to this work.