Development

The development of the Terraform provider plugin is based on the Terraform SDKv2 framework. You should familiarize yourself with the Terraform plugin development workflow and how Terraform uses plugins during runtime: developer.hashicorp.com/terraform/plugin.

Basically, you write a binary that provides a gRPC interface for the Terraform binary. During a Terraform run (e.g. terraform apply), the plugin binary is executed and the provider supplies Terraform with its functionality. In this case the Foreman and Katello API wrapping.

Build the provider from source code

To test bug fixes and new features without having to release a new version to the Terraform registry, it is useful to know how to build the provider from source code.

On Linux systems, the following script builds the provider as the test version 0.99.0 and copies a successfully built binary into the user's Terraform plugin folder:

#!/bin/bash
set -ev 
export CGO_ENABLED=0
go build -v -o "terraform-provider-foreman_v0.99.0" -trimpath -ldflags '-s -w -X main.version=0.99.0' 
cp -av terraform-provider-foreman_v0.99.0 ~/.terraform.d/plugins/registry.terraform.io/terraform-coop/foreman/0.99.0/linux_amd64/

Your config may vary, depending on OS and your Terraform config. In general, the steps are similar: build, copy to standard location, set the debug version as your project's provider version.

Regarding the correct paths on your system, refer to the Terraform plugin discovery article: https://developer.hashicorp.com/terraform/plugin/how-terraform-works#discovery

Use the self-compiled provider binary

In your project, configure the provider as shown below and run terraform init -upgrade afterwards.

terraform {
  required_providers {
    foreman = {
      source  = "terraform-coop/foreman"
      version = "0.99.0"
    }
  }
}

If you rebuild multiple times with different code (if you change something and want to test it right away), Terraform will probably complain about mismatching hashes. To circumvent this, edit the .terraform.lock.hcl in your project folder and remove the hashes key:

# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.

provider "registry.terraform.io/terraform-coop/foreman" {
  version     = "0.99.0"
  constraints = "0.99.0"
  hashes = [       # <-- remove this list, leaving only "version" and "constraints"
    "h1:<hash>",
  ]
}

Run the provider in debug mode (for delve debugger or IDEs)

Terraform provider plugins are in fact standalone Go binaries. Though they are not meant to be executed themselves, debugging a plugin is done the same way you would debug any other binary.

• Terraform provider debugging: https://developer.hashicorp.com/terraform/plugin/debugging
• Relevant initialization code for SDKv2: https://developer.hashicorp.com/terraform/plugin/sdkv2/debugging
• Useful article: https://medium.com/@narinderkaurmakkar1/terraform-provider-debugging-a-step-by-step-guide-8c6d771637a1

The main.go for the Foreman provider looks like this in debug mode:

func main() {
	// opts contains the configurations to serve the Foreman plugin.
	opts := plugin.ServeOpts{
		Debug:        true,
		ProviderAddr: "registry.terraform.io/terraform-coop/foreman",
		ProviderFunc: foreman.Provider,
	}
	// Serves the foreman plugin in the defined configurations.
	plugin.Serve(&opts)
}

Repo structure

The most important folders are

• foreman with its Terraform definitions (resources and data sources) and
• foreman/api which holds the HTTP API wrapper code.

In between these two "layers", conversions of the objects are handled via Go structs and conversion functions. Especially the handling of Terraform states (ResourceData) is very important, because they reflect the input from the user's manifests.

After cloning, you'll find the following directory structure. See the annotations to better understand the usage.

.
├── autodoc
├── cmd                     <-- autodoc related code
│   └── autodoc
├── CONTRIBUTING.md
├── docs                    <-- auto-generated mkdocs output (from code)
│   ├── data-sources
│   │   └── foreman_*.md
│   ├── index.md
│   └── resources
│       └── foreman_*.md
├── examples                <-- Terraform manifests (*.tf) with tested use-cases
│   ├── architecture            for each resource
│   │   └── main.tf
│   ├── *
│   └── verify_provider     <-- special case, see README
│       ├── main.tf
│       └── tfoverriderc
├── foreman                 <-- main provider plugin code
│   ├── api                 <-- Foreman HTTP API wrapper code
│   │   ├── ...
│   │   ├── client.go       <-- HTTP client used in all calls
│   │   ├── client_test.go
│   │   ├── ...
│   │   ├── core.go
│   │   ├── core_test.go
│   │   └── *.go
│   ├── config.go
│   ├── data_source_foreman_*.go       <-- all data source implementations of Foreman/Katello objects
│   ├── data_source_foreman_*_test.go
│   ├── foreman_api_test.go
│   ├── provider.go                    <-- provider meta code
│   ├── resource_foreman_*.go          <-- all resource implementations of Foreman/Katello objects
│   ├── resource_foreman_*_test.go
│   ├── ...
│   ├── resource_helper.go
│   ├── testdata            <-- JSON files with testdata for API mocking
│   │   ├── 1.11
│   │   ├── 1.20
│   │   ├── 3.1.2
│   │   └── 3.6
│   └── utils
│       └── utils.go        <-- internal utils (function trace calls, debug logging)
├── go.mod
├── go.sum
├── LICENSE
├── main.go                 <-- main entry point for the plugin binary
├── mkdocs.yml
├── README.md
├── templates
│   ├── ...
└── terraform-provider-foreman.log     <-- in case you run Terraform with debugging in this directory

Foreman/Katello API

The Foreman API is a basic REST API and documented at apidocs.theforeman.org/foreman/ and apidocs.theforeman.org/katello/. The API has some inconsistencies which require special care in the API wrapper. This is mostly noticeable between Foreman and Katello, because Katello as a plugin for Foreman was written by a different team and therefore follows a different implementation strategy.