Implement rancher2_node_pool_v2

[neilfarmer]

Is your feature request related to a problem? Please describe.

Im creating a terraform module for creating an RKE2 cluster using the rancher2 terraform provider. I create the cluster and machine_pools within the rancher2_cluster_v2 resource in my module like so:

resource "rancher2_cluster_v2" "cluster" {
  name                         = "${local.namespace}-cluster"
  kubernetes_version           = local.kube_rke2_version
  enable_network_policy        = local.network_policy
  cloud_credential_secret_name = rancher2_cloud_credential.openstack.id
  labels = merge({
    "NAMESPACE" = local.namespace,
    "TENANT_NAME" = local.openstack_tenant_name }, local.cluster_labels
  )

  rke_config {
    machine_pools {
      name                         = "master"
      cloud_credential_secret_name = rancher2_cloud_credential.openstack.id
      control_plane_role           = true
      etcd_role                    = true
      worker_role                  = false
      quantity                     = local.master_count
      drain_before_delete          = true
      machine_config {
        kind = rancher2_machine_config_v2.openstack_master_node.kind
        name = rancher2_machine_config_v2.openstack_master_node.name
      }
    }

    machine_pools {
      name                           = "worker"
      cloud_credential_secret_name   = rancher2_cloud_credential.openstack.id
      control_plane_role             = false
      etcd_role                      = false
      worker_role                    = true
      quantity                       = local.worker_count
      drain_before_delete            = true
      unhealthy_node_timeout_seconds = "3600"
      machine_config {
        kind = rancher2_machine_config_v2.openstack_worker_node.kind
        name = rancher2_machine_config_v2.openstack_worker_node.name
      }
    }
...omitted

But I cannot add more pools in unless I copy and paste more machine pools into this resource.

I have different host configurations with different memory, vcpu, etc and will require more unique pools. I would like to dynamically create these pools by passing in terraform variables. Currently I do not see a way to do this without updating the terraform module with multiple pools which defeats the purpose of the module.

Describe the solution you'd like

Please create a rancher2_node_pool_v2 to add new machine pools

resource "rancher2_node_pool_v2" "foo" {
  cluster_id =  rancher2_cluster.foo-custom.id
  name = "foo"
  machine_pools {
    name                         = "worker"
    cloud_credential_secret_name = rancher2_cloud_credential.openstack.id
    control_plane_role           = true
    etcd_role                    = true
    worker_role                  = false
    quantity                     = 3
    drain_before_delete          = true
    machine_config {
      kind = rancher2_machine_config_v2.openstack_master_node.kind
      name = rancher2_machine_config_v2.openstack_master_node.name
    }
}

If that were created, I could do a for_each loop on that resource by passing a variable to my module and I could dynamically create multiple pools

machine_pools = [
{
    "name"     = "worker"
    "quantity" = "1"
    "taints"   = ""
    "image"    = "ubuntu-noble"
    "flavor"   = "medium"
},
{
    "name"     = "worker2"
    "quantity" = "1"
    "taints"   = ""
    "image"    = "ubuntu-noble"
    "flavor"   = "medium"
},
{
    "name"     = "worker3"
    "quantity" = "1"
    "taints"   = ""
    "image"    = "ubuntu-noble"
    "flavor"   = "medium"
}
]

resource "rancher2_cluster_v2" "cluster" {
  name                         = "${local.namespace}-cluster"
  kubernetes_version           = local.kube_rke2_version
  enable_network_policy        = local.network_policy
  cloud_credential_secret_name = rancher2_cloud_credential.openstack.id
  labels = merge({
    "NAMESPACE" = local.namespace,
    "TENANT_NAME" = local.openstack_tenant_name }, local.cluster_labels
  )

  rke_config {
    machine_pools {
      name                         = "master"
      cloud_credential_secret_name = rancher2_cloud_credential.openstack.id
      control_plane_role           = true
      etcd_role                    = true
      worker_role                  = false
      quantity                     = local.master_count
      drain_before_delete          = true
      machine_config {
        kind = rancher2_machine_config_v2.openstack_master_node.kind
        name = rancher2_machine_config_v2.openstack_master_node.name
      }
    }

    machine_pools {
      name                           = "worker"
      cloud_credential_secret_name   = rancher2_cloud_credential.openstack.id
      control_plane_role             = false
      etcd_role                      = false
      worker_role                    = true
      quantity                       = local.worker_count
      drain_before_delete            = true
      unhealthy_node_timeout_seconds = "3600"
      machine_config {
        kind = rancher2_machine_config_v2.openstack_worker_node.kind
        name = rancher2_machine_config_v2.openstack_worker_node.name
      }
    }
...omitted
}

resource "rancher2_node_pool_v2" "foo" {
  for_each = { for pool in var.machine_pools }
  cluster_id =  rancher2_cluster.cluster.id
  name = each.value.name
  machine_pools {
    name                         = each.value.name
    cloud_credential_secret_name = rancher2_cloud_credential.openstack.id
    control_plane_role           = false
    etcd_role                    = false
    worker_role                  = true
    quantity                     = each.value.quantity
    drain_before_delete          = true
    machine_config {
      kind = rancher2_machine_config_v2.openstack_worker[machine_pool.key].kind
      name = rancher2_machine_config_v2.openstack_worker[machine_pool.key].name
    }
}

Of course that is all pseudo code. Hopefully that helps understand the feature and reason for it.

Describe alternatives you've considered

I investigated terraforms dynamic blocks to build out the rancher2_cluster_v2 machine_pools but it was not supported by terraform.

Additional context

None

Thanks for maintaining and improving this module. Terraform is awesome and im glad to see SUSE maintain this provider.

[gkalandadze]

Hi,

You can use terraform's dynamic blocks feature to achieve what you want. Not sure what issue you faced with it but here is my example which I'm currently using:

    dynamic "machine_pools" {
      for_each = var.node_groups
      content {
        cloud_credential_secret_name = data.rancher2_cloud_credential.vsphere.id
        control_plane_role           = machine_pools.value.control_plane_role
        etcd_role                    = machine_pools.value.etcd_role
        worker_role                  = machine_pools.value.worker_role
        name                         = machine_pools.key
        quantity                     = machine_pools.value.quantity
        machine_labels               = try(machine_pools.value.machine_labels, null)
        annotations                  = try(machine_pools.value.machine_annotations, null)

        dynamic "taints" {
          for_each = machine_pools.value.machine_taints
          content {
            key    = taints.value.key
            value  = taints.value.value
            effect = taints.value.effect
          }
        }
        machine_config {
          kind = rancher2_machine_config_v2.machineconfig[machine_pools.key].kind
          name = replace(rancher2_machine_config_v2.machineconfig[machine_pools.key].name, "_", "-")
        }
      }
    }

Where var.node_groups would be object like this:

node_groups = {
  dc1-master = {
    control_plane_role = true
    etcd_role          = true
    worker_role        = false
    quantity           = 2
    cpu_count          = 2
    disk_size          = 80000
    memory_size        = 4096
    template_name      = "my-template"
    datastore          = "my-datastore"
    cloud_config_name  = "cloud-config.yaml"

    machine_labels = {
      zone = "dc1"
    }
    machine_taints     = [
      {
        key    = "test"
        value  = "true"
        effect = "PreferNoSchedule"
      },
      {
        key    = "test2"
        value  = "true"
        effect = "PreferNoSchedule"
      }
    ]
    machine_annotations = {}
  }
  dc1-worker = {
    control_plane_role = false
    etcd_role          = false
    worker_role        = true
    quantity           = 1
    cpu_count          = 2
    disk_size          = 80000
    memory_size        = 4096
    template_name      = "my-template"
    datastore          = "my-datastore"
    cloud_config_name  = "cloud-config.yaml"

    machine_labels = {
      zone = "dc1"
    }
    machine_taints            = []
    machine_annotations = {
      "cluster.provisioning.cattle.io/autoscaler-min-size" = "1"
      "cluster.provisioning.cattle.io/autoscaler-max-size" = "2"
      "cluster.provisioning.cattle.io/autoscaler-resource-cpu" = "2"
      "cluster.provisioning.cattle.io/autoscaler-resource-ephemeral-storage" = "50Gi"
      "cluster.provisioning.cattle.io/autoscaler-resource-memory" = "4Gi"
    }
  }
}

Only problem that I'm currently facing is issue with not being able to ignore changes to quantity of each machine pool, this is not supported by terraform yet. Basically this is the issue I'm describing: #1257