Variable substitution and patches don't mix?

[bozho]

Hi all,

I may be doing something wrong, but it looks like flux variable substitution and patching resources doesn't really work. Here's my example with a Gateway:

# ./infrastructure/gateway/kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: ${namespace_name}
resources:
  - gateway.yaml

# ./infrastructure/gateway/gateway.yaml
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: ${gateway_name}
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  gatewayClassName: nginx
  listeners:
    - name: dummy
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: All

# ./clusters/_base/kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component
resources:
  # ...
  - gateway.yaml
  # ...

# ./clusters/_base/gateway.yaml
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: gateway
  namespace: flux-system
spec:
  dependsOn:
    - name: gateway-api
  interval: 5m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./infrastructure/gateway
  prune: true
  wait: true
  timeout: 5m
  postBuild:
    substitute:
      namespace_name: networking
      gateway_name: frontend
  patches:
    - target:
        kind: Gateway
        name: ${gateway_name}
      patch: |-
        kind: Gateway
        metadata:
          name: _
        spec:
          listeners:
            - name: http
              port: 80
              protocol: HTTP
              hostname: www.example.com
              allowedRoutes:
                namespaces:
                  from: All
            - name: https
              port: 443
              protocol: HTTPS
              hostname: www.example.com
              allowedRoutes:
                namespaces:
                  from: All
              tls:
                mode: Terminate
                certificateRefs:
                  - name: example-cert

The _base kustomization is then used by actual cluster kustomizations.

There seems to be a problem when ${gateway_name} variable placeholder is used in the patch.

If my base gateway.yaml kustomization doesn't specify the patch, ${gateway_name} in Gateway's metadata.name is substituted properly, and Gateway with the name frontend and the dummy listener is created.

If I specify the patch as above (with target.name set to ${gateway_name}), the patch silently doesn't get applied.

If I set patch target.name to frontend explicitly, the patch still doesn't get applied.

Only if I set both the patch target.name and Gateway's metadata.name explicitly to frontend does the patch get applied. Is this a bug, or am I doing something wrong here?

[stefanprodan]

Variable substitution are a post build action, so you can't use vars in some patch target nor anywhere in the kustomization.yaml fields. The API naming should make this clear, as it's under postBuild. How all this works, it that Flux runs the substitutions on the kustomize build result, right before applying them. Docs here: https://fluxcd.io/flux/components/kustomize/kustomizations/#post-build-variable-substitution

[bozho]

Disclaimer: I'm obviously still struggling to understand how all these moving parts fit together :-)

I understand that variable substitution is a post build action, but I was assuming that kustomize will match metadata.name and target.name on the pre-substitution string ${gateway_name}, apply the patch, and flux will then apply variable substitution after the build.

Curiously enough, if I remove target.name completely, both patching and variable substitution work. What happens during kustomize build if target.name is defined and metadata.name on the target is set to a variable template?

Is there documentation explaining where flux variable substitution doesn't/can't work?

[stefanprodan]

I suggest you experiment with this locally using kustomize build ./ | flux envsubst --strict. You can move the in-line patch from the Flux Kustomization in the config file kustomization.yaml and figure out how Kustomize works. When you remove the target.name, then Kustomize will apply the patch to all objects that match the kind. The Kustomize docs can be found here: https://kubectl.docs.kubernetes.io/references/kustomize/

[bozho]

Ok, I can see the pattern now. I've moved the patch to ./infrastructure/gateway/kustomization.yaml for testing.

If both target.name in patches and metadata.name is frontend, kustomize build . works as expected and expands Gateway with patches.

If target.name is ${gateway_name} and metadata.name is frontend (or vice versa), kustomize build . doesn't match entities, so no patches are applied, as expected, since name strings don't match.

However, if both target.name and metadata.name are ${gateway_name}, patches are not applied. It seems like a reasonable expectation that if kustomize build does string matching on the name fields that it would match ${gateway_name} strings. Or do strings in the form ${...} have special meaning in kustomize, too?

[bozho]

Did a bit more testing...

When the patch is in ./infrastructure/gateway/kustomization.yaml and I set target.name to \${gateway_name} (with a backslash), kustomize build . actually applies the patch. Progress!

So, I moved the patch up a level, back to ./clusters/_base/kustomization.yaml. If I just leave target.name as \${gateway_name}, kustomize build of the cluster kustomization will emit it as-is and flux envsubst will fail to substitute the gateway_name variable, which is expected. Then I tried setting target.name to \$${gateway_name}, which make kustomize build of the cluster kustomization emit target.name as \${gateway_name}, which I was expecting would get passed down the level to the infrastructure kustomization, allowing it to match the patch correctly, but it doesn't work.