Kubernetes HelmChart deployment inside of namespace, CRDs problem.

[FishyFishPat]

Hi everyone.

I am trying to deploy a teleport-cluster using a chart inside one of our namespaces. The namespace doesn't have cluster wide permissions, but only for that namespace. We keep on getting the following error:

Helm install failed: template: teleport-cluster/charts/operator/templates/crds.yaml:14:11: executing "teleport-cluster/charts/operator/templates/crds.yaml" at <include "teleport-cluster.operator.shouldInstallCRDs" .>: error calling include: template: teleport-cluster/charts/operator/templates/_helpers.tpl:97:10: executing "teleport-cluster.operator.shouldInstallCRDs" at <include "teleport-cluster.operator.checkExistingCRDs" .>: error calling include: template: teleport-cluster/charts/operator/templates/_helpers.tpl:108:21: executing "teleport-cluster.operator.checkExistingCRDs" at <lookup "apiextensions.k8s.io/v1" "CustomResourceDefinition" "" "teleportrolesv7.resources.teleport.dev">: error calling lookup: customresourcedefinitions.apiextensions.k8s.io "teleportrolesv7.resources.teleport.dev" is forbidden: User "system:serviceaccount:exp-x:exp-x" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope Last Helm logs:

This is an example of our teleport-release.yaml:

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: teleport
  namespace: exp-x
spec:
  serviceAccountName: exp-x
  targetNamespace: exp-x
  releaseName: teleport
  install:
    crds: Skip
  chart:
    spec:
      chart: ./examples/chart/teleport-cluster
      sourceRef:
        kind: GitRepository
        name: teleport-helm
  interval: 10m
  values:
    chartMode: standalone
    clusterName: teleport.example.net
    installCRDs: false
    auth:
      teleportConfig:
        # put any teleport.yaml auth configuration overrides here
        teleport:
          log:
            output: stderr
            severity: DEBUG

        auth_service:
          enabled: true
          web_idle_timeout: 1h
          authentication:
            locking_mode: best_effort

    proxy:
      teleportConfig:
        # put any teleport.yaml proxy configuration overrides here
        teleport:
          log:
            output: stderr
            severity: DEBUG
        proxy_service:
          https_keypairs_reload_interval: 12h
          https_keypairs:
            - key_file:
                valueFrom:
                  secretKeyRef:
                    name: wildcard-example-tls
                    key: tls.key
              cert_file:
                valueFrom:
                  secretKeyRef:
                    name: wildcard-example-tls
                    key: tls.crt

    operator:
      enabled: false

    # If you are running Kubernetes 1.23 or above, disable PodSecurityPolicies
    podSecurityPolicy:
      enabled: false
    

My question is. Does the service account needs a read permission to see if the CRD is installed. Or do I need a separate chart for the operator?

[webvictim]

Yes, the operator's ServiceAccount will need permissions to read CRDs cluster-wide. CRD resources can be created inside namespaces, but the actual definitions are cluster-wide.