Skip to content
This repository has been archived by the owner on Oct 20, 2022. It is now read-only.

Secure Nifi Cluster without CertManager not working #186

Open
iordaniordanov opened this issue Feb 1, 2022 · 0 comments
Open

Secure Nifi Cluster without CertManager not working #186

iordaniordanov opened this issue Feb 1, 2022 · 0 comments

Comments

@iordaniordanov
Copy link

Bug Report

What did you do?
Deploy nifi secure nifi cluster without Cert-Manager deployed in my K8S cluster. Yaml used to test

apiVersion: nifi.orange.com/v1alpha1
kind: NifiCluster
metadata:
  name: <name>
  labels:
    app: <label>
spec:
  service:
    headlessEnabled: true
  zkAddress: <zk_address>
  zkPath: <zk_path>
  clusterImage: <Nifi Image>
  oneNifiNodePerNode: true
  clusterSecure: true
  readOnlyConfig:
    logbackConfig:
      replaceConfigMap:
        data: logback.xml
        name: <logback configmap name>
        namespace: <logback configmap namespace>
    bootstrapProperties:
      nifiJvmMemory: <Memory>
    nifiProperties:
      webProxyHosts:
        - "<proxy_host>"
      overrideConfigs: |
        nifi.nar.library.autoload.directory=./extensions
        nifi.web.http.network.interface.default=eth0   
        nifi.web.http.network.interface.lo=lo
        nifi.web.proxy.context.path=<context_path>
        nifi.database.directory=/opt/nifi/data/database_repository
        nifi.flow.configuration.archive.dir=/opt/nifi/data/archive
        nifi.flow.configuration.file=/opt/nifi/data/flow.xml.gz
        nifi.templates.directory=/opt/nifi/data/templates
        nifi.provenance.repository.max.storage.size=<max_size>GB
        nifi.provenance.repository.indexed.attributes=<attributes>
  nodeConfigGroups:
    default_group:
      isNode: true
      nodeSelector:
        <node_selectors>
      storageConfigs:
        - mountPath: "/opt/nifi/data"
          name: data
          pvcSpec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: <size>Gi
            storageClassName: <storage_class>
        - mountPath: "/opt/nifi/content_repository"
          name: content-repository
          pvcSpec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: <size>Gi
            storageClassName: <storage_class>
        - mountPath: "/opt/nifi/flowfile_repository"
          name: flowfile-repository
          pvcSpec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: <size>Gi
            storageClassName: <storage_class>
        - mountPath: "/opt/nifi/provenance_repository"
          name: provenance-repository
          pvcSpec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: <size>Gi
            storageClassName: <storage_class>
        - mountPath: "/opt/nifi/nifi-current/work"
          name: work
          pvcSpec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: <size>Gi
            storageClassName: <storage_class>
      serviceAccountName: "default"
      resourcesRequirements:
        <resource_requirements>
  propagateLabels: true
  nifiClusterTaskSpec:
    retryDurationMinutes: 10
  listenersConfig:
    internalListeners:
      - type: "http"
        name: "http"
        containerPort: <port>
      - type: "cluster"
        name: "cluster"
        containerPort: <port>
      - type: "s2s"
        name: "s2s"
        containerPort: <port>
      - type: "prometheus"
        name: "prometheus"
        containerPort: <port>
    sslSecrets:                                                                                                                                               
      tlsSecretName: "test-nifikop"                                                                                                                           
      create: false
  externalServices:
    - name: "clusterip"
      spec:
        type: ClusterIP
        portConfigs:
          - port: <port>
            internalListenerName: "http"

The same YAML works for our insecure Nifi deployments just by removing

    sslSecrets:                                                                                                                                               
      tlsSecretName: "test-nifikop"                                                                                                                           
      create: false

and setting

  clusterSecure: false

What did you expect to see?
I'm sure there are issues with our config and I'm not expecting it to work from the first time, but I was expecting at least to see pods being brought up and having some kind of errors in the Nifi logs.

What did you see instead? Under which circumstances?
No pods are being brought up and I see errors in the Nifi Operator

{"level":"error","ts":1643719642.1879852,"logger":"controller-runtime.manager.controller.nifiparametercontext","msg":"Reconciler error","reconciler group":"nifi.orange.com","reconciler kind":"NifiParameterContext","name":"<parameter_context_name>","namespace":"<namespace>","error":"controller secret not found: Secret \"<cluster_name>-controller\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:267\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:198\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}
{"level":"info","ts":1643719644.999143,"logger":"controllers.NifiCluster","msg":"Reconciling cert-manager PKI","component":"nifi","clusterName":"<cluster_name>","clusterNamespace":"<namespace>"}
{"level":"error","ts":1643719647.3088443,"logger":"controller-runtime.manager.controller.nifiparametercontext","msg":"Reconciler error","reconciler group":"nifi.orange.com","reconciler kind":"NifiParameterContext","name":"<parameter_context_name>","namespace":"<namespace>","error":"controller secret not found: Secret \"<cluster_name>-controller\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:267\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:198\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}
{"level":"info","ts":1643719652.2984219,"logger":"controllers.NifiCluster","msg":"Reconciling cert-manager PKI","component":"nifi","clusterName":"<cluster_name>","clusterNamespace":"<namespace>"}

From what I understand the operator is looking for a secret named <cluster_name>-controller but I haven't referenced it anywhere I don't understand why it is looking for it

Environment

  • nifikop version:

0.7.1
btw, you should implemet --version command

  • go version:

Which ever version is included in your docker image - https://hub.docker.com/layers/orangeopensource/nifikop/v0.7.1-release/images/sha256-94eb3f1ec21f52041cc832fe9c1124cd6ad87666af8ffb41cd9173ae97f9a0c3?context=explore

  • Kubernetes version information:

v1.19.15-eks-9c63c4

  • Kubernetes cluster kind: EKS

  • NiFi version:

1.15.3

Possible Solution

Additional context
Add any other context about the problem here.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@iordaniordanov