Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating an ingress object with no http rules makes the controller crash loop #428

Open
alex-bezek opened this issue Sep 11, 2024 · 2 comments
Labels
area/controller Issues dealing with the controller bug Something isn't working needs-triage Issues that need triage

Comments

@alex-bezek
Copy link
Collaborator

Kubernetes Version

1.28

Helm Chart Version

0.15.0

Ingress Controller Version

0.12.0

System Info

Testing in EKS

Kubernetes Logs

{"level":"info","ts":"2024-09-11T16:34:29Z","logger":"setup","msg":"starting manager","version":"0.12.2","commit":"b28f15b633ca48c2d8ca771df4f4ff82affc7875"}
{"level":"info","ts":"2024-09-11T16:34:29Z","logger":"setup","msg":"configured API client","base_url":"http://api.controlplane.svc.cluster.local"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x15f42c3]

goroutine 1 [running]:
github.com/ngrok/kubernetes-ingress-controller/internal/store.Store.shouldHandleIngressIsValid({{{0x1cf47c0, 0xc0006301c8}, {0x1cf47c0, 0xc0006301e0}, {0x1cf47c0, 0xc000630210}, {0x1cf47c0, 0xc000630228}, {0x1cf47c0, 0xc000630240}, ...}, ...}, ...)
	github.com/ngrok/kubernetes-ingress-controller/internal/store/store.go:427 +0xc3
github.com/ngrok/kubernetes-ingress-controller/internal/store.Store.shouldHandleIngress({{{0x1cf47c0, 0xc0006301c8}, {0x1cf47c0, 0xc0006301e0}, {0x1cf47c0, 0xc000630210}, {0x1cf47c0, 0xc000630228}, {0x1cf47c0, 0xc000630240}, ...}, ...}, ...)
	github.com/ngrok/kubernetes-ingress-controller/internal/store/store.go:391 +0x58
github.com/ngrok/kubernetes-ingress-controller/internal/store.Store.ListNgrokIngressesV1({{{0x1cf47c0, 0xc0006301c8}, {0x1cf47c0, 0xc0006301e0}, {0x1cf47c0, 0xc000630210}, {0x1cf47c0, 0xc000630228}, {0x1cf47c0, 0xc000630240}, ...}, ...})
	github.com/ngrok/kubernetes-ingress-controller/internal/store/store.go:301 +0x11b
github.com/ngrok/kubernetes-ingress-controller/internal/store.(*Driver).PrintState(0xc00001a600, {{0x1cf17f8?, 0xc00047a180?}, 0x7f907095fb80?})
	github.com/ngrok/kubernetes-ingress-controller/internal/store/driver.go:258 +0x5d
main.getDriver({_, _}, {_, _}, {{0x7ffce9dd6c7f, 0x5}, {0x7ffce9dd6c93, 0x24}, {0x7ffce9dd6c62, 0x5}, ...})
	./main.go:413 +0x4d8
main.runController({_, _}, {{0x7ffce9dd6c7f, 0x5}, {0x7ffce9dd6c93, 0x24}, {0x7ffce9dd6c62, 0x5}, {0x7ffce9dd6bb2, 0x1b}, ...})
	./main.go:196 +0x8ce
main.cmd.func1(0xc00029c400?, {0x1a3b72d?, 0x4?, 0x1a3b731?})
	./main.go:111 +0x5b
github.com/spf13/cobra.(*Command).execute(0xc00028a608, {0xc0001b4010, 0xa, 0xa})
	github.com/spf13/[email protected]/command.go:940 +0x882
github.com/spf13/cobra.(*Command).ExecuteC(0xc00028a608)
	github.com/spf13/[email protected]/command.go:1068 +0x3a5
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/[email protected]/command.go:992
main.main()
	./main.go:76 +0x18

Helm Chart configuration

standard

What happened

Tried to create a minimal ingress object in order to attach a traffic policy to. Since its a redirect, I don't care about the path or other rules, just the domain, so I tried to create an ingress object like this

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    k8s.ngrok.com/traffic-policy: k8s-redirect
  creationTimestamp: null
  name: k8s-redirect
  namespace: ngrok-operators
spec:
  ingressClassName: ngrok
  rules:
  - host: k8s.dev-ngrok.com

This ingress object causes the leader pod to crash, so a new pod takes over as leader which crashes, which continues till they are all down and ingress stops working

NAME                                                     READY   STATUS             RESTARTS        AGE
kubernetes-ingress-controller-manager-849d479c4d-2lqn8   0/1     CrashLoopBackOff   6 (2m40s ago)   10m
kubernetes-ingress-controller-manager-849d479c4d-hhxnm   0/1     CrashLoopBackOff   6 (118s ago)    10m
kubernetes-ingress-controller-manager-849d479c4d-lb5m5   0/1     CrashLoopBackOff   6 (2m22s ago)   10m

What you think should happen instead

At the very least this should not cause the controller to crash which would result in an outage for someone.
Ideally, this configuration would be accepted and result in a end with no routes assuming that would work with a traffic policy to handle redirects

How to reproduce

create an ingress object in the cluster with no rules, sit back and watch them start crashing

@alex-bezek alex-bezek added bug Something isn't working area/controller Issues dealing with the controller needs-triage Issues that need triage labels Sep 11, 2024
@alex-bezek
Copy link
Collaborator Author

Looks like its probably

for _, path := range ing.Spec.Rules[0].HTTP.Paths {

@brandonros
Copy link

If you configure an ingress by accident with no host at all,

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: chess-bot-ngrok
  namespace: chess-bot
spec:
  ingressClassName: ngrok
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: chess-bot
            port:
              number: 8080

ngrok-operator-manager-6cc495dbbc-vhn6v {"level":"info","ts":"2024-11-10T15:35:02Z","msg":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","controller":"ingress","controllerGroup":"networking.k8s.io","controllerKind":"Ingress","Ingress":{"name":"chess-bot-ngrok","namespace":"chess-bot"},"namespace":"chess-bot","name":"chess-bot-ngrok","reconcileID":"c8a85fa3-ecef-4ab0-9ffd-f31766b2e1af"}
ngrok-operator-manager-6cc495dbbc-vhn6v panic: runtime error: invalid memory address or nil pointer dereference [recovered]
ngrok-operator-manager-6cc495dbbc-vhn6v     panic: runtime error: invalid memory address or nil pointer dereference
ngrok-operator-manager-6cc495dbbc-vhn6v [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xb5eab0]
ngrok-operator-manager-6cc495dbbc-vhn6v 
ngrok-operator-manager-6cc495dbbc-vhn6v goroutine 429 [running]:
ngrok-operator-manager-6cc495dbbc-vhn6v sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
ngrok-operator-manager-6cc495dbbc-vhn6v     sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:116 +0x19c
ngrok-operator-manager-6cc495dbbc-vhn6v panic({0x132ea00?, 0x24c82e0?})
ngrok-operator-manager-6cc495dbbc-vhn6v     runtime/panic.go:785 +0x124
ngrok-operator-manager-6cc495dbbc-vhn6v k8s.io/api/networking/v1.(*Ingress).GetObjectKind(0x24d3620?)
ngrok-operator-manager-6cc495dbbc-vhn6v     <autogenerated>:1
ngrok-operator-manager-6cc495dbbc-vhn6v k8s.io/client-go/tools/reference.GetReference(0x4000170e00, {0x17f9960, 0x0})
ngrok-operator-manager-6cc495dbbc-vhn6v     k8s.io/[email protected]/tools/reference/ref.go:59 +0x124
ngrok-operator-manager-6cc495dbbc-vhn6v k8s.io/client-go/tools/record.(*recorderImpl).generateEvent(0x400078f7c0, {{0x1811d78?, 0x4000054700?}, 0x1?}, {0x17f9960, 0x0}, 0x0, {0x155a76a, 0x7}, {0x1567a2c, ...}, ...)
ngrok-operator-manager-6cc495dbbc-vhn6v     k8s.io/[email protected]/tools/record/event.go:423 +0x6c
ngrok-operator-manager-6cc495dbbc-vhn6v k8s.io/client-go/tools/record.recorderImplLogger.Event(...)
ngrok-operator-manager-6cc495dbbc-vhn6v     k8s.io/[email protected]/tools/record/event.go:497
ngrok-operator-manager-6cc495dbbc-vhn6v k8s.io/client-go/tools/record.recorderImplLogger.Eventf({0x400078f7c0?, {{0x1811d78?, 0x4000054700?}, 0xffffa8ea9130?}}, {0x17f9960, 0x0}, {0x155a76a, 0x7}, {0x1567a2c, 0x12}, ...)
ngrok-operator-manager-6cc495dbbc-vhn6v     k8s.io/[email protected]/tools/record/event.go:501 +0xa8
ngrok-operator-manager-6cc495dbbc-vhn6v sigs.k8s.io/controller-runtime/pkg/internal/recorder.(*lazyRecorder).Eventf(0x400025bb00, {0x17f9960, 0x0}, {0x155a76a, 0x7}, {0x1567a2c, 0x12}, {0x1589d82, 0x27}, {0x400092bbf0, ...})
ngrok-operator-manager-6cc495dbbc-vhn6v     sigs.k8s.io/[email protected]/pkg/internal/recorder/recorder.go:169 +0xec
ngrok-operator-manager-6cc495dbbc-vhn6v github.com/ngrok/ngrok-operator/internal/controller/ingress.(*IngressReconciler).Reconcile(0x40004a32c0, {0x180d768, 0x4000b53c80}, {{{0x4000c94147, 0x9}, {0x4000c94150, 0xf}}})
ngrok-operator-manager-6cc495dbbc-vhn6v     github.com/ngrok/ngrok-operator/internal/controller/ingress/ingress_controller.go:107 +0x534
ngrok-operator-manager-6cc495dbbc-vhn6v sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1812048?, {0x180d768?, 0x4000b53c80?}, {{{0x4000c94147?, 0xb?}, {0x4000c94150?, 0x0?}}})
ngrok-operator-manager-6cc495dbbc-vhn6v     sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119 +0x80
ngrok-operator-manager-6cc495dbbc-vhn6v sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0x4000195e00, {0x180d7a0, 0x40002031d0}, {0x13ba8c0, 0x400045bea0})
ngrok-operator-manager-6cc495dbbc-vhn6v     sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316 +0x2d0
ngrok-operator-manager-6cc495dbbc-vhn6v sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0x4000195e00, {0x180d7a0, 0x40002031d0})
ngrok-operator-manager-6cc495dbbc-vhn6v     sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266 +0x158
ngrok-operator-manager-6cc495dbbc-vhn6v sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
ngrok-operator-manager-6cc495dbbc-vhn6v     sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227 +0x70
ngrok-operator-manager-6cc495dbbc-vhn6v created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 126
ngrok-operator-manager-6cc495dbbc-vhn6v     sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223 +0x400
Stream closed EOF for ngrok/ngrok-operator-manager-6cc495dbbc-vhn6v (ngrok-operator)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/controller Issues dealing with the controller bug Something isn't working needs-triage Issues that need triage
Projects
None yet
Development

No branches or pull requests

2 participants