PVC Monitoring not working as expected

[gorokhovdenis]

Describe the bug

When using kwatch to monitor PVC usage, even when the PVC exceeds the threshold set in the configuration, there are no logs or notifications. Even though the pvcMonitor appears to be initialized and started in the code, it doesn't produce logs that it has started, unlike the pod-crash controller which does log its start-up.

To Reproduce

Steps to reproduce the behavior:

1. Set up kwatch as per the README instructions.
2. Configure the pvcMonitor to be enabled with a threshold (e.g., 80%).
3. Use a PVC in a namespace that kwatch is monitoring and fill it to exceed the threshold.
4. Observe that there are no logs or notifications indicating the PVC threshold breach.

Expected behavior

When a PVC's usage exceeds the specified threshold, I expected kwatch to send a notification. Additionally, I expected some logs indicating the pvcMonitor has started and is actively monitoring PVCs.

Actual behavior

There are no logs indicating pvcMonitor initialization or start-up, and there are no notifications sent out even when PVC usage exceeds the set threshold.

Version/Commit

[email protected]
Kubernetes Cluster: v1.25.6

Additional Information:

After reviewing the code, the PVC monitoring feature seems to be initiated correctly. I also modified the ClusterRole to ensure the necessary permissions for monitoring PVCs and PVs were granted. The current permissions for the kwatch ClusterRole include:

Resources: pods, pods/log, events, nodes, persistentvolumeclaims, persistentvolumes
Verbs: get, watch, list

Despite these configurations and permissions, the PVC monitoring feature does not appear to be working. I'm seeking guidance on troubleshooting this issue further or any potential fixes.

Configuration for Reproduction

Here is the relevant portion of my config.yaml:

pvcMonitor:
  enabled: true
  threshold: 80
maxRecentLogLines: 20
disableUpdateCheck: true
alert:
  slack:
    webhook: https://hooks.slack.com/services/*********
    title: K8S Slack
namespaces:
  - kwatch

Here are the Kubernetes resources I set up to reproduce the issue:

cat kwatch-test.yaml
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kwatch-test-pvc
  namespace: kwatch
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: nfs-storage
  resources:
    requests:
      storage: 100Mi
---
apiVersion: v1
kind: Pod
metadata:
  name: test-pod
  namespace: kwatch
spec:
  containers:
  - name: test-container
    image: nginx
    volumeMounts:
    - name: test-volume
      mountPath: /usr/share/nginx/html
  volumes:
  - name: test-volume
    persistentVolumeClaim:
      claimName: kwatch-test-pv

Verification of PVC Usage

To demonstrate the PVC usage, I executed a du command in the test-pod:

kubectl exec test-pod -n kwatch -- du -sh /usr/share/nginx/html
91M	/usr/share/nginx/html