Usability and Security enhancements for NDB Kubernetes Operator

[RashmiD25]

What this PR does / why we need it:
Problem Statement: The current system faces several security and operational challenges that need urgent attention:

1. Securing Secrets: Currently, sensitive information is stored in plaintext, making it vulnerable to unauthorized access and potential breaches. We need to implement robust security measures such as Vault or External Secrets to ensure the secrets are securely managed and encrypted.
2. Access Control for NDB Resources: There is a need to establish a strict access control mechanism for NDB resources within the Kubernetes environment. This involves configuring ServiceAccounts, Roles, and related policies to restrict unauthorized access and ensure that only authorized users can interact with NDB resources.
3. Cluster Identification: The system currently uses cluster IDs as inputs, which can be confusing and error-prone. To enhance clarity and ease of use, we should change the input to cluster names and internally resolve these names to their corresponding IDs.

Enhancements to Secrets Management:
We have implemented a solution using RBAC policies at the Kubernetes cluster level to create new users, and restrict their secrets only to their context, such that no two users in the same K8 cluster can access each other's secrets.

As per the above diagram, there are two main contexts namely the administrator context (Usually known as “minikube” or any other name depending on the K8 cluster), and the user contexts. The administrator has the role of creating users and assigning their NDB secrets, while the users have the role to view their secrets and also depending on the access they have been granted, they can perform certain NDB operations as well.

In addition to the contexts, there are three namespaces namely the administrator namespace, the user namespaces and the ndb-operator-namespace.

Steps for recreation of the implementation:

1. Generate client certificates for the users, in this case it is user1 and user2 using the folowing commands:
   openssl genpkey -algorithm RSA -out user1-key.pem
   openssl req -new -key user1-key.pem -out user1.csr -subj “/CN=user1”
   Next, approve the certificate signing requests:
   openssl x509 -req -in user1.csr -CA /path_to_cluster_ca_crt/ca.crt
   -CAkey /path_to_cluster_ca_crt/ca.key -CAcreateserial -out user1.crt
2. Now, create the users and their respective contexts:
   kubectl config set-credentials user1 –client-certificate=user1.crt –client-key=user1.pem
   kubectl create namespace user1-namespace
   kubectl config set-context user1-context –cluster= –namespace=user1-namespace –user=user1
3. Now the cluster role bindings need to be created for each user. To do this create two YAML files for each user. One YAML file is used to define the role of the user within the cluster and another one is used to bind the same to the cluster. Note, each of these roles need to be in the created user’s namespace.
   With these roles created, save the YAML files and apply the same using the commands
   kubectl apply -f user-namespace-role.yaml
   kubectl apply -f user-namespace-rolebinding.yaml
4. Now we need to create another YAML file which will allow the user to be bound to only certain NDB resources. Some of these roles are - ndb-operator-manager-role and ndb-operator-viewer-role. The functionality of these roles have been discussed in the sections above.
   Apply the cluster role using the command
   kubectl apply -f user1-ndb-rolebinding.yaml
5. Finally we need to create the secrets for each user. To create the secret for a user, we can create the following YAML files and issue the commands to create them -
   Save this file as user1-secret and apply the same using the command - kubectl apply -f user1-secret.yaml
   Similarly create the database secret using the YAML file
   Save this as user1-db.yaml and apply using kubectl.
6. With the secrets created, we need to secure them using RBAC such that only the intended user has access to the same. To do this we need to create a role using the following YAML file. Call this as user1-role.yaml
   Save and apply the same using kubectl apply.
7. With the users and secrets created, we can now switch to a user context, and try accessing secrets. It will be observed that it can only access secrets using the kubectl get secrets command only for the secrets it has been granted access to in the previous file.
   Thus with the aid of the above steps, we can now create a secret and secure the same using RBAC. Moreover, a particular user can only apply the secrets that they have been granted access to using an appropriate manifest and not any other secret that belongs to another user.

Now we can resolve the clusterID using the clusterName internally.

How Has This Been Tested?:

To test the chosen secrets management and access control system, various users will need to be set up and configured. Those users will be given different access permissions to various fake secrets that will be created to make sure access is restricted. Below are more specific tests that will be created to test access to secrets.

• Ensure the Secret Management System is Installed: This test will ensure that the chosen secret management system has been successfully installed on the system. It will call various functions of the management system to ensure that it is present.

• Verify Configuration: This test’s main purpose is to ensure that the configuration of the system is correct. It will check the desired authentication methods and policies by using various users using those authentication methods.

• Verify Secret Creation: Fake secrets will be created and will be tested to ensure that only verified users can access them. This test will also ensure that unauthorized users cannot create secrets.

• Verify Retrieval of Secrets: The fake secrets will be retrieved both inside the system’s storage through its database as well as through an interface. This test will also ensure that users cannot retrieve secrets they are not supposed to.

• Verify System Retrieval of Secrets: This test will ensure that authorized workloads within the system are able to access secrets that they are required to access. This test will also ensure that unauthorized workloads are unable to access certain secrets.

• Verify encryption at all Levels: Tests will be created to ensure that secrets that are being transferred from one component to another are encrypted and are inaccessible without the proper authorization.

• Verify Proper Access Control: Tests will be created to ensure that unauthorized users cannot access resources they shouldn’t. Fake resources and fake users will be created with different authorization to ensure this. It will also ensure that authorized users can access what they should be able to.