Using the Argo CD based OpenShift GitOps Operator (Recommended)

This methodology using the Argo CD based OpenShfit GitOps Operator is the preferred method of deploying policies via GitOps as a concept as of RHACM 2.9,

Prerequisite: OpenShift GitOps Operator on the RHACM Hub Cluster

This section outlines how to install and configure the OpenShift GitOps Operator on the RHACM Hub Cluster. If this has already been done, you can skip this section.

Installing OpenShift GitOps Operator on the RHACM Hub Cluster

This section will outline how to use the CLI to install the OpenShift GitOps Operator based on the following documentation: Product Documentation: Red Hat OpenShift GitOps: Installing GitOps.

Create a openshift-gitops-operator namespace if one doesn't already exist, and switch to it.

$ oc create ns openshift-gitops-operator
$ oc project openshift-gitops-operator

The reason you're using the oc create namespace is running oc new-project will fail. For example:?

$ oc new-project openshift-gitops-operator
Error from server (Forbidden): project.project.openshift.io "openshift-gitops-operator" is forbidden: cannot request a project starting with "openshift-"

Apply the OperatorGroup object.

$ cat ./argocd/gitops-operator-group.yaml
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-gitops-operator
  namespace: openshift-gitops-operator
spec:
  upgradeStrategy: Default
$ oc apply -f ./argocd/gitops-operator-group.yaml

Apply the Subscription object to subscribe the operator in the openshift-gitops-operator namespace.

$ cat ./argocd/openshift-gitops-sub.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: openshift-gitops-operator
  namespace: openshift-gitops-operator
spec:
  channel: latest 
  installPlanApproval: Automatic
  name: openshift-gitops-operator 
  source: redhat-operators 
  sourceNamespace: openshift-marketplace
$ oc apply -f ./argocd/openshift-gitops-sub.yaml

After the installation is complete, verify that all the pods in the openshift-gitops namespace are running. This can take a few minutes depending on your network to even return anything.

$ oc get pods -n openshift-gitops
NAME                                                      	      READY   STATUS    RESTARTS   AGE
cluster-b5798d6f9-zr576                                   	      1/1 	  Running   0          65m
kam-69866d7c48-8nsjv                                      	      1/1 	  Running   0          65m
openshift-gitops-application-controller-0                 	      1/1 	  Running   0          53m
openshift-gitops-applicationset-controller-6447b8dfdd-5ckgh       1/1 	  Running   0          65m
openshift-gitops-dex-server-569b498bd9-vf6mr                      1/1     Running   0          65m
openshift-gitops-redis-74bd8d7d96-49bjf                   	      1/1 	  Running   0          65m
openshift-gitops-repo-server-c999f75d5-l4rsg              	      1/1 	  Running   0          65m
openshift-gitops-server-5785f7668b-wj57t                  	      1/1 	  Running   0          53m

Verify that the pod/s in the openshift-gitops-operator namespace are running.

$ oc get pods -n openshift-gitops-operator
NAME                                                            READY   STATUS    RESTARTS   AGE
openshift-gitops-operator-controller-manager-664966d547-vr4vb   2/2     Running   0          65m

Configure Basic RBAC Policy For OpenShift GitOps

OpenShift GitOps manages access to Argo CD resources through the RBAC policy set under Argo CD instance. By default any cluster-admin users don't have rights to Argo CD.

Review the current RBAC policy.

$ oc get argocd -n openshift-gitops -o yaml | grep -i rbac -A10
rbac:
  defaultPolicy: ""
  policy: |
    g, system:cluster-admins, role:admin    <--[group `system:cluster-admins` has `role:admin`]
    g, cluster-admins, role:admin           <--[group `cluster-admins`        has `role:admin`]
  scopes: '[groups]'
redis:
  resources:
    limits:
      cpu: 500m
  scopes: '[groups]'

Check to see if your current user is a member of any groups.
```
$ oc get groups
```
If your user wasn't a member of any of the groups mentioned in step one, we can either add them to one of those groups or create one. In this case, we are making one argocdadmins.
```
$ oc adm groups new argocdadmins
```

Add your user to the group.

$ oc adm groups add-users argocdadmins $(oc whoami)

Edit the argocd instance in the openshift-gitops namespace to add a new admin group by adding g, argocdadmins, role:admin.

$ oc edit argocd -n openshift-gitops
[...]
  rbac:
    defaultPolicy: ""
    policy: |
      g, system:cluster-admins, role:admin
      g, cluster-admins, role:admin
      g, argocdadmins, role:admin
    scopes: '[groups]'
[...]

Configure OpenShift GitOps to Integrate with the Kustomize Policy Generator Plugin

Now that OpenShift GitOps (Argo CD) has been installed and basic RBAC enabled, we can configure it to support the Kustomize Policy Generator plugin, and integrate it with RHACM.

We will need to modify ArgoCD instance to provision a init container to copy the Kustomize Policy Generator plugin to the GitOp Container, and add the --enable-alpha-plugins argument. First backup the existing ArgoCD object for reference.
```
$ oc get argocd openshift-gitops -n openshift-gitops -o yaml > openshift-gitops.yaml
```

Now we will apply apply a patch file. Further details on this can be found here. Not that this differs from the RHACM 2.10 docs which specify an object of apiVersion: argoproj.io/v1alpha1 but for GitOps 1.12, the object created is of type apiVersion: argoproj.io/v1beta1, therefore this is why the patch differs from the aforementioned documentation. Also there is a error in the mount points in the docs at time of writing, which I have corrected in my repository.

$ cat ./argocd/argocd-patch.yaml
apiVersion: argoproj.io/v1beta1
kind: ArgoCD
metadata:
  name: openshift-gitops
  namespace: openshift-gitops
spec:
  kustomizeBuildOptions: --enable-alpha-plugins
  repo:
    env:
    - name: KUSTOMIZE_PLUGIN_HOME
      value: /etc/kustomize/plugin
    initContainers:
    - args:
      - -c
      - cp /policy-generator/PolicyGenerator-not-fips-compliant /policy-generator-tmp/PolicyGenerator
      command:
      - /bin/bash
      image: registry.redhat.io/rhacm2/multicluster-operators-subscription-rhel9:v2.10
      name: policy-generator-install
      volumeMounts:
      - mountPath: /policy-generator-tmp
        name: policy-generator
    volumeMounts:
    - mountPath: /etc/kustomize/plugin/policy.open-cluster-management.io/v1/policygenerator
      name: policy-generator
    volumes:
    - emptyDir: {}
      name: policy-generator
$ oc -n openshift-gitops patch argocd openshift-gitops --type merge --patch "$(cat ./argocd/argocd-patch.yaml)"

You should see a new openshift-gitops-repo-server-XXXXXXXX-XXXXX pod start to spin up. Waitt for it to become ready 1/1.

$ oc get pods -n openshift-gitops
NAME                                                          READY   STATUS    RESTARTS   AGE
cluster-84c744d68d-k6ngt                                      1/1     Running   0          6m40s
kam-67b754468-gzrpl                                           1/1     Running   0          6m40s
openshift-gitops-application-controller-0                     1/1     Running   0          6m37s
openshift-gitops-applicationset-controller-84ddbb9469-hxvwz   1/1     Running   0          6m37s
openshift-gitops-dex-server-864c46c7b9-rx6fk                  1/1     Running   0          6m37s
openshift-gitops-redis-5684c6fc5b-gc87d                       1/1     Running   0          6m38s
openshift-gitops-repo-server-75d686dbbf-vc46m                 0/1     Running   0          6s
openshift-gitops-repo-server-774f57885c-vlg84                 1/1     Running   0          6m38s
openshift-gitops-server-76cf7bd6c8-rg6xb                      1/1     Running   0          6m37s

Now that OpenShift GitOps has the binary available, we will need to grant access for the ArgoCD instance to create policies on the RHACM hub instance. First create a custom ClusterRole with full rights to the various open-cluster-management.io objects we want Argo CD to interact and manage on our behalf.

$ cat ./argocd/openshift-gitops-policy-admin.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: openshift-gitops-policy-admin
rules:
  - verbs:
      - get
      - list
      - watch
      - create
      - update
      - patch
      - delete
    apiGroups:
      - policy.open-cluster-management.io
    resources:
      - policies
      - policysets
      - placementbindings
  - verbs:
      - get
      - list
      - watch
      - create
      - update
      - patch
      - delete
    apiGroups:
      - apps.open-cluster-management.io
    resources:
      - placementrules
  - verbs:
      - get
      - list
      - watch
      - create
      - update
      - patch
      - delete
    apiGroups:
      - cluster.open-cluster-management.io
    resources:
      - placements
      - placements/status
      - placementdecisions
      - placementdecisions/status
$ oc apply -f ./argocd/openshift-gitops-policy-admin.yaml

Lastly create a ClusterRoleBinding to grant Argo CD access to the newly created ClusterRole.

$ cat ./argocd/openshift-gitops-policy-admin-clusterolebinding.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: openshift-gitops-policy-admin
subjects:
  - kind: ServiceAccount
    name: openshift-gitops-argocd-application-controller
    namespace: openshift-gitops
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: openshift-gitops-policy-admin
$ oc apply -f ./argocd/openshift-gitops-policy-admin-clusterolebinding.yaml

Deploying our policies

Apply labels to your various managed clusters to reflect their status as env=hub, env=dev, or env=prod. This is used to correctly assigned the banner color and message in the various files in policies/hub-cluster-templates-configs/. You can also create other environment names if these are not set.
We're first going to create a policies namespace to deploy our Policies in.
```
$ oc create ns policies
$ oc project policies
```

The policies here are expected to run on all clusters, so we're going to bind the newly created policies namespace to the default global ClusterSet.

$ cat ./argocd/global-managedclustersetbinding.yaml 
apiVersion: cluster.open-cluster-management.io/v1beta2
kind: ManagedClusterSetBinding 
metadata:
  name: global
  namespace: policies
spec:
  clusterSet: global
$ oc create -f ./argocd/global-managedclustersetbinding.yaml

Now we will create an ArgoCD application to listen to this repository and push out the Policies.

$ cat ./argocd/gitops-policies-application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: gitops-policies
  namespace: openshift-gitops
spec:
  destination:
    namespace: policies
    server: https://kubernetes.default.svc
  project: default
  source:
    repoURL: https://github.com/kfrankli/gitops-rhacm-policies.git
    targetRevision: main
    path: policies
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
$ oc create -f ./argocd/gitops-policies-application.yaml

We should be able to see our application deployed in Argo CD. To get to the Argo CD console, retrieve the route:
```
$ oc get route openshift-gitops-server -n openshift-gitops
```
If you go to the RHACM console you should see the deployed policies.

References

Product Documentation: RHACM: GitOps Overview
Product Documentation: OpenShift: CI/CD: GitOps
Product Documentation: OpenShift GitOps
Product Documentation: OpenShift GitOps: Configuring Argo CD RBAC
Product Documentation: RHACM: Managing policy definitions with OpenShift Container Platform GitOps (Argo CD)
Red Hat Solutions: Unable to create application with user having cluster-admin privileges in Argo CD.
Red Hat Solutions: Add a Kustomize plugin to OpenShift GitOps in RHOCP4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

argocd.md

argocd.md

Using the Argo CD based OpenShift GitOps Operator (Recommended)

Prerequisite: OpenShift GitOps Operator on the RHACM Hub Cluster

Installing OpenShift GitOps Operator on the RHACM Hub Cluster

Configure Basic RBAC Policy For OpenShift GitOps

Configure OpenShift GitOps to Integrate with the Kustomize Policy Generator Plugin

Deploying our policies

References

Files

argocd.md

Latest commit

History

argocd.md

File metadata and controls

Using the Argo CD based OpenShift GitOps Operator (Recommended)

Prerequisite: OpenShift GitOps Operator on the RHACM Hub Cluster

Installing OpenShift GitOps Operator on the RHACM Hub Cluster

Configure Basic RBAC Policy For OpenShift GitOps

Configure OpenShift GitOps to Integrate with the Kustomize Policy Generator Plugin

Deploying our policies

References