Test: Adding e2e tests, test README, and pipeline infra for tests (#695)

### PR Process
- Add pull request template, including a checklist for testing
- Add a semantic conventional commit check for PR titles (for creating
the changelog for releases)

### Pipeline
- Split pipeline into two stages: `Build` (images and charts) and
`Deploy` (on CI/CD clusters)
- Add an environment lock to the `Deploy` stage so that only one merge
to main can acquire that lock at a time
- Add running the TestKube tests on the CI/CD clusters to the `Deploy`
stage, so that a new version cannot be deployed on the CI/CD clusters
until the current tests have finished running

### Docs
- Add a testing README for processes, Ginkgo and TestKube info, how to
run the tests and bootstrap clusters, how to add new tests, etc.

### Tests
- Add the `otelcollector/tests` folder that has the starting e2e tests,
split up into test suites

---------

Co-authored-by: bragi92 <[email protected]>

.github/pull_request_template.md

@@ -0,0 +1,33 @@
+
+[comment]: # (Note that your PR title should follow the conventional commit format: https://conventionalcommits.org/en/v1.0.0/#summary)
+# PR Description
+
+[comment]: # (The below checklist is for PRs adding new features. If a box is not checked, add a reason why it's not needed.)
+# New Feature Checklist
+
+- [ ] List telemetry added about the feature.
+- [ ] Link to the one-pager about the feature.
+- [ ] List any tasks necessary for release (3P docs, AKS RP chart changes, etc.) after merging the PR.
+- [ ] Attach results of scale and perf testing.
+
+[comment]: # (The below checklist is for code changes. Not all boxes necessarily need to be checked. Build, doc, and template changes do not need to fill out the checklist.)
+# Tests Checklist
+
+- [ ] Have end-to-end Ginkgo tests been run on your cluster and passed? To bootstrap your cluster to run the tests, follow [these instructions](/otelcollector/test/README.md#bootstrap-a-dev-cluster-to-run-ginkgo-tests).
+  - Labels used when running the tests on your cluster:
+    - [ ] `operator`
+    - [ ] `windows`
+    - [ ] `arm64`
+    - [ ] `arc-extension`
+- [ ] Have new tests been added? For features, have tests been added for this feature? For fixes, is there a test that could have caught this issue and could validate that the fix works?
+  - [ ] Is a new scrape job needed?
+    - [ ] The scrape job was added to the folder [test-cluster-yamls](/otelcollector/test/test-cluster-yamls/) in the correct configmap or as a CR. 
+  - [ ] Was a new test label added?
+    - [ ] A string constant for the label was added to [constants.go](/otelcollector/test/utils/constants.go).
+    - [ ] The label and description was added to the [test README](/otelcollector/test/README.md).
+    - [ ] The label was added to this [PR checklist](/.github/pull_request_template).
+    - [ ] The label was added as needed to [testkube-test-crs.yaml](/otelcollector/test/testkube/testkube-test-crs.yaml).
+  - [ ] Are additional API server permissions needed for the new tests?
+    - [ ] These permissions have been added to [api-server-permissions.yaml](/otelcollector/test/testkube/api-server-permissions.yaml).
+  - [ ] Was a new test suite (a new folder under `/tests`) added?
+    - [ ] The new test suite is included in [testkube-test-crs.yaml](/otelcollector/test/testkube/testkube-test-crs.yaml).

internal/docs/BUILDANDRELEASE.md

@@ -66,6 +66,7 @@ Each merge into `main` will push the image to the public mcr and deploy to the d
 
 ## Release Process
 - **PR 1**: Bump the version in the VERSION file following semantic versioning.
+    - Add the latest `addon-token-adapter-linux` and `addon-token-adapter-windows` versions in the values-template.yaml file by checking the version [here](https://msazure.visualstudio.com/CloudNativeCompute/_git/aks-rp?path=%2Fccp%2Fcharts%2Fkube-control-plane%2Ftemplates%2F_images.tpl&_a=contents&version=GBmaster).
     - If you know your PR with the last feature changes will be the last one before the release, you can do this then.
     - **Build 1**: The `values.yaml` and `Chart.yaml` templates for the HELM chart will automatically be replaced with the image tag and the HELM chart version during the CI/CD build.
 - **Release**: Create a release in [ADO](https://github-private.visualstudio.com/azure/_release?_a=releases&view=mine&definitionId=79).

internal/docs/UpgradeDependencies.md

@@ -7,7 +7,7 @@ Updating KSM and NE charts
 2. Update the Version and Appversion in all the yaml files for both 1p and addon charts from the example PR above.
 3. Update both the images in our pipeline build file : azure-pipeline-build.yml
 4. Before creating a pull request in the main repository for both KSM and NE, make sure to run the "build and push dependent helm chart pipeline". You can find the pipeline [here](https://github.com/Azure/prometheus-collector/actions/workflows/build-and-push-dependent-helm-charts.yml).
-5. The current service principal responsible for controlling the Azure Container Registry (ACR) pull and push on the containerinsightsprod registry can be accessed [here](https://ms.portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationMenuBlade/~/Credentials/appId/c58817c2-f216-4308-bb6c-126e0d82b824). If necessary, you can renew the password for the service principal from that page. The current password's expiry date is 7/26/2024.
+5. The current service principal responsible for controlling the Azure Container Registry (ACR) pull and push on the containerinsightsprod registry can be accessed [here](https://ms.portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationMenuBlade/~/Credentials/appId/c58817c2-f216-4308-bb6c-126e0d82b824). If necessary, you can renew the password for the service principal from that page. The current password's expiry date is 3/13/2024.
 6. If you renew the password, remember to update the following two secrets in the [GitHub Actions settings](https://github.com/Azure/prometheus-collector/settings/secrets/actions):
     a. MANAGED_PROM_SERVICE_PRINCIPAL_OBJECT_ID (This will likely remain the same unless you create a new service principal).
     b. MANAGED_PROM_SERVICE_PRINCIPAL_PASSWORD

internal/referenceapp/prometheus-mtls-ref-app.yaml

@@ -24,8 +24,10 @@ spec:
           ports:
             - containerPort: 2112
               protocol: TCP
+              name: "weather-app"
             - containerPort: 2113
               protocol: TCP
+              name: "untyped-metrics"
       nodeSelector:
           kubernetes.io/os: linux
           architecture: amd64

internal/referenceapp/prometheus-reference-app.yaml

@@ -25,16 +25,19 @@ spec:
           ports:
             - containerPort: 2112
               protocol: TCP
+              name: "weather-app"
             - containerPort: 2113
               protocol: TCP
+              name: "untyped-metrics"
         - name: prometheus-reference-app-python
           image: mcr.microsoft.com/azuremonitor/containerinsights/cidev/prometheus-collector/images:6.7.5-main-09-12-2023-8fbde9ca-ref-app-python
           ports:
           - containerPort: 2114
             protocol: TCP
+            name: "python-client"
       nodeSelector:
           kubernetes.io/os: linux
-          architecture: amd64
+          kubernetes.io/arch: amd64
 ---
 apiVersion: v1
 kind: Service

internal/windowsExporterInstaller/windows-exporter-daemonset.yaml

@@ -1,3 +1,10 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: monitoring
+  labels:
+    name: monitoring
+---
 apiVersion: apps/v1
 kind: DaemonSet
 metadata:
@@ -25,7 +32,7 @@ spec:
           command: ["powershell"]
           args: ["New-NetFirewallRule", "-DisplayName", "'windows-exporter'", "-Direction", "inbound", "-Profile", "Any", "-Action", "Allow", "-LocalPort", "9182", "-Protocol", "TCP"]
       containers:
-      - args:
+      - args: 
         - --config.file=%CONTAINER_SANDBOX_MOUNT_POINT%/config.yml
         name: windows-exporter
         image: ghcr.io/prometheus-community/windows-exporter:latest