Migrate singlestore collector example to kubernetes (#630)

other-examples/collector/singlestore/README.md

@@ -1,33 +1,79 @@
-# SingleStore OpenTelemetry metrics example setup
+# Monitoring Singlestore with OpenTelemetry Collector
 
-This example shows a setup for running a prometheus OpenTelemetry Collector in a docker container to scrape metrics from your SingleStore environment and post them the New Relic OTLP Collector Endpoint.
+This simple example demonstrates monitoring [Singlestore](https://www.singlestore.com/) prometheus metrics with the [OpenTelemetry collector](https://opentelemetry.io/docs/collector/), using the [prometheus receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/prometheusreceiver) and sending the data to New Relic via OTLP.
 
-TODO: add link to doc site when available
+## Requirements
 
-## Prerequisites
-
-1. You must have a Docker daemon running.
-2. You must have [Docker compose](https://docs.docker.com/compose/) installed.
-3. You must have a SingleStore workspace running.
+* You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. Docker desktop [includes a standalone Kubernetes server and client](https://docs.docker.com/desktop/kubernetes/) which is useful for local testing.
+* [A New Relic account](https://one.newrelic.com/)
+* [A New Relic license key](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key)
+* [A Singlestore account](https://www.singlestore.com/cloud-trial/) with a [workspace group](https://docs.singlestore.com/cloud/getting-started-with-singlestore-helios/about-workspaces/creating-and-using-workspaces/)
+* [A Singlestore API key](https://support.singlestore.com/hc/en-us/articles/12396018910228-Creating-Management-API-Key)
 
 ## Running the example
 
-First, set your environment variables in the `.env` file in this directory. For more information on the individual variables, reference the docs available below.
+1. Update the `NEW_RELIC_API_KEY` values `SINGLESTORE_API_KEY` values in [secrets.yaml](./k8s/secrets.yaml) to your New Relic license key, and singlestore API key respectively. See [Singlestore docs](https://support.singlestore.com/hc/en-us/articles/12396018910228-Creating-Management-API-Key) for obtaining API key / secret.
+
+    ```yaml
+    # ...omitted for brevity
+   stringData:
+      # New Relic API key to authenticate the export requests.
+      # docs: https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key
+      NEW_RELIC_API_KEY: <INSERT_API_KEY>
+      # Set your Singlestore API Key.
+      # docs: https://support.singlestore.com/hc/en-us/articles/12396018910228-Creating-Management-API-Key
+      SINGLESTORE_API_KEY: <INSERT_API_KEY>>
+    ```
+   
+    * Note, be careful to avoid inadvertent secret sharing when modifying `secrets.yaml`. To ignore changes to this file from git, run `git update-index --skip-worktree k8s/secrets.yaml`.
+
+    * If your account is based in the EU, update the `NEW_RELIC_OTLP_ENDPOINT` value in [collector.yaml](./k8s/collector.yaml) the endpoint to: [https://otlp.eu01.nr-data.net](https://otlp.eu01.nr-data.net)
+
+    ```yaml
+    # ...omitted for brevity
+   env:
+     # The default US endpoint is set here. You can change the endpoint and port based on your requirements if needed.
+     # docs: https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#configure-endpoint-port-protocol
+     - name: NEW_RELIC_OTLP_ENDPOINT
+       value: https://otlp.eu01.nr-data.net
+    ```
+
+2. Set the `SINGLESTORE_ORG_ID` and `SINGLESTORE_WORKSPACE_GROUP_ID` env var values in [collector.yaml](./k8s/collector.yaml). See [Singlestore docs](https://support.singlestore.com/hc/en-us/articles/12396547132564-Workspace-Group-ID-or-Cluster-ID) for details on obtaining org and workspace group ids.
+
+    ```yaml
+   # ...omitted for brevity
+   # The Singlestore Org ID.
+   # docs: https://support.singlestore.com/hc/en-us/articles/12396547132564-Workspace-Group-ID-or-Cluster-ID
+   - name: SINGLESTORE_ORG_ID
+     value: <INSERT_SINGLESTORE_ORG_ID>
+   # The Singlestore Workspace Group ID.
+   # docs: https://support.singlestore.com/hc/en-us/articles/12396547132564-Workspace-Group-ID-or-Cluster-ID
+   - name: SINGLESTORE_WORKSPACE_GROUP_ID
+     value: <INSERT_SINGLESTORE_WORKSPACE_GROUP_ID>
+    ```
 
-Once the variables are set, run the following command from the root directory to start the collector.
+3. Run the application with the following command.
 
-```shell
-cd ./other-examples/collector/singlstore
+    ```shell
+    kubectl apply -f k8s/
+    ```
+
+   * When finished, cleanup resources with the following command. This is also useful to reset if modifying configuration.
+
+   ```shell
+   kubectl delete -f k8s/
+   ```
+
+## Viewing your data
+
+To review your Singlestore data in New Relic, navigate to "New Relic -> Query Your Data". To list the metrics reported, query for:
 
-docker compose up
 ```
+FROM Metric SELECT uniques(metricName) WHERE otel.library.name = 'otelcol/prometheusreceiver' AND metricName LIKE 'singlestoredb%'
+```
+
+See [get started with querying](https://docs.newrelic.com/docs/query-your-data/explore-query-data/get-started/introduction-querying-new-relic-data/) for additional details on querying data in New Relic.
 
-## Local Variable information
+## Additional notes
 
-| Variable | Description | Docs |
-| -------- | ----------- | ---- |
-| **NEW_RELIC_API_KEY** |New Relic Ingest API Key |[API Key docs](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/) |
-| **NEW_RELIC_OTLP_ENDPOINT** |Default US OTLP endpoint is `https://otlp.nr-data.net` | [OTLP endpoint config docs](https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/get-started/opentelemetry-set-up-your-app/#review-settings) |
-| **SINGLESTORE_ORG** | ID of your SingleStore Organization |[SingleStore Organization docs](https://docs.singlestore.com/cloud/user-and-workspace-administration/manage-organizations/)|
-| **SINGLESTORE_WORKSPACE_GROUP** | ID of your SingleStore Workspace Group | [SingleStore workspace docs](https://docs.singlestore.com/cloud/getting-started-with-singlestoredb-cloud/about-workspaces/what-is-a-workspace/) |
-| **SINGLESTORE_API_KEY** | SingleStore API Key | [SingleStore API Key docs](https://support.singlestore.com/hc/en-us/articles/12396018910228-Creating-Management-API-Key)|
+The prometheus receiver includes `service.name` and `service.instance.id` resource attributes derived from job name and target configured in `.receivers.prometheus.config.scrape_configs`. As documented [here](https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-best-practices-resources/#services), New Relic considers any data with `service.name` as a service despite the fact that not all prometheus data sources are services. As a result, you can find a `singlestore` entity under "New Relic -> All Entities -> Services - OpenTelemetry", although the panels will not contain data because the scraped metrics do not represent APM data.

other-examples/collector/singlestore/k8s/collector.yaml

@@ -0,0 +1,91 @@
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: nr-singlestore
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: collector-config
+  namespace: nr-singlestore
+  labels:
+    app.kubernetes.io/name: collector-config
+data:
+  collector-config: |
+    receivers:
+      prometheus:
+        config:
+          scrape_configs:
+            - job_name: "singlestore"
+              scrape_interval: 60s # Do not go any lower than this or you'll hit rate limits
+              static_configs:
+                - targets: ["api.singlestore.com"]
+              scheme: https
+              bearer_token: "${SINGLESTORE_API_KEY}"
+              metrics_path: /v2/organizations/${SINGLESTORE_ORG_ID}/workspaceGroups/${SINGLESTORE_WORKSPACE_GROUP_ID}/metrics
+    
+    processors:
+      batch:
+
+    exporters:
+      otlphttp:
+        endpoint: ${NEW_RELIC_OTLP_ENDPOINT}
+        headers:
+          api-key: ${NEW_RELIC_API_KEY}
+
+    service:
+      pipelines:
+        metrics:
+          receivers: [prometheus]
+          processors: [batch]
+          exporters: [otlphttp]
+---
+apiVersion: v1
+kind: Pod
+metadata:
+  name: collector
+  namespace: nr-singlestore
+  labels:
+    app.kubernetes.io/name: collector
+spec:
+  containers:
+    - name: collector
+      image: otel/opentelemetry-collector-contrib:0.98.0
+      env:
+        # The default US endpoint is set here. You can change the endpoint and port based on your requirements if needed.
+        # docs: https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#configure-endpoint-port-protocol
+        - name: NEW_RELIC_OTLP_ENDPOINT
+          value: https://otlp.nr-data.net/
+        # The New Relic API key used to authenticate export requests.
+        # Defined in secrets.yaml
+        - name: NEW_RELIC_API_KEY
+          valueFrom:
+            secretKeyRef:
+              name: nr-singlestore-secret
+              key: NEW_RELIC_API_KEY
+        # The Singlestore API key.
+        # Defined in secrets.yaml
+        - name: SINGLESTORE_API_KEY
+          valueFrom:
+            secretKeyRef:
+              name: nr-singlestore-secret
+              key: SINGLESTORE_API_KEY
+        # The Singlestore Org ID.
+        # docs: https://support.singlestore.com/hc/en-us/articles/12396547132564-Workspace-Group-ID-or-Cluster-ID
+        - name: SINGLESTORE_ORG_ID
+          value: <INSERT_SINGLESTORE_ORG_ID>
+        # The Singlestore Workspace Group ID.
+        # docs: https://support.singlestore.com/hc/en-us/articles/12396547132564-Workspace-Group-ID-or-Cluster-ID
+        - name: SINGLESTORE_WORKSPACE_GROUP_ID
+          value: <INSERT_SINGLESTORE_WORKSPACE_GROUP_ID>
+      volumeMounts:
+        - name: collector-config-vol
+          mountPath: /etc/otelcol-contrib
+  volumes:
+    - name: collector-config-vol
+      configMap:
+        name: collector-config
+        items:
+          - key: collector-config
+            path: config.yaml

other-examples/collector/singlestore/k8s/secrets.yaml

@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: nr-singlestore-secret
+  namespace: nr-singlestore
+stringData:
+  # New Relic API key to authenticate the export requests.
+  # docs: https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key
+  NEW_RELIC_API_KEY: <INSERT_API_KEY>
+  # Set your Singlestore API Key.
+  # docs: https://support.singlestore.com/hc/en-us/articles/12396018910228-Creating-Management-API-Key
+  SINGLESTORE_API_KEY: <INSERT_API_KEY>>