Skip to content

Commit

Permalink
Collector redis k8s (#610)
Browse files Browse the repository at this point in the history
* Add redis example

* Update redis example to use k8s instead of docker
  • Loading branch information
jack-berg authored Jun 6, 2024
1 parent e145861 commit dd19a25
Show file tree
Hide file tree
Showing 5 changed files with 186 additions and 5 deletions.
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,12 @@ The [Getting Started Guides](./getting-started-guides/README.md) demonstrate how
OpenTelemetry is a big ecosystem and everything doesn't fit into the goals of the [getting started guides](#getting-started-guides). These "other examples" demonstrate how other areas of OpenTelemetry fit in with New Relic.

* Collector
* [OpenTelemetry Collector with OTLP and New Relic](./other-examples/collector/nr-config)
* [OpenTelemetry Collector with Host Monitoring and New Relic](./other-examples/collector/host-monitoring)
* [OpenTelemetry Collector with Confluent Cloud and New Relic](./other-examples/collector/confluentcloud)
* [OpenTelemetry Collector with Singlestore and New Relic](./other-examples/collector/singlestore)
* [OpenTelemetry Collector with HCP Consul and New Relic](./other-examples/collector/hcp-consul)
* [Telemetry data processing](./other-examples/collector/nr-config)
* [Host monitoring](./other-examples/collector/host-monitoring)
* [Confluent cloud monitoring](./other-examples/collector/confluentcloud)
* [Singlestore monitoring](./other-examples/collector/singlestore)
* [HCP Consul monitoring](./other-examples/collector/hcp-consul)
* [Redis monitoring](./other-examples/collector/redis)
* Java
* [OpenTelemetry Agent New Relic Config](./other-examples/java/agent-nr-config)
* [Micrometer Shim with OTLP Export](./other-examples/java/micrometer-shim)
Expand Down
54 changes: 54 additions & 0 deletions other-examples/collector/redis/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Monitoring Redis with OpenTelemetry Collector

This simple example demonstrates monitoring redis with the [OpenTelemetry collector](https://opentelemetry.io/docs/collector/), using the [redis receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/redisreceiver) and sending the data to New Relic via OTLP.

## Requirements

* You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. Docker desktop [includes a standalone Kubernetes server and client](https://docs.docker.com/desktop/kubernetes/) which is useful for local testing.
* [A New Relic account](https://one.newrelic.com/)
* [A New Relic license key](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key)

## Running the example

1. Update the `NEW_RELIC_API_KEY` value in [secrets.yaml](./k8s/secrets.yaml) to your New Relic license key.

```yaml
# ...omitted for brevity
stringData:
# New Relic API key to authenticate the export requests.
# docs: https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key
NEW_RELIC_API_KEY: <INSERT_API_KEY>
```
* Note, be careful to avoid inadvertent secret sharing when modifying `secrets.yaml`. To ignore changes to this file from git, run `git update-index --skip-worktree k8s/secrets.yaml`.

* If your account is based in the EU, update the `NEW_RELIC_OTLP_ENDPOINT` value in [collector.yaml](./k8s/collector.yaml) the endpoint to: [https://otlp.eu01.nr-data.net](https://otlp.eu01.nr-data.net)

```yaml
# ...omitted for brevity
env:
# The default US endpoint is set here. You can change the endpoint and port based on your requirements if needed.
# docs: https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#configure-endpoint-port-protocol
- name: NEW_RELIC_OTLP_ENDPOINT
value: https://otlp.eu01.nr-data.net
```

2. Run the application with the following command.

```shell
kubectl apply -f k8s/
```

* When finished, cleanup resources with the following command. This is also useful to reset if modifying configuration.

```shell
kubectl delete -f k8s/
```

## Viewing your data

To review your redis data in New Relic, navigate to "New Relic -> All Entities -> Redis instances" and click on the instance with name "redis" to view the instance summary. Click on "Metric explorer" to view all metrics associated with the redis instance, or use [NRQL](https://docs.newrelic.com/docs/query-your-data/explore-query-data/get-started/introduction-querying-new-relic-data/) to perform ad-hoc analysis.

## Additional notes

This example monitors a redis instance defined in [redis.yaml](./k8s/redis.yaml), which is not receiving any load. To use in production, you'll need to modify the `.receivers.redis.endpoint` value in [collector.yaml](k8s/collector.yaml) ConfigMap to point to the endpoint of your redis instance. Additionally, update the `server.address` and `server.port` resource attributes defined in `attributes/redis_metrics` to values which reflect the redis instance being monitored.
89 changes: 89 additions & 0 deletions other-examples/collector/redis/k8s/collector.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: nr-redis
---
apiVersion: v1
kind: ConfigMap
metadata:
name: collector-config
namespace: nr-redis
labels:
app.kubernetes.io/name: collector-config
data:
collector-config: |
receivers:
redis:
# Connect to redis pod defined in redis.yaml using service env vars set by k8s
endpoint: ${REDIS_SERVICE_HOST}:${REDIS_SERVICE_PORT}
metrics:
# Enable redis.maxmemory optional metric
redis.maxmemory:
enabled: true
processors:
batch:
# Add identifying resource attributes, which is required for New Relic entity synthesis.
# The redis receiver does not currently include any identifying attributes on the metrics it produces.
# We manually assign values to server.address and server.port, since values for ${REDIS_SERVICE_HOST} and ${REDIS_SERVICE_PORT} are unstable.
attributes/redis_metrics:
include:
match_type: regexp
metric_names:
# Notice that if with single or without quotes just one backslash is needed 'redis\..*'
- "redis\\..*"
actions:
- action: upsert
key: server.address
value: "redis"
- action: upsert
key: server.port
value: "6379"
exporters:
otlphttp:
endpoint: ${NEW_RELIC_OTLP_ENDPOINT}
headers:
api-key: ${NEW_RELIC_API_KEY}
service:
pipelines:
metrics:
receivers: [redis]
processors: [attributes/redis_metrics, batch]
exporters: [otlphttp]
---
apiVersion: v1
kind: Pod
metadata:
name: collector
namespace: nr-redis
labels:
app.kubernetes.io/name: collector
spec:
containers:
- name: collector
image: otel/opentelemetry-collector-contrib:0.98.0
env:
# The default US endpoint is set here. You can change the endpoint and port based on your requirements if needed.
# docs: https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#configure-endpoint-port-protocol
- name: NEW_RELIC_OTLP_ENDPOINT
value: https://otlp.nr-data.net/
# The New Relic API key used to authenticate export requests.
# Defined in secrets.yaml
- name: NEW_RELIC_API_KEY
valueFrom:
secretKeyRef:
name: nr-redis-secret
key: NEW_RELIC_API_KEY
volumeMounts:
- name: collector-config-vol
mountPath: /etc/otelcol-contrib
volumes:
- name: collector-config-vol
configMap:
name: collector-config
items:
- key: collector-config
path: config.yaml
28 changes: 28 additions & 0 deletions other-examples/collector/redis/k8s/redis.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
apiVersion: v1
kind: Pod
metadata:
name: redis
namespace: nr-redis
labels:
app.kubernetes.io/name: redis
spec:
containers:
- name: redis
image: redis:7.2-alpine
ports:
- containerPort: 6379
---
apiVersion: v1
kind: Service
metadata:
name: redis
namespace: nr-redis
labels:
app.kubernetes.io/name: redis
spec:
ports:
- port: 6379
protocol: TCP
selector:
app.kubernetes.io/name: redis
9 changes: 9 additions & 0 deletions other-examples/collector/redis/k8s/secrets.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: v1
kind: Secret
metadata:
name: nr-redis-secret
namespace: nr-redis
stringData:
# New Relic API key to authenticate the export requests.
# docs: https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key
NEW_RELIC_API_KEY: <INSERT_API_KEY>

0 comments on commit dd19a25

Please sign in to comment.