Skip to content

Commit

Permalink
Add statsd colletor example
Browse files Browse the repository at this point in the history
  • Loading branch information
jack-berg committed Jun 17, 2024
1 parent fe874c8 commit ac3ab7f
Show file tree
Hide file tree
Showing 5 changed files with 178 additions and 1 deletion.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,10 @@ OpenTelemetry is a big ecosystem and everything doesn't fit into the goals of th
* [Telemetry data processing](./other-examples/collector/nr-config)
* [Host monitoring](./other-examples/collector/host-monitoring)
* [Confluent cloud monitoring](./other-examples/collector/confluentcloud)
* [Singlestore monitoring](./other-examples/collector/singlestore)
* [HCP Consul monitoring](./other-examples/collector/hcp-consul)
* [Redis monitoring](./other-examples/collector/redis)
* [Singlestore monitoring](./other-examples/collector/singlestore)
* [StatsD monitoring](./other-examples/collector/statsd)
* Java
* [OpenTelemetry Agent New Relic Config](./other-examples/java/agent-nr-config)
* [Micrometer Shim with OTLP Export](./other-examples/java/micrometer-shim)
Expand Down
60 changes: 60 additions & 0 deletions other-examples/collector/statsd/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Monitoring StatsD with OpenTelemetry Collector

This simple example demonstrates monitoring statsd sources with the [OpenTelemetry collector](https://opentelemetry.io/docs/collector/), using the [statsd receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/statsdreceiver) and sending the data to New Relic via OTLP. A simple load statsd load generator is configured to send dummy data to the statsd receiver.

## Requirements

* You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. Docker desktop [includes a standalone Kubernetes server and client](https://docs.docker.com/desktop/kubernetes/) which is useful for local testing.
* [A New Relic account](https://one.newrelic.com/)
* [A New Relic license key](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key)

## Running the example

1. Update the `NEW_RELIC_API_KEY` value in [secrets.yaml](./k8s/secrets.yaml) to your New Relic license key.

```yaml
# ...omitted for brevity
stringData:
# New Relic API key to authenticate the export requests.
# docs: https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key
NEW_RELIC_API_KEY: <INSERT_API_KEY>
```
* Note, be careful to avoid inadvertent secret sharing when modifying `secrets.yaml`. To ignore changes to this file from git, run `git update-index --skip-worktree k8s/secrets.yaml`.

* If your account is based in the EU, update the `NEW_RELIC_OTLP_ENDPOINT` value in [collector.yaml](./k8s/collector.yaml) the endpoint to: [https://otlp.eu01.nr-data.net](https://otlp.eu01.nr-data.net)

```yaml
# ...omitted for brevity
env:
# The default US endpoint is set here. You can change the endpoint and port based on your requirements if needed.
# docs: https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#configure-endpoint-port-protocol
- name: NEW_RELIC_OTLP_ENDPOINT
value: https://otlp.eu01.nr-data.net
```

2. Run the application with the following command.

```shell
kubectl apply -f k8s/
```

* When finished, cleanup resources with the following command. This is also useful to reset if modifying configuration.

```shell
kubectl delete -f k8s/
```

## Viewing your data

To review your statsd data in New Relic, navigate to "New Relic -> Query Your Data". To list the metrics reported, query for:

```
FROM Metric SELECT uniques(metricName) WHERE otel.library.name = 'otelcol/statsdreceiver' LIMIT MAX
```

See [get started with querying](https://docs.newrelic.com/docs/query-your-data/explore-query-data/get-started/introduction-querying-new-relic-data/) for additional details on querying data in New Relic.

## Additional notes

A simple load statsd load generator is configured to send dummy data to the statsd receiver, specified in [gen-statsd.yaml](./k8s/gen-statsd.yaml). To use in production, you'll need to configure your statsd data sources to point to the collector's statsd receiver endpoint. In this example, the collector's statsd receiver is listing for UDP on `0.0.0.0:8125`, which is exposed in a [service](https://kubernetes.io/docs/concepts/services-networking/service/). The load generator is configured point to this using service env vars set by kubernetes.
87 changes: 87 additions & 0 deletions other-examples/collector/statsd/k8s/collector.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: nr-statsd
---
apiVersion: v1
kind: ConfigMap
metadata:
name: collector-config
namespace: nr-statsd
labels:
app.kubernetes.io/name: collector-config
data:
collector-config: |
receivers:
statsd:
endpoint: 0.0.0.0:8125
is_monotonic_counter: true
processors:
batch:
exporters:
otlphttp:
endpoint: ${NEW_RELIC_OTLP_ENDPOINT}
headers:
api-key: ${NEW_RELIC_API_KEY}
service:
pipelines:
metrics:
receivers: [statsd]
processors: [batch]
exporters: [otlphttp]
---
apiVersion: v1
kind: Pod
metadata:
name: collector
namespace: nr-statsd
labels:
app.kubernetes.io/name: collector
spec:
containers:
- name: collector
image: otel/opentelemetry-collector-contrib:0.98.0
env:
# The default US endpoint is set here. You can change the endpoint and port based on your requirements if needed.
# docs: https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#configure-endpoint-port-protocol
- name: NEW_RELIC_OTLP_ENDPOINT
value: https://otlp.nr-data.net/
# The New Relic API key used to authenticate export requests.
# Defined in secrets.yaml
- name: NEW_RELIC_API_KEY
valueFrom:
secretKeyRef:
name: nr-statsd-secret
key: NEW_RELIC_API_KEY
volumeMounts:
- name: collector-config-vol
mountPath: /etc/otelcol-contrib
ports:
- containerPort: 8125
protocol: UDP
volumes:
- name: collector-config-vol
configMap:
name: collector-config
items:
- key: collector-config
path: config.yaml
---
apiVersion: v1
kind: Service
metadata:
name: collector
namespace: nr-statsd
labels:
app.kubernetes.io/name: collector
spec:
ports:
- name: statsd
port: 8125
protocol: UDP
selector:
app.kubernetes.io/name: collector
19 changes: 19 additions & 0 deletions other-examples/collector/statsd/k8s/gen-statsd.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
apiVersion: v1
kind: Pod
metadata:
name: gen-statsd
namespace: nr-statsd
labels:
app.kubernetes.io/name: gen-statsd
spec:
containers:
- name: gen-statsd
image: circonus/gen-statsd:1.0.0
env:
- name: STATSD_HOST
value: $(COLLECTOR_SERVICE_HOST):$(COLLECTOR_SERVICE_PORT_STATSD)
- name: PROTOCOL
value: udp
- name: AGENTS
value: "1"
10 changes: 10 additions & 0 deletions other-examples/collector/statsd/k8s/secrets.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
apiVersion: v1
kind: Secret
metadata:
name: nr-statsd-secret
namespace: nr-statsd
stringData:
# New Relic API key to authenticate the export requests.
# docs: https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key
NEW_RELIC_API_KEY: <INSERT_API_KEY>

0 comments on commit ac3ab7f

Please sign in to comment.