Skip to content

Commit

Permalink
Commercial node updates (#298)
Browse files Browse the repository at this point in the history
  • Loading branch information
amozoss authored Oct 27, 2023
1 parent 9574b30 commit c0f3d1f
Show file tree
Hide file tree
Showing 5 changed files with 1,579 additions and 4 deletions.
123 changes: 123 additions & 0 deletions app/node/commercial-node/dashboard/page.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---
title: Node Dashboard
docId: eCupaiZizohpah7I
metadata:
title: Setting Up a Dashboard for Commercial Storage Nodes
description:
A guide to setting up a node dashboard to monitor commercial storage
nodes using Prometheus and Grafana, including prerequisites, metrics endpoint
setup, Prometheus and Grafana configuration.
---

This guide assists commercial Storage Node operators in setting up advanced monitoring using Prometheus and Grafana. There are alternative monitoring tools that also will work. Consider the following as guidelines rather than a strict requirement for how it do it.

{% callout type="danger" %}

This guide is for Commerical Storage Nodes, if you're looking to join the public network follow [](docId:kjMiGo7HTr4v_qwD5Iqc7) instead.

{% /callout %}

## Prerequisites

- Familiarity with or installed [Prometheus](https://prometheus.io/docs/prometheus/latest/installation/)
- Familiarity with or installed [Grafana](https://grafana.com/docs/grafana/latest/setup-grafana/installation/)

## Metrics

Storage nodes have a Prometheus metrics endpoint that can be scraped periodically Prometheus for better insight to the health of your network.

To enable the endpoint set the `STORJ_DEBUG_ADDR` or `--debug.addr` flag for each Storage node (incrementing the port by 1 if multiple nodes are on the same machine).

```
STORJ_DEBUG_ADDR=127.0.0.1:6000
```

Visit 127.0.0.1:6000 for an overview page of monitoring capabilities. More specifically the `/metrics` endpoint can be view for more specific overview of what is available

### Prometheus configuration

Next create a `prometheus.yaml` configured to scrape the Storage nodes in your network.

List each Storage Node's debug address and port as targets.

{% callout type="note" %}

`/metrics` is added by prometheus targets by default

{% /callout %}

{% code-group label="prometheus.yaml" %}

```yaml
global:
scrape_interval: 15s

scrape_configs:
- job_name: 'local-metrics'
static_configs:
- targets:
- host.docker.internal:6000 # if prometheus and node are on the same docker host
- host.docker.internal:6001
- 192.168.1.5:6000
- 192.168.1.5:6001
- 192.168.1.6:6000
- 192.168.1.6:6001
```
{% /code-group %}
Run the [Prometheus](https://prometheus.io/docs/prometheus/latest/installation/) docker container or update your existing Prometheus server config
```
docker run -p 9090:9090 -v ./prometheus.yaml:/etc/prometheus/prometheus.yml prom/prometheus
```

Visit the Prometheus targets (<http://localhost:9090/targets>) to see if the Storage Nodes are being successfully scraped.

## Grafana Dashboard

Once metrics are being successfully gathered, they can be visualized and analyzed with Grafana.

Run the [Grafana](https://grafana.com/docs/grafana/latest/setup-grafana/installation/) docker container or add the following to your existing one.

```
docker run -d --name=grafana -p 3000:3000 grafana/grafana-enterprise
```

Login to Grafana by visiting <http://localhost:3000> (username and password is `admin` by default).

### Connect Prometheus to your Grafana instance

In order to view the metrics data in Grafana, you'll need to create a new Prometheus data source.

1. Click **Add new connection** from the left sidebar under connections.
![](https://link.storjshare.io/raw/jua7rls6hkx5556qfcmhrqed2tfa/docs/images/grafana-add-new-data-source.png)

1. Select **Prometheus** as the data source

1. Enter your Prometheus server URL (e.g. http://localhost:9090) under connections

1. Click **Save & test**

### Add Storage Node Dashboard

1. Navigate to the Dashboards <http://localhost:3000/dashboards>

1. Click **Create Dashboard** in the middle of the screen or the **New** if you already have an existing one

1. Select the **Import a dashboard** option

1. Paste the JSON from [storage-node-grafana.json](/storage-node-grafana.json) into **Import via dashboard JSON model** text area.

1. Click **Load** and **Import** to create the Dashboard.

You've successfully setup your Storage node dashboard.

{% callout type="note" %}

It may say "No data" for each graph until an upload or download to your Storage Nodes occurs.

{% /callout %}

![](https://link.storjshare.io/raw/jua7rls6hkx5556qfcmhrqed2tfa/docs/images/grafana-storage-node-dashboard.png)

7 changes: 6 additions & 1 deletion app/node/commercial-node/page.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
title: Commercial Node
weight: 4
docId: eisoh4oa2uRaac1n
metadata:
title: Understanding the Commercial Storage Node Operator Program
description: Guide to Storj's Commercial Storage Node Operator Program, offering
detailed insights into the aim, eligibility, advantages, and comparison with the
public network.
---

Welcome to the Commercial Storage Node Operator Program documentation. This program is designed to facilitate the participation of node operators in data centers and enterprise-grade facilities within the Storj network, ensuring that their nodes meet specific certifications and characteristics. This guide will provide information about the program's purpose, eligibility, advantages, and how it differs from the public network.
Expand Down Expand Up @@ -53,4 +58,4 @@ We have already recruited an initial set of node operators with SOC2-certified f

If your facility is suitable for the Commercial Storage Node Operator Program, please [request to join the program](https://www.storj.io/partners/commercial-nodes), and someone from our partnership team will contact you shortly.

For further assistance or inquiries, please contact our support team.
For further assistance or inquiries, please contact our support team.
9 changes: 6 additions & 3 deletions app/node/commercial-node/setup/ansible/page.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
---
title: Sample Ansible Configuration
docId: aiwee4RieY4cooMa
metadata:
title: Creating Storage Nodes Using Ansible
description: This document provides an example for creating multiple Storj Storage
nodes using Ansible. A playbook is provided that details how to run a single host
machine with multiple hard drives to run multiple Storage nodes.
---

{% callout type="danger" %}
Expand Down Expand Up @@ -107,6 +112,4 @@ The `docker logs` command displays the standard output and standard error logs o

```
docker logs --tail 20 storagenode1
```


```
14 changes: 14 additions & 0 deletions app/node/commercial-node/setup/page.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
---
title: Setup
docId: ohngie2AhcuiX1ce
weight: 5
metadata:
title: Setting Up Commercial Storage Nodes
description:
Guide for Cloud Operations Engineers on how to set up commercial storage
nodes with Storj including prerequisites and node acceptance.
---

This guide is tailored for Cloud Operations Engineers who manage multiple racked servers, data centers, and enterprise-grade facilities and wish to run [Commercial Storage Nodes](docId:eisoh4oa2uRaac1n).
Expand Down Expand Up @@ -49,6 +55,14 @@ STORJ_STORAGE2_TRUST_SOURCES="https://www.storj.io/trusted-satellites-soc2"
WALLET="0x0000000000000000000000000000000000000000"
```

## Multi Storage node per host

Since it's recommended to run one Storage Node per drive a [sample ansible configuration](docId:aiwee4RieY4cooMa) is provided for some ideas on how to manage the nodes. Metrics for the Storage nodes can also be setup by following [the node dashboard guide](docId:eCupaiZizohpah7I).

## Auto updating

To ensure optimal functionality and security, it's mandatory to update your nodes to at least the minimum required version (usually updates every 2 to 3 weeks). We recommend utilizing the built-in auto-updater in the Storage Node docker image for hassle-free updates.

## Node acceptance

Once you have 1 or more Storage Nodes running, reply back to your contact with a csv file or document of all your Storage node IDs. The vetting process will begin and if the nodes are acceptable, they will be enabled to receive data.
Loading

0 comments on commit c0f3d1f

Please sign in to comment.