-
Notifications
You must be signed in to change notification settings - Fork 35
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Pages for commercial storage nodes (#294)
- Loading branch information
Showing
3 changed files
with
222 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
--- | ||
title: Commercial Node | ||
weight: 4 | ||
docId: eisoh4oa2uRaac1n | ||
--- | ||
|
||
Welcome to the Commercial Storage Node Operator Program documentation. This program is designed to facilitate the participation of node operators in data centers and enterprise-grade facilities within the Storj network, ensuring that their nodes meet specific certifications and characteristics. This guide will provide information about the program's purpose, eligibility, advantages, and how it differs from the public network. | ||
|
||
## Why We Started the Program | ||
|
||
The Commercial Storage Node Operator Program was initiated to address the needs of large enterprise customers who require data storage facilities with specific security and compliance certifications such as SOC2 or ISO 27001. These customers are attracted to the Storj network's price, performance, and security but are constrained by their internal security controls from using the public network. To accommodate these requirements, we developed product capabilities that restrict the distribution of data to subsets of nodes operating in facilities that meet these enterprise demands. | ||
|
||
## What is the Commercial Storage Node Operator Program? | ||
|
||
The Commercial Storage Node Operator Program is a way for node operators in data centers and enterprise-grade facilities to participate in the Storj network and certify that their nodes meet specific requirements. It also offers these operators the opportunity to earn revenue at a scale that is economically viable for them while sharing large volumes of underutilized storage and bandwidth capacity. | ||
|
||
## Who is Eligible | ||
|
||
The Commercial Storage Node Operator Program is open to: | ||
|
||
- Operators in enterprise-grade facilities, typically with one or more compliance-oriented certifications. | ||
- Operators with substantial underutilized storage capacity. | ||
- Operators with fully depreciated operational hardware that is in storage or offline. | ||
- Operators with access to low-cost storage hardware. | ||
|
||
## Advantages of the Program | ||
|
||
This program provides several benefits to the Storj network, including: | ||
|
||
- Attracting enterprise customers that would otherwise be inaccessible due to the limitations of the public network. | ||
- Enabling data center and commercial facility operators to participate economically, increasing the network's storage capacity without introducing concentration risk. | ||
- Facilitating smaller operators in winning more data storage opportunities when large professional operators are not competing directly on the same network. | ||
- Supporting the growth of Storj use cases by serving the needs of enterprise customers with diverse storage requirements. | ||
|
||
## How Does it Compare to the Public Network | ||
|
||
The Commercial Storage Node Operator Program differs from the public network in several ways: | ||
|
||
- Commercial node operators face different terms of service and node selection criteria, especially regarding the /24 IP address block restriction. | ||
- Data sets with specific requirements stored on nodes in this program are not subject to the /24 restriction, allowing them to fill their nodes more rapidly. | ||
- Commercial node operators have a distinct cost structure, often accepting lower payout rates due to their operational efficiency. Still, they can earn meaningful revenue thanks to the removal of the /24 restriction on these data sets. | ||
- Both types of nodes (public and commercial) operate on the Storj network but function effectively as separate tiers, providing a choice for customers to store data on one tier or the other. | ||
- Performance, durability, and reliability remain comparable between the two tiers. | ||
- Storj offers premium enterprise pricing for customers storing data on nodes meeting special criteria. | ||
|
||
The end result is a fair opportunity for all participants to achieve an economically rewarding outcome without direct competition between individual and commercial operators for the same workloads. | ||
|
||
## Current State | ||
|
||
We have already recruited an initial set of node operators with SOC2-certified facilities. The program has undergone market testing with this tier of service, and we are now onboarding the first set of enterprise customers. We are also extending invitations to our existing node operators to join the program. | ||
|
||
## How to Participate | ||
|
||
If your facility is suitable for the Commercial Storage Node Operator Program, please [request to join the program](https://www.storj.io/partners/commercial-nodes), and someone from our partnership team will contact you shortly. | ||
|
||
For further assistance or inquiries, please contact our support team. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,112 @@ | ||
--- | ||
title: Sample Ansible Configuration | ||
docId: aiwee4RieY4cooMa | ||
--- | ||
|
||
{% callout type="danger" %} | ||
|
||
This guide is for Commerical Storage Nodes, if you're looking to join the public network follow [](docId:kjMiGo7HTr4v_qwD5Iqc7) instead. | ||
|
||
{% /callout %} | ||
|
||
## Prerequisites | ||
|
||
- Completed setup of one [Storage Node](docId:ohngie2AhcuiX1ce) | ||
- Installed [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/index.html) | ||
- Installed [Docker](docId:EW9B_0fJujL3Z5aTLUW7d) | ||
|
||
## Ansible Configuration | ||
|
||
There are a number of different tools to do cloud operations, this guide should be considered only an example and not a requirement for how to run [Commerical Storage nodes](docId:eisoh4oa2uRaac1n). | ||
|
||
{% callout type="note" %} | ||
|
||
Any environment that can reliability run the [storage node image](https://hub.docker.com/r/storjlabs/storagenode) can be also be used. | ||
|
||
{% /callout %} | ||
|
||
This Ansible playbook is showing how a single host machine with multiple hard drives can run multiple Storage nodes. Each container's environment variables, storage volumes, and network ports are set according to the individual dictionary elements in `storj_containers`. | ||
|
||
Each Storage node requires its own `external_port`, for communication with the Storj network. Similarly, each node ideally has a dedicated drive or mount point for its storage, identified by `volume_mount` like /mnt/sdb or /mnt/sdc. | ||
|
||
The Ansible `loop` iterates over the list of dictionaries stored in the `storj_containers` variable, executing the container deployment tasks for each dictionary item. This allows multiple Storage nodes with varying configurations to be deployed in a single playbook run. | ||
|
||
{% code-group label="storagenode_playbook.yml" %} | ||
```ini | ||
- name: Run Storj Storage Nodes | ||
hosts: storagenode | ||
become: yes | ||
vars: | ||
base_email: 'name+{{ item.name }}@example.com' | ||
public_ip: xxx.xx.xx.xx | ||
storj_containers: | ||
- name: storagenode1 | ||
external_port: 20001 | ||
internal_dashboard_port: 14001 | ||
volume_mount: '/mnt/sdb' | ||
storage: '10TB' | ||
- name: storagenode2 | ||
external_port: 20002 | ||
internal_dashboard_port: 14002 | ||
volume_mount: '/mnt/sdc' | ||
storage: '10TB' | ||
# Add more here | ||
tasks: | ||
- name: Pull Storj image | ||
community.docker.docker_image: | ||
name: storjlabs/storagenode:latest | ||
source: pull | ||
|
||
- name: Run Storj containers | ||
community.docker.docker_container: | ||
name: '{{ item.name }}' | ||
image: storjlabs/storagenode:latest | ||
env: | ||
WALLET: '0x0000000000000000000000000000000000000000' | ||
EMAIL: '{{ base_email }}' | ||
ADDRESS: '{{ public_ip }}:{{ item.external_port }}' | ||
STORAGE: '{{ item.storage }}' | ||
STORJ_CONSOLE_ADDRESS: '127.0.0.1:{{ item.internal_dashboard_port }}' | ||
volumes: | ||
- '{{ item.volume_mount }}/identity:/app/identity' | ||
- '{{ item.volume_mount }}/storage:/app/config' | ||
ports: | ||
- '{{ item.external_port }}:{{ item.external_port }}/tcp' | ||
- '{{ item.external_port }}:{{ item.external_port }}/udp' | ||
- '127.0.0.1:{{ item.internal_dashboard_port }}:{{ item.internal_dashboard_port }}' | ||
state: started | ||
restart_policy: unless-stopped | ||
loop: '{{ storj_containers }}' | ||
``` | ||
{% /code-group %} | ||
|
||
|
||
Next define the hosts where Ansible tasks will be executed with an inventory file `hosts.ini`, where you list the IP addresses or hostnames of your target machines, optionally grouped under bracketed headers to organize them. Here's a basic example: | ||
|
||
{% code-group label="hosts.ini" %} | ||
```ini | ||
[storagenode] | ||
192.168.1.5 | ||
192.168.1.6 | ||
``` | ||
{% /code-group %} | ||
|
||
With this inventory, you can target specific groups (e.g., `[storagenode]`) or individual hosts in your Ansible playbooks. | ||
|
||
To execute the playbook, specify your inventory file using the `-i` option. This will run the playbook tasks on the hosts listed in your inventory. | ||
|
||
``` | ||
ansible-playbook -i hosts.ini storagenode_playbook.yml | ||
``` | ||
|
||
On your host machine, you can view the status of your docker containers. Here are some useful commands | ||
|
||
The `docker ps` command lists all running Docker containers, it's useful for quickly checking which containers are active and their configurations. | ||
|
||
The `docker logs` command displays the standard output and standard error logs of a specified Docker container. To debug `storagenode1` container for example you could do | ||
|
||
``` | ||
docker logs --tail 20 storagenode1 | ||
``` | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
--- | ||
title: Setup | ||
docId: ohngie2AhcuiX1ce | ||
--- | ||
|
||
This guide is tailored for Cloud Operations Engineers who manage multiple racked servers, data centers, and enterprise-grade facilities and wish to run [Commercial Storage Nodes](docId:eisoh4oa2uRaac1n). | ||
|
||
Please note that joining the Commercial Storage Node Operator Program is a prerequisite for this setup. | ||
|
||
{% callout type="danger" %} | ||
|
||
This guide is for Commerical Storage Nodes, if you're looking to join the public network follow [](docId:kjMiGo7HTr4v_qwD5Iqc7) instead. | ||
|
||
{% /callout %} | ||
|
||
## Prerequisites | ||
|
||
- Admitted into the Commercial Storage Node Operator Program ([request to join the program](https://www.storj.io/partners/commercial-nodes)) | ||
- Understand the [difference between Public and Commercial Storage Nodes](docId:eisoh4oa2uRaac1n#how-does-it-compare-to-the-public-network) | ||
|
||
## Setup | ||
|
||
A commercial Storage Node runs the same software as a public Storage Node, but is not subject to the same restrictions. | ||
|
||
Run one Storage Node per hard drive. | ||
|
||
{% callout type="note" %} | ||
If you have a virtualized environment, try to target the nodes size to be similar to the average drive size underlying that virtualized environment. So if the infrastructure is mostly on 10 TB HDD then set the node size to be 10 TB. | ||
{% /callout %} | ||
|
||
Follow Steps 1 through 6 in the [Storage Node setup guide](docId:kjMiGo7HTr4v_qwD5Iqc7) for each Commercial Storage Node: | ||
|
||
[](docId:hbCGTv1ZLLR2-kpSaGEXw) | ||
|
||
[](docId:v-fUvPqySvUwTMF-od6hD) | ||
|
||
[](docId:y0jltT-HzKPmDefi532sd) | ||
|
||
[](docId:owZeAc56KSDnUzDhsBfB8) | ||
|
||
[](docId:aT6VAB297OWLd4vqeXxf5) | ||
|
||
[](docId:XC--4Jtp1o309gbWFOHPn) | ||
|
||
For step 6, Commercial Storage Nodes require additional environment variables | ||
|
||
```yaml | ||
STORJ_STORAGE2_TRUST_SOURCES="https://www.storj.io/trusted-satellites-soc2" | ||
WALLET="0x0000000000000000000000000000000000000000" | ||
``` | ||
|
||
## Node acceptance | ||
|
||
Once you have 1 or more Storage Nodes running, reply back to your contact with a csv file or document of all your Storage node IDs. The vetting process will begin and if the nodes are acceptable, they will be enabled to receive data. |