Skip to content

Commit

Permalink
Merge pull request #745 from oasisprotocol/amela/update-system-conf
Browse files Browse the repository at this point in the history 
docs/node/: Reorganize system configuration
  • Loading branch information
@amela
amela authored Feb 29, 2024
2 parents 5a2eec7 + 791ed28 commit 6400214
Show file tree
Hide file tree
Showing 2 changed files with 105 additions and 98 deletions.
201 changes: 104 additions & 97 deletions docs/node/run-your-node/prerequisites/system-configuration.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,155 +3,164 @@ import TabItem from '@theme/TabItem';

# System Configuration

This page describes changes that should be made to the configuration of the
system where you are running an Oasis Node instance.

## File Descriptor Limit

Make sure that the user under which you are running your Oasis Node has a high-enough file descriptor limit as the databases can have many files open and running out of file descriptors can lead to the node stopping unexpectedly.

You can check the file descriptor limit by running the following as the same user that will run Oasis Node:

```bash
ulimit -n
```

If this number is lower than 102400 you should consider increasing it by updating your system configuration. You can configure _temporary_ limits by running:

```bash
ulimit -n 102400
This page outlines the modifications necessary that should be made to the
configuration of the system where you are running an Oasis Node instance. To
prepare the system for running an Oasis Node, we will guide you through the
process of creating a dedicated user account and increasing the file descriptor
limit.

## Create a user
Nothing in Oasis Node requires elevated privileges, so running the Oasis Node
with root privileges is not allowed. Attempting to run the `oasis-node` process
as the root user will terminate immediately on startup. You will need to
create a dedicated user account to manage the Oasis Node processes.

To create a new user run as root:

```shell
adduser oasis
```

Note that this limit only applies to any processes started from the same shell after the command was executed. If you want to make the change permanent, you have the following options.
If you have an existing data directory, change its ownership to the `oasis`
user. If not, you may skip this step.

### System-wide Resource Limits Configuration File

As `root`, create a file in `/etc/security/limits.d/99-oasis-node.conf` with content similar to the following example:

```bash
* soft nofile 102400
* hard nofile 1048576
```shell
chown -R oasis /node/data
```

You can replace `*` with the name of the user that is running the Oasis Node in case you only want to change the limits for that particular user.
:::tip

:::caution
Adjust the user as appropriate to your setup. For example, setting the `oasis`
user's Shell to `/usr/sbin/nologin` prevents (accidentally) logging in as this
user. See the following examples on how to create a user on different systems.

In order for the changes to take effect a system restart may be required.
<Tabs>
<TabItem value="Ubuntu">

:::
As root, run:

### Systemd Service Configuration
```shell
adduser --system oasis --shell /usr/sbin/nologin
```

In case you are running your Oasis Node process via [systemd](https://systemd.io/), you can add the following directive under the `[Service]` section:
</TabItem>
<TabItem value="Fedora">

```text
LimitNOFILE=102400
As root, run:
```shell
useradd -r -s /usr/sbin/nologin
```

### Docker

If you are running Oasis Node via [Docker](https://www.docker.com/) you can pass the following option to `docker run` in order to increase the limit to desired values:
</TabItem>

```text
--ulimit nofile=102400:1048576
```
<TabItem value="Ansible">

## Running Oasis Services with Non-root System User {#non-root}
Add the following task to your playbook:

:::caution
```yml
- name: Create oasis user
user:
name: oasis
comment: Oasis Services user
system: yes
shell: /usr/sbin/nologin
```
Beginning with **Oasis Core 22.1.x release series** it is
**no longer allowed to run Oasis Node** (i.e. the `oasis-node` binary)
**as root** (effective user ID of 0).
</TabItem>
</Tabs>
:::
Running network accessible services as the root user is extremely bad for
system security as a general rule. While it would be "ok" if we could drop
privileges, `syscall.AllThreadsSyscall` does not work if the binary uses `cgo`
at all.
## Increase file descriptor limit
Nothing in Oasis Node will ever require elevated privileges.
Attempting to run the `oasis-node` process as the root user will now terminate
immediately on startup.
Make sure that the user under which you are running your Oasis Node has a
high-enough file descriptor limit as the databases can have many opened files.
Running out of file descriptors can lead to the node stopping unexpectedly.
While there may be specific circumstances where it is safe to run network
services with the effective user ID set to 0, the overwhelming majority of cases
where this is done is a misconfiguration.
You can check the file descriptor limit by running the following as the same
user that will run the Oasis Node:
### Changing Your Setup to Run Oasis Services with Non-root System User {#change-to-non-root}
```shell
ulimit -n
```

:::info
In these examples, we change the setup to run Oasis Services (e.g. Oasis Node)
with a non-root system user named `oasis`.
These instructions also assume that the node's datadir is `/node/data`.
If this number is lower than 102400 you should consider increasing it by
updating your system configuration. You can configure _temporary_ limits by
running:

Adjust these as appropriate to your setup.
:::
```shell
ulimit -n 102400
```

1. Create the `oasis` system user:
This limit will only apply to any processes started from the same shell after
the command was executed. If you want to make the change _permanent_, you have
the following options:

<Tabs>
<TabItem value="Ubuntu">
<TabItem value="systemd">

As root, run:
In case you are running your Oasis Node process via
[systemd](https://systemd.io/), you can add the following directive under the
`[Service]` section:

```bash
adduser --system oasis --shell /usr/sbin/nologin
```
LimitNOFILE=102400
```

</TabItem>
<TabItem value="Fedora">
<TabItem value="Docker">

As root, run:
```bash
useradd -r -s /usr/sbin/nologin
If you are running the Oasis Node via [Docker](https://www.docker.com/) you can
pass the following option to `docker run` in order to increase the limit to
desired values:

```
--ulimit nofile=102400:1048576
```

</TabItem>

<TabItem value="Ansible">
<TabItem value="limits.conf">

Add the following task to your playbook:
As `root`, create a system-wide *resource limits* configuration File in
`/etc/security/limits.d/99-oasis-node.conf` with content similar to the
following example:

```yml
- name: Create oasis user
user:
name: oasis
comment: Oasis Services user
system: yes
shell: /usr/sbin/nologin
```
* soft nofile 102400
* hard nofile 1048576
```

</TabItem>
</Tabs>
You can replace `*` with the name of the user that is running the Oasis Node in
case you only want to change the limits for that particular user.

:::tip
:::caution

Setting `oasis` user's Shell to `/usr/sbin/nologin` prevents (accidentally)
logging-in as this user.
The change above will increase the limit for all processes and not just the
Oasis Node.

:::

2. Stop your Oasis Node.
:::info

3. Transfer ownership of the datadir to the `oasis` user:
In order for the changes to take effect a system restart may be required.

```sh
chown -R oasis /node/data
```
:::

See [Invalid Permissions] troubleshooting guide for more information.
</TabItem>
</Tabs>

## Example snippets for different setups

4. Update how you run Oasis Node:
You may find the following snippets helpful in case you are running `oasis-node`
process with systemd, Docker or runit.

<Tabs>
<TabItem value="systemd">

Add a [`User` directive] to the Oasis service's systemd unit file:

```conf
```
...
User=oasis
...
Expand All @@ -160,7 +169,7 @@ User=oasis
Below can be found a simple systemd unit file for `oasis-node` that can be used
as a starting point.

```conf
```ini
[Unit]
Description=Oasis Node
After=network.target
Expand All @@ -182,7 +191,7 @@ WantedBy=multi-user.target

Add [`USER` instruction] to your Oasis service's Dockerfile:

```conf
```
...
USER oasis
...
Expand All @@ -192,14 +201,12 @@ USER oasis
<TabItem value="runit">

Wrap the invocation in a [`chpst` command]:
```sh
```shell
chpst -u oasis oasis-node ...
```
</TabItem>
</Tabs>

5. Start your Oasis Node.

[`User` directive]:
https://www.freedesktop.org/software/systemd/man/systemd.exec.html#User=
[`User` instruction]:
Expand Down
2 changes: 1 addition & 1 deletion docusaurus.config.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ Apache 2.0</a>. Built with &#x2665; and Docusaurus.</p>`,
prism: {
theme: prismThemes.github,
darkTheme: prismThemes.dracula,
additionalLanguages: ['diff', 'rust', 'solidity', 'toml'],
additionalLanguages: ['diff', 'rust', 'solidity', 'toml', 'ini'],
},
colorMode: {
respectPrefersColorScheme: true,
Expand Down

0 comments on commit 6400214

Please sign in to comment.