Skip to content

Commit

Permalink
feat(erigon): only run init snapshot container when enabled
Browse files Browse the repository at this point in the history
  • Loading branch information
chriswessels committed Sep 25, 2022
1 parent 6c7a438 commit eb8f549
Show file tree
Hide file tree
Showing 5 changed files with 55 additions and 56 deletions.
2 changes: 1 addition & 1 deletion charts/erigon/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.3.3
version: 0.3.4

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
Expand Down
12 changes: 6 additions & 6 deletions charts/erigon/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Deploy and scale [Erigon](https://github.com/ledgerwatch/erigon) inside Kubernetes with ease

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) ![Version: 0.3.3](https://img.shields.io/badge/Version-0.3.3-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v2022.09.03](https://img.shields.io/badge/AppVersion-v2022.09.03-informational?style=flat-square)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) ![Version: 0.3.4](https://img.shields.io/badge/Version-0.3.4-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v2022.09.03](https://img.shields.io/badge/AppVersion-v2022.09.03-informational?style=flat-square)

## Features

Expand Down Expand Up @@ -90,19 +90,19 @@ statefulNode:

## Restoring chaindata from a snapshot

You can specify a snapshot URL that will be used to bootstrap Erigon's `chaindata` state. When enabled, an init container will perform a streaming extraction of the snapshot into storage. The snapshot should be a gzipped tarball of `chaindata`.
You can specify a snapshot URL that will be used to restore Erigon's `chaindata` state. When enabled, an init container will perform a streaming extraction of the snapshot into storage. The snapshot should be a gzipped tarball of `chaindata`.

Example:
```yaml
# values.yaml
statefulNode:
fromSnapshot:
restoreSnapshot:
enable: true
snapshotUrl: https://matic-blockchain-snapshots.s3-accelerate.amazonaws.com/matic-mainnet/erigon-archive-snapshot-2022-07-15.tar.gz
```

Once Erigon's state has been bootstrapped, the snapshot URL will be saved to storage at `/from_snapshot`. Any time the Erigon Pod starts, as long as the snapshot configuration has not changed, Erigon will boot with the existing state. If you modify the snapshot configuration, the init container will remove existing chaindata and bootstrap state again.
Once Erigon's state has been restored, the snapshot URL will be saved to storage at `/from_snapshot`. Any time the Erigon Pod starts, as long as the snapshot configuration has not changed, Erigon will boot with the existing state. If you modify the snapshot configuration, the init container will remove existing chaindata and restore state again.

You can monitor progress by following the logs of the `stateful-node-init` container: `kubectl logs --since 1m -f release-name-stateful-node-0 -c stateful-node-init`

Expand Down Expand Up @@ -160,8 +160,6 @@ We do not recommend that you upgrade the application by overriding `image.tag`.
| statefulNode.affinity | | object | `{}` |
| statefulNode.affinityPresets.antiAffinityByHostname | Configure anti-affinity rules to prevent multiple Erigon instances on the same host | bool | `true` |
| statefulNode.extraArgs | Additional CLI arguments to pass to `erigon` | list | `[]` |
| statefulNode.fromSnapshot.enabled | Enable initialising Erigon state from a remote Snapshot | bool | `false` |
| statefulNode.fromSnapshot.snapshotUrl | URL for snapshot to download and extract to bootstrap storage | string | `nil` |
| statefulNode.jwt | JWT for clients to authenticate with the Engine API. Specify either `existingSecret` OR `fromLiteral`. | object | `{"existingSecret":{"key":"","name":""},"fromLiteral":""}` |
| statefulNode.jwt.existingSecret | Load the JWT from an existing Kubernetes Secret. Takes precedence over `fromLiteral` if set. | object | `{"key":"","name":""}` |
| statefulNode.jwt.existingSecret.key | Data key for the JWT in the Secret | string | `""` |
Expand All @@ -176,6 +174,8 @@ We do not recommend that you upgrade the application by overriding `image.tag`.
| statefulNode.podAnnotations | Annotations for the `Pod` | object | `{}` |
| statefulNode.podSecurityContext | Pod-wide security context | object | `{"fsGroup":101337,"runAsGroup":101337,"runAsNonRoot":true,"runAsUser":101337}` |
| statefulNode.resources | | object | `{}` |
| statefulNode.restoreSnapshot.enabled | Enable initialising Erigon state from a remote snapshot | bool | `false` |
| statefulNode.restoreSnapshot.snapshotUrl | URL for snapshot to download and extract to restore state | string | `""` |
| statefulNode.service.ports.grpc-erigon | Service Port to expose Erigon GRPC interface on | int | `9090` |
| statefulNode.service.ports.http-engineapi | Service Port to expose engineAPI interface on | int | `8551` |
| statefulNode.service.ports.http-jsonrpc | Service Port to expose JSON-RPC interface on | int | `8545` |
Expand Down
6 changes: 3 additions & 3 deletions charts/erigon/README.md.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -90,19 +90,19 @@ statefulNode:

## Restoring chaindata from a snapshot

You can specify a snapshot URL that will be used to bootstrap Erigon's `chaindata` state. When enabled, an init container will perform a streaming extraction of the snapshot into storage. The snapshot should be a gzipped tarball of `chaindata`.
You can specify a snapshot URL that will be used to restore Erigon's `chaindata` state. When enabled, an init container will perform a streaming extraction of the snapshot into storage. The snapshot should be a gzipped tarball of `chaindata`.

Example:
```yaml
# values.yaml

statefulNode:
fromSnapshot:
restoreSnapshot:
enable: true
snapshotUrl: https://matic-blockchain-snapshots.s3-accelerate.amazonaws.com/matic-mainnet/erigon-archive-snapshot-2022-07-15.tar.gz
```

Once Erigon's state has been bootstrapped, the snapshot URL will be saved to storage at `/from_snapshot`. Any time the Erigon Pod starts, as long as the snapshot configuration has not changed, Erigon will boot with the existing state. If you modify the snapshot configuration, the init container will remove existing chaindata and bootstrap state again.
Once Erigon's state has been restored, the snapshot URL will be saved to storage at `/from_snapshot`. Any time the Erigon Pod starts, as long as the snapshot configuration has not changed, Erigon will boot with the existing state. If you modify the snapshot configuration, the init container will remove existing chaindata and restore state again.

You can monitor progress by following the logs of the `stateful-node-init` container: `kubectl logs --since 1m -f release-name-stateful-node-0 -c stateful-node-init`

Expand Down
83 changes: 41 additions & 42 deletions charts/erigon/templates/stateful-node/statefulset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -61,39 +61,8 @@ spec:
emptyDir: {}
{{- end }}
initContainers:
{{- if $values.p2pNodePort.enabled }}
- name: {{ $componentName }}-nodeport
image: "{{ $values.p2pNodePort.initContainer.image.repository }}:{{ $values.p2pNodePort.initContainer.image.tag }}"
imagePullPolicy: {{ $values.p2pNodePort.initContainer.image.pullPolicy }}
securityContext:
runAsNonRoot: false
runAsUser: 0
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
command:
- sh
- -c
- >
set -ex;
export EXTERNAL_PORT=$(kubectl get services -l "pod in (${POD_NAME}), type in (p2p)" -o jsonpath='{.items[0].spec.ports[0].nodePort}');
export EXTERNAL_IP=$(kubectl get nodes "${NODE_NAME}" -o jsonpath='{.status.addresses[?(@.type=="ExternalIP")].address}');
export EXTERNAL_IP=${EXTERNAL_IP:-$(curl --silent v4.ident.me)};
echo "EXTERNAL_PORT=$EXTERNAL_PORT" > /env/init-nodeport;
echo "EXTERNAL_IP=$EXTERNAL_IP" >> /env/init-nodeport;
cat /env/init-nodeport;
volumeMounts:
- name: env-nodeport
mountPath: /env
{{- end }}
# TODO: Move this out of chart and into a separate DaemonSet that sets this on every host
- name: {{ $componentName }}-init
{{- if $values.restoreSnapshot.enabled }}
- name: init-snapshot
image: busybox:stable
imagePullPolicy: IfNotPresent
command:
Expand All @@ -103,9 +72,9 @@ spec:
set -x
STORAGE_PATH="/storage"
SNAPSHOT_RESTORE_PATH="$STORAGE_PATH/chaindata"
# If enabled and snapshot URL has been provided, bootstrap snapshot
if [ "${BOOTSTRAP_SNAPSHOT}" = "true" ] && [ "${SNAPSHOT_URL}" != "" ]; then
echo "Snapshot bootstrapping enabled"
# If enabled and snapshot URL has been provided, restore snapshot
if [ "${RESTORE_SNAPSHOT}" = "true" ] && [ "${SNAPSHOT_URL}" != "" ]; then
echo "Snapshot restoration enabled"
if [ ! -f "$STORAGE_PATH/from_snapshot" ] || [ "$(cat $STORAGE_PATH/from_snapshot)" != "${SNAPSHOT_URL}" ]; then
echo "Clearing existing chaindata..."
rm -rf "$SNAPSHOT_RESTORE_PATH"
Expand All @@ -114,27 +83,57 @@ spec:
wget -c "${SNAPSHOT_URL}" -O - | tar -xz -C "$SNAPSHOT_RESTORE_PATH"
echo "${SNAPSHOT_URL}" > ${STORAGE_PATH}/from_snapshot
else
echo "Snapshot configuration already bootstrapped, continuing..."
echo "Snapshot configuration already restored, continuing..."
fi
else
echo "Snapshot bootstrapping not enabled, skipping..."
echo "Snapshot restoration not enabled, skipping..."
fi
volumeMounts:
- name: storage
mountPath: /storage
env:
{{- if $values.fromSnapshot.enabled }}
- name: BOOTSTRAP_SNAPSHOT
- name: RESTORE_SNAPSHOT
value: "true"
- name: SNAPSHOT_URL
value: {{ $values.fromSnapshot.snapshotUrl }}
{{- end }}
value: {{ $values.restoreSnapshot.snapshotUrl }}
{{- with $values.env }}
{{- range $key, $val := .}}
- name: {{ $key }}
value: {{ $val | quote }}
{{- end }}
{{- end }}
{{- end }}
{{- if $values.p2pNodePort.enabled }}
- name: {{ $componentName }}-nodeport
image: "{{ $values.p2pNodePort.initContainer.image.repository }}:{{ $values.p2pNodePort.initContainer.image.tag }}"
imagePullPolicy: {{ $values.p2pNodePort.initContainer.image.pullPolicy }}
securityContext:
runAsNonRoot: false
runAsUser: 0
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
command:
- sh
- -c
- >
set -ex;
export EXTERNAL_PORT=$(kubectl get services -l "pod in (${POD_NAME}), type in (p2p)" -o jsonpath='{.items[0].spec.ports[0].nodePort}');
export EXTERNAL_IP=$(kubectl get nodes "${NODE_NAME}" -o jsonpath='{.status.addresses[?(@.type=="ExternalIP")].address}');
export EXTERNAL_IP=${EXTERNAL_IP:-$(curl --silent v4.ident.me)};
echo "EXTERNAL_PORT=$EXTERNAL_PORT" > /env/init-nodeport;
echo "EXTERNAL_IP=$EXTERNAL_IP" >> /env/init-nodeport;
cat /env/init-nodeport;
volumeMounts:
- name: env-nodeport
mountPath: /env
{{- end }}
containers:
- name: {{ $componentName }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
Expand Down
8 changes: 4 additions & 4 deletions charts/erigon/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -92,11 +92,11 @@ statefulNode:
# -- The amount of disk space to provision for Erigon
storage: 3Ti

fromSnapshot:
# -- Enable initialising Erigon state from a remote Snapshot
restoreSnapshot:
# -- Enable initialising Erigon state from a remote snapshot
enabled: false
# -- URL for snapshot to download and extract to bootstrap storage
snapshotUrl:
# -- URL for snapshot to download and extract to restore state
snapshotUrl: ""

# Increasing the grace termination period prevents Kubernetes
# from killing the node process prematurely. Premature shutdown
Expand Down

0 comments on commit eb8f549

Please sign in to comment.