Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Enhance helper Pod interface and configuration.
Browse files Browse the repository at this point in the history
* Make the helper Pod receive only environment variables instead of args (this changes the interface in a non-backward compatible way but is simpler to use and potentially provides more backward compatibility in the future).
* Adds the manager options `--pvc-annotation[-required]` to pass through annotations from the PVC to the PV and to the helper Pod.
* Merge the helper Pod's `data` VolumeMount with the one provided with the template to be able to specify `mountPropagation` within the template.
* Rename `helperPod.yaml` to `helper-pod.yaml` (more convenient and if we break sth we can break this as well).
* Expose `--helper-pod-timeout` option.
* Provide a basic usage example of the new features (`examples/cache`).
* Support forceful termination of the manager binary (2xCtrl+c - since this is annoying during development otherwise).

Closes #164
Closes #165

Signed-off-by: Max Goltzsche <[email protected]>
mgoltzsche committed Dec 30, 2020

Verified

This commit was signed with the committer’s verified signature. The key has expired.
mgoltzsche Max Goltzsche
1 parent d253f2b commit 9818353
Showing 28 changed files with 675 additions and 478 deletions.
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
./.dapper
./.cache
./dist
./examples/cache/testmount
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -5,3 +5,5 @@
*.swp
.idea
.vscode/
local-path-provisioner
/examples/cache/testmount
62 changes: 21 additions & 41 deletions README.md
Original file line number Diff line number Diff line change
@@ -105,7 +105,7 @@ Now you've verified that the provisioner works as expected.

### Customize the ConfigMap

The configuration of the provisioner is a json file `config.json` and two bash scripts `setup` and `teardown`, stored in the a config map, e.g.:
The configuration of the provisioner is a json file `config.json`, a Pod template `helper-pod.yaml` and two bash scripts `setup` and `teardown`, e.g.:
```
kind: ConfigMap
apiVersion: v1
@@ -132,41 +132,11 @@ data:
}
setup: |-
#!/bin/sh
while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done
mkdir -m 0777 -p ${absolutePath}
mkdir -m 0777 -p "$VOL_DIR"
teardown: |-
#!/bin/sh
while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done
rm -rf ${absolutePath}
helperPod.yaml: |-
rm -rf "$VOL_DIR"
helper-pod.yaml: |-
apiVersion: v1
kind: Pod
metadata:
@@ -195,16 +165,26 @@ The configuration must obey following rules:
3. No duplicate paths allowed for one node.
4. No duplicate node allowed.

#### Scripts `setup` and `teardown` and `helperPod.yaml`
#### Scripts `setup` and `teardown` and the `helper-pod.yaml` template

The script `setup` will be executed before the volume is created, to prepare the directory on the node for the volume.
* The `setup` script is run before the volume is created, to prepare the volume directory on the node.
* The `teardown` script is run after the volume is deleted, to cleanup the volume directory on the node.
* The `helper-pod.yaml` template is used to create a helper Pod that runs the `setup` or `teardown` script.

The script `teardown` will be executed after the volume is deleted, to cleanup the directory on the node for the volume.
The scripts receive their input as environment variables:

The yaml file `helperPod.yaml` will be created by local-path-storage to execute `setup` or `teardown` script with three paramemters `-p <path> -s <size> -m <mode>` :
* path: the absolute path provisioned on the node
- size: pvc.Spec.resources.requests.storage in bytes
* mode: pvc.Spec.VolumeMode
| Environment variable | Description |
| -------------------- | ----------- |
| `VOL_DIR` | Volume directory that should be created or removed. |
| `VOL_NAME` | Name of the PersistentVolume. |
| `VOL_TYPE` | Type of the PersistentVolume (`Block` or `Filesystem`). |
| `VOL_SIZE_BYTES` | Requested volume size in bytes. |
| `PVC_NAME` | Name of the PersistentVolumeClaim. |
| `PVC_NAMESPACE` | Namespace of the PersistentVolumeClaim. |
| `PVC_ANNOTATION` | Value of the PersistentVolumeClaim annotation specified by the manager's `--pvc-annotation` option. |
| `PVC_ANNOTATION_{SUFFIX}` | Value of the PersistentVolumeClaim annotation with the prefix specified by the manager's `--pvc-annotation` option. The `SUFFIX` is the normalized path within the annotation name after the `/`. E.g. if `local-path-provisioner` is run with `--pvc-annotation=storage.example.org` the PVC annotation `storage.example.org/cache-name` is passed through to the Pod as env var `PVC_ANNOTATION_CACHE_NAME`. If the helper Pod requires such an annotation `local-path-provisioner` can be run with e.g. `--pvc-annotation-required=storage.example.org/cache-name`. |

Additional environment variables and defaults for the optional `PVC_ANNOTATION*` can be specified within the helper Pod template.

#### Reloading

36 changes: 3 additions & 33 deletions debug/config.yaml
Original file line number Diff line number Diff line change
@@ -39,41 +39,11 @@ data:
}
setup: |-
#!/bin/sh
while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done
mkdir -m 0777 -p ${absolutePath}
mkdir -m 0777 -p "$VOL_DIR"
teardown: |-
#!/bin/sh
while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done
rm -rf ${absolutePath}
helperPod.yaml: |-
rm -rf "$VOL_DIR"
helper-pod.yaml: |-
apiVersion: v1
kind: Pod
metadata:
2 changes: 1 addition & 1 deletion deploy/chart/templates/configmap.yaml
Original file line number Diff line number Diff line change
@@ -13,6 +13,6 @@ data:
{{ .Values.configmap.setup | nindent 4 }}
teardown: |-
{{ .Values.configmap.teardown | nindent 4 }}
helperPod.yaml: |-
helper-pod.yaml: |-
{{ .Values.configmap.helperPod | nindent 4 }}
35 changes: 2 additions & 33 deletions deploy/chart/values.yaml
Original file line number Diff line number Diff line change
@@ -93,41 +93,10 @@ configmap:
# specify the custom script for setup and teardown
setup: |-
#!/bin/sh
while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done
mkdir -m 0777 -p ${absolutePath}
mkdir -m 0777 -p "$VOL_DIR"
teardown: |-
#!/bin/sh
while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done
rm -rf ${absolutePath}
# specify the custom helper pod yaml
rm -rf "$VOL_DIR"
helperPod: |-
apiVersion: v1
kind: Pod
38 changes: 3 additions & 35 deletions deploy/example-config.yaml
Original file line number Diff line number Diff line change
@@ -23,41 +23,11 @@ data:
}
setup: |-
#!/bin/sh
while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done
mkdir -m 0777 -p ${absolutePath}
mkdir -m 0777 -p "$VOL_DIR"
teardown: |-
#!/bin/sh
while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done
rm -rf ${absolutePath}
helperPod.yaml: |-
rm -rf "$VOL_DIR"
helper-pod.yaml: |-
apiVersion: v1
kind: Pod
metadata:
@@ -66,5 +36,3 @@ data:
containers:
- name: helper-pod
image: busybox
36 changes: 3 additions & 33 deletions deploy/local-path-storage.yaml
Original file line number Diff line number Diff line change
@@ -104,41 +104,11 @@ data:
}
setup: |-
#!/bin/sh
while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done
mkdir -m 0777 -p ${absolutePath}
mkdir -m 0777 -p "$VOL_DIR"
teardown: |-
#!/bin/sh
while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done
rm -rf ${absolutePath}
helperPod.yaml: |-
rm -rf "$VOL_DIR"
helper-pod.yaml: |-
apiVersion: v1
kind: Pod
metadata:
56 changes: 56 additions & 0 deletions examples/cache/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Example cache provisioner

This example shows how to use short-lived PersistentVolumes for caching.
A [buildah](https://github.com/containers/buildah)-based helper Pod is used to provision a container file system based on an image as PersistentVolume and commit it when deprovisioning.
Users can select the desired cache or rather the image using a PersistentVolumeClaim annotation that is passed through to the helper Pod as environment variable.

While it is not part of this example caches could also be synchronized across nodes using an image registry.
The [cache-provisioner](https://github.com/mgoltzsche/cache-provisioner) project aims to achieve this as well as other cache management features.

## Test

### Test the helper Pod separately

The helper Pod can be tested separately using docker locally:
```sh
./helper-test.sh
```

### Test the integration

_Please note that a non-overlayfs storage directory (`/data/example-cache-storage`) must be configured._
_The provided configuration is known to work with minikube (`minikube start`) and kind (`kind create cluster; kind export kubeconfig`)._

Install the example kustomization:
```sh
kustomize build . | kubectl apply -f -
```

If you want to test changes to the `local-path-provisioner` binary locally:
```sh
kubectl delete -n example-cache-storage deploy local-path-provisioner
(
cd ../..
go build .
./local-path-provisioner --debug start \
--namespace=example-cache-storage \
--configmap-name=local-path-config \
--service-account-name=local-path-provisioner-service-account \
--provisioner-name=storage.example.org/cache \
--pvc-annotation=storage.example.org \
--pvc-annotation-required=storage.example.org/cache-name
)
```

Within another terminal create an example Pod and PVC that pulls and runs a container image using [podman](https://github.com/containers/podman):
```sh
kubectl apply -f test-pod.yaml
kubectl logs -f cached-build
```

If the Pod and PVC are removed and recreated you can observe that, during the 2nd Pod execution on the same node, the image for the nested container doesn't need to be pulled again since it is cached:
```sh
kubectl delete -f test-pod.yaml
kubectl apply -f test-pod.yaml
kubectl logs -f cached-build
```
16 changes: 16 additions & 0 deletions examples/cache/config/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"nodePathMap": [
{
"node": "DEFAULT_PATH_FOR_NON_LISTED_NODES",
"paths": ["/data/example-cache-storage"]
},
{
"node": "minikube",
"paths": ["/data/example-cache-storage"]
},
{
"node": "kind-control-plane",
"paths": ["/var/opt/example-cache-storage"]
}
]
}
15 changes: 15 additions & 0 deletions examples/cache/config/helper-pod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: v1
kind: Pod
metadata:
name: helper-pod
spec:
containers:
- name: helper
image: quay.io/buildah/stable:v1.17.0
imagePullPolicy: IfNotPresent
securityContext:
privileged: true
hostPID: true
volumeMounts:
- name: data
mountPropagation: Bidirectional
76 changes: 76 additions & 0 deletions examples/cache/config/setup
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#!/bin/sh

set -eu

MOUNT_NAME="$(basename "$VOL_DIR")"
CACHE_DIR="$(dirname "$VOL_DIR")/.cache"
CACHE_NAME="${PVC_ANNOTATION_CACHE_NAME:-$(echo "$PVC_NAME" | sed -E 's/^(.+)-[^-]+$/\1/')}"
CACHE_IMAGE="cache/$CACHE_NAME"

# Args: NAME VALUE
validate() {
PATTERN='^[-_a-z0-9]+$'
echo "$2" | grep -Eq "$PATTERN" \
|| (echo "invalid $1 argument provided: $2 (must match $PATTERN)" >&2; false)
}

buildah() {
/usr/bin/buildah \
--root=$CACHE_DIR/containers/storage \
--storage-driver=overlay \
"$@"
}

# Mounts a volume directory based on the latest CACHE_NAME image.
mountCache() {
echo "Creating volume $VOL_DIR from cache '$CACHE_NAME'" >&2
mkdir -m 0777 "$VOL_DIR" || exit 2
(
# Create new volume from cache's latest container image
# (The latest cache image could be pulled from a registry here)
(buildah from --pull-never --name "$MOUNT_NAME" "$CACHE_IMAGE" \
|| ([ $? -eq 125 ] && (
buildah delete "$MOUNT_NAME"
buildah from --name "$MOUNT_NAME" scratch
))) >/dev/null &&
CONTAINERDIR="$(buildah mount "$MOUNT_NAME")" &&
mount -o bind,rshared "$CONTAINERDIR" "$VOL_DIR" &&
chmod 0777 "$VOL_DIR"
) || (
umount "$VOL_DIR" 2>/dev/null 1>&2
buildah umount "$MOUNT_NAME" 2>/dev/null 1>&2
buildah delete "$MOUNT_NAME" 2>/dev/null 1>&2
rm -rf "$VOL_DIR"
false
)
echo "$VOL_DIR"
}

# Unmounts a cache volume directory, commits it and tags it as latest image for the given CACHE_NAME.
umountCache() {
# Commit volume only if dir is mounted (node restart results in unmounted volumes).
if mountpoint -q "$VOL_DIR"; then
echo "Committing volume $VOL_DIR to cache '$CACHE_NAME'" >&2
IMGID="$(buildah commit -q --timestamp 1 "$MOUNT_NAME")" &&
buildah tag "$IMGID" "$CACHE_IMAGE" &&
# The latest cache image could be pushed to a registry here
umount "$VOL_DIR"
fi

# Delete volume / container
echo "Deleting volume $VOL_DIR" >&2
buildah umount "$MOUNT_NAME" >/dev/null || true
buildah delete "$MOUNT_NAME" >/dev/null || true
rm -rf "$VOL_DIR" || (printf 'error: volume deletion blocked by mount: '; grep $MOUNT_NAME /etc/mtab; false) >&2
}


mkdir -p "$CACHE_DIR/containers/storage"
validate CACHE_NAME "$CACHE_NAME"
validate MOUNT_NAME "$MOUNT_NAME"

if [ "${1:-}" = teardown ]; then
umountCache
else
mountCache
fi
3 changes: 3 additions & 0 deletions examples/cache/config/teardown
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh

sh /script/setup teardown
63 changes: 63 additions & 0 deletions examples/cache/helper-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/bin/sh

cd "$(dirname "$0")"

VOLNAME=pv-xyz1_default_build-cache
EXPECTED_CONTENT="testcontent $(date)"

# ARGS: SCRIPTNAME
runScript() {
mkdir -p testmount
docker run --rm --privileged \
-e VOL_DIR=/data/$VOLNAME \
-e VOL_NAME=pv-xyz \
-e VOL_SIZE_BYTES=12345678 \
-e PVC_NAME=pvc-xyz \
-e PVC_NAMESPACE=test-namespace \
-e PVC_ANNOTATION_CACHE_NAME=mycache \
--mount "type=bind,source=`pwd`/config,target=/script" \
--mount "type=bind,source=`pwd`/testmount,target=/data,bind-propagation=rshared" \
--entrypoint=/bin/sh \
quay.io/buildah/stable:v1.17.0 \
/script/$1
}

set -e

mkdir -p testmount
rm -rf testmount/$VOLNAME

echo
echo TEST setup
echo
(
set -ex
runScript setup

echo "$EXPECTED_CONTENT" > testmount/$VOLNAME/testfile
)

echo
echo TEST teardown
echo
(
set -ex
runScript teardown

[ ! -d testmount/$VOLNAME ] || (echo fail: volume should be removed >&2; false)
)

echo
echo TEST restore
echo
(
set -ex
VOLNAME=pv-xyz2_default_build-cache

runScript setup

CONTENT="$(cat testmount/$VOLNAME/testfile)"
[ "$CONTENT" = "$EXPECTED_CONTENT" ] || (echo fail: volume should return what was last written into that cache key >&2; false)

runScript teardown
)
39 changes: 39 additions & 0 deletions examples/cache/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: example-cache-storage

commonLabels:
app: example-cache-provisioner

resources:
- ../../deploy

patchesStrategicMerge:
- provisioner-patch.yaml

patchesJson6902:
- target:
version: v1
kind: Namespace
name: local-path-storage
path: namespace-patch.yaml
- target:
group: storage.k8s.io
version: v1
kind: StorageClass
name: local-path
path: storageclass-patch.yaml

configMapGenerator:
- name: local-path-config
namespace: local-path-storage
behavior: merge
files:
- config/config.json
- config/helper-pod.yaml
- config/setup
- config/teardown

generatorOptions:
disableNameSuffixHash: true
3 changes: 3 additions & 0 deletions examples/cache/namespace-patch.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
- op: replace
path: /metadata/name
value: example-cache-storage
19 changes: 19 additions & 0 deletions examples/cache/provisioner-patch.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: local-path-provisioner
namespace: local-path-storage
spec:
template:
spec:
containers:
- name: local-path-provisioner
env:
- name: PROVISIONER_NAME
value: storage.example.org/cache
- name: PVC_ANNOTATION
value: storage.example.org
- name: PVC_ANNOTATION_REQUIRED
value: storage.example.org/cache-name
- name: HELPER_POD_TIMEOUT
value: "2m"
6 changes: 6 additions & 0 deletions examples/cache/storageclass-patch.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
- op: replace
path: /metadata/name
value: example-cache
- op: replace
path: /provisioner
value: storage.example.org/cache
51 changes: 51 additions & 0 deletions examples/cache/test-pod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
apiVersion: v1
kind: Pod
metadata:
name: cached-build
spec:
restartPolicy: Never
securityContext:
runAsUser: 9000
runAsGroup: 9000
fsGroup: 9000
containers:
- name: build
image: mgoltzsche/podman:2.2.1-rc2
command: ["/bin/sh"]
args:
- -c
- |
set -ex
mktemp -d -p $HOME
([ -f $HOME/date ] || date > $HOME/date) && cat $HOME/date
OOMSCORE=$(cat /proc/self/oom_score_adj)
podman run --name build --rm --oom-score-adj "$OOMSCORE" alpine:3.12 echo hello from nested container
env:
- name: HOME
value: /podman
securityContext:
privileged: true
volumeMounts:
# simply cache home directory
- mountPath: /podman
name: cache
volumes:
- name: cache
persistentVolumeClaim:
claimName: build-cache
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: build-cache
annotations:
storage.example.org/cache-name: example-project
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage: 1Gi
storageClassName: example-cache
7 changes: 4 additions & 3 deletions examples/quota/README.md
Original file line number Diff line number Diff line change
@@ -2,8 +2,8 @@
this is an example to enable quota for xfs

# Usage
> 1. build a helper image using the sample dockerfile to replace helper image xxx/storage-xfs-quota:v0.1 at configmap(helperPod.yaml) of debug.yaml.
> 2. use the sample setup and teardown script at configmap of debug.yaml
> 1. build a helper image using the sample dockerfile to replace helper image xxx/storage-xfs-quota:v0.1 at configmap(config/helper-pod.yaml).
> 2. use the sample setup and teardown scripts contained within the kustomization.
Notice:
> 1. make sure the path at nodePathMap is the mountpoint of xfs which enables pquota
@@ -13,6 +13,7 @@ Notice:
> git clone https://github.com/rancher/local-path-provisioner.git
> cd local-path-provisioner
> go build
> kubectl apply -f debug.yaml
> kustomize build example/quota | kubectl apply -f -
> kubectl delete -n local-path-storage deploy local-path-provisioner
> ./local-path-provisioner --debug start --namespace=local-path-storage
```
File renamed without changes.
27 changes: 7 additions & 20 deletions examples/quota/setup → examples/quota/config/setup
Original file line number Diff line number Diff line change
@@ -1,23 +1,10 @@
#!/bin/sh

while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done

xfsPath=$(dirname "$absolutePath")
pvcName=$(basename "$absolutePath")
mkdir -p ${absolutePath}
xfsPath=$(dirname "$VOL_DIR")
pvcName=$(basename "$VOL_DIR")
sizeInBytes=$VOL_SIZE_BYTES

mkdir -p "$VOL_DIR"

# support xfs quota
type=`stat -f -c %T ${xfsPath}`
@@ -34,10 +21,10 @@ if [ ${type} == 'xfs' ]; then
id=$[${id}+1]
fi

echo "${id}:${absolutePath}" >> /etc/projects
echo "${id}:${VOL_DIR}" >> /etc/projects
echo "${pvcName}:${id}" >> /etc/projid

xfs_quota -x -c "project -s ${pvcName}"
xfs_quota -x -c "limit -p bhard=${sizeInBytes} ${pvcName}" ${xfsPath}
xfs_quota -x -c "report -pbih" ${xfsPath}
fi
fi
23 changes: 4 additions & 19 deletions examples/quota/teardown → examples/quota/config/teardown
Original file line number Diff line number Diff line change
@@ -1,22 +1,7 @@
#!/bin/sh

while getopts "m:s:p:" opt
do
case $opt in
p)
absolutePath=$OPTARG
;;
s)
sizeInBytes=$OPTARG
;;
m)
volMode=$OPTARG
;;
esac
done

xfsPath=$(dirname "$absolutePath")
pvcName=$(basename "$absolutePath")
xfsPath=$(dirname "$VOL_DIR")
pvcName=$(basename "$VOL_DIR")

# support xfs quota
type=`stat -f -c %T ${xfsPath}`
@@ -26,10 +11,10 @@ if [ ${type} == 'xfs' ]; then
xfs_quota -x -c "limit -p bhard=0 ${pvcName}" ${xfsPath}
fi

rm -rf ${absolutePath}
rm -rf "$VOL_DIR"

if [ ${type} == 'xfs' ]; then
echo "$(sed "/${pvcName}/d" /etc/projects)" > /etc/projects
echo "$(sed "/${pvcName}/d" /etc/projid)" > /etc/projid
xfs_quota -x -c "report -pbih" ${xfsPath}
fi

141 changes: 0 additions & 141 deletions examples/quota/debug.yaml

This file was deleted.

17 changes: 17 additions & 0 deletions examples/quota/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../../deploy

configMapGenerator:
- name: local-path-config
namespace: local-path-storage
behavior: merge
files:
- config/helper-pod.yaml
- config/setup
- config/teardown

generatorOptions:
disableNameSuffixHash: true
116 changes: 75 additions & 41 deletions main.go
Original file line number Diff line number Diff line change
@@ -5,7 +5,9 @@ import (
"os"
"os/signal"
"path/filepath"
"strings"
"syscall"
"time"

"github.com/Sirupsen/logrus"
"github.com/pkg/errors"
@@ -19,26 +21,34 @@ import (
)

var (
VERSION = "0.0.1"
FlagConfigFile = "config"
FlagProvisionerName = "provisioner-name"
EnvProvisionerName = "PROVISIONER_NAME"
DefaultProvisionerName = "rancher.io/local-path"
FlagNamespace = "namespace"
EnvNamespace = "POD_NAMESPACE"
DefaultNamespace = "local-path-storage"
FlagHelperImage = "helper-image"
EnvHelperImage = "HELPER_IMAGE"
DefaultHelperImage = "rancher/library-busybox:1.31.1"
FlagServiceAccountName = "service-account-name"
DefaultServiceAccount = "local-path-provisioner-service-account"
EnvServiceAccountName = "SERVICE_ACCOUNT_NAME"
FlagKubeconfig = "kubeconfig"
DefaultConfigFileKey = "config.json"
DefaultConfigMapName = "local-path-config"
FlagConfigMapName = "configmap-name"
FlagHelperPodFile = "helper-pod-file"
DefaultHelperPodFile = "helperPod.yaml"
VERSION = "0.0.1"
FlagConfigFile = "config"
FlagProvisionerName = "provisioner-name"
EnvProvisionerName = "PROVISIONER_NAME"
DefaultProvisionerName = "rancher.io/local-path"
FlagNamespace = "namespace"
EnvNamespace = "POD_NAMESPACE"
DefaultNamespace = "local-path-storage"
FlagHelperImage = "helper-image"
EnvHelperImage = "HELPER_IMAGE"
DefaultHelperImage = "rancher/library-busybox:1.31.1"
FlagServiceAccountName = "service-account-name"
DefaultServiceAccount = "local-path-provisioner-service-account"
EnvServiceAccountName = "SERVICE_ACCOUNT_NAME"
FlagKubeconfig = "kubeconfig"
DefaultConfigFileKey = "config.json"
DefaultConfigMapName = "local-path-config"
FlagConfigMapName = "configmap-name"
EnvConfigMapName = "CONFIGMAP_NAME"
FlagHelperPodFile = "helper-pod-file"
DefaultHelperPodFile = "helper-pod.yaml"
EnvHelperPodFile = "HELPER_POD_FILE"
FlagHelperPodTimeout = "helper-pod-timeout"
EnvHelperPodTimeout = "HELPER_POD_TIMEOUT"
FlagPVCAnnotation = "pvc-annotation"
EnvPVCAnnotation = "PVC_ANNOTATION"
FlagPVCAnnotationRequired = "pvc-annotation-required"
EnvPVCAnnotationRequired = "PVC_ANNOTATION_REQUIRED"
)

func cmdNotFound(c *cli.Context, command string) {
@@ -50,12 +60,15 @@ func onUsageError(c *cli.Context, err error, isSubcommand bool) error {
}

func RegisterShutdownChannel(done chan struct{}) {
sigs := make(chan os.Signal, 1)
sigs := make(chan os.Signal, 2)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
go func() {
sig := <-sigs
logrus.Infof("Receive %v to exit", sig)
logrus.Infof("Received %v signal - terminating", sig)
close(done)
<-sigs
logrus.Info("Received 2nd termination signal - exiting forcefully")
os.Exit(254)
}()
}

@@ -80,21 +93,16 @@ func StartCmd() cli.Command {
EnvVar: EnvNamespace,
Value: DefaultNamespace,
},
cli.StringFlag{
Name: FlagHelperImage,
Usage: "Required. The helper image used for create/delete directories on the host",
EnvVar: EnvHelperImage,
Value: DefaultHelperImage,
},
cli.StringFlag{
Name: FlagKubeconfig,
Usage: "Paths to a kubeconfig. Only required when it is out-of-cluster.",
Value: "",
},
cli.StringFlag{
Name: FlagConfigMapName,
Usage: "Required. Specify configmap name.",
Value: DefaultConfigMapName,
Name: FlagConfigMapName,
Usage: "Required. Specify configmap name.",
EnvVar: EnvConfigMapName,
Value: DefaultConfigMapName,
},
cli.StringFlag{
Name: FlagServiceAccountName,
@@ -103,9 +111,28 @@ func StartCmd() cli.Command {
Value: DefaultServiceAccount,
},
cli.StringFlag{
Name: FlagHelperPodFile,
Usage: "Paths to the Helper pod yaml file",
Value: "",
Name: FlagHelperPodFile,
Usage: "Paths to the Helper pod yaml file",
EnvVar: EnvHelperPodFile,
Value: "",
},
cli.StringFlag{
Name: FlagHelperPodTimeout,
Usage: "Helper pod command execution timeout",
EnvVar: EnvHelperPodTimeout,
Value: "2m",
},
cli.StringFlag{
Name: FlagPVCAnnotation,
Usage: "A PersistentVolumeClaim annotation or prefix passed through to the helper pod as env vars (PVC_ANNOTATION[_<ANNOTATION_PATH_SUFFIX>]=<ANNOTATION_VALUE>)",
EnvVar: EnvPVCAnnotation,
Value: "",
},
cli.StringFlag{
Name: FlagPVCAnnotationRequired,
Usage: "Annotation that must be specified on PersistentVolumeClaims (multiple comma-separated)",
EnvVar: EnvPVCAnnotationRequired,
Value: "",
},
},
Action: func(c *cli.Context) {
@@ -196,17 +223,19 @@ func startDaemon(c *cli.Context) error {
return fmt.Errorf("invalid empty flag %v and it also does not exist at ConfigMap %v/%v with err: %v", FlagConfigFile, namespace, configMapName, err)
}
}
helperImage := c.String(FlagHelperImage)
if helperImage == "" {
return fmt.Errorf("invalid empty flag %v", FlagHelperImage)
}

serviceAccountName := c.String(FlagServiceAccountName)
if serviceAccountName == "" {
return fmt.Errorf("invalid empty flag %v", FlagServiceAccountName)
}

// if helper pod file is not specified, then find the helper pod by configmap with key = helperPod.yaml
pvcAnnotation := c.String(FlagPVCAnnotation)
pvcAnnotationsRequired := c.String(FlagPVCAnnotationRequired)
var requiredPVCAnnotations []string
if pvcAnnotationsRequired != "" {
requiredPVCAnnotations = strings.Split(pvcAnnotationsRequired, ",")
}

// if helper pod file is not specified, then find the helper pod by configmap with key = helper-pod.yaml
// if helper pod file is specified with flag FlagHelperPodFile, then load the file
helperPodFile := c.String(FlagHelperPodFile)
helperPodYaml := ""
@@ -221,8 +250,13 @@ func startDaemon(c *cli.Context) error {
return fmt.Errorf("could not open file %v with err: %v", helperPodFile, err)
}
}
helperPodTimeoutStr := c.String(FlagHelperPodTimeout)
helperPodTimeout, err := time.ParseDuration(helperPodTimeoutStr)
if err != nil {
return fmt.Errorf("invalid %s option provided: %s", FlagHelperPodTimeout, err)
}

provisioner, err := NewProvisioner(stopCh, kubeClient, configFile, namespace, helperImage, configMapName, serviceAccountName, helperPodYaml)
provisioner, err := NewProvisioner(stopCh, kubeClient, configFile, namespace, configMapName, serviceAccountName, helperPodYaml, helperPodTimeout, pvcAnnotation, requiredPVCAnnotations)
if err != nil {
return err
}
260 changes: 182 additions & 78 deletions provisioner.go

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions util.go
Original file line number Diff line number Diff line change
@@ -33,5 +33,8 @@ func loadHelperPodFile(helperPodYaml string) (*v1.Pod, error) {
if err != nil {
return nil, fmt.Errorf("invalid unmarshal the helper pod with helperPodJson: %v", string(helperPodJSON))
}
if len(p.Spec.Containers) == 0 {
return nil, fmt.Errorf("helper pod template does not specify any container")
}
return &p, nil
}

0 comments on commit 9818353

Please sign in to comment.