Support publishing as Docker Engine managed plugin

[olljanat]

I collected need files based @nickbreen great work on https://github.com/nickbreen/docker-lvm-plugin , tested that it works, updated CI and readme.

Will allow you close #66

[shishir-a412ed]

@olljanat This is great! Thanks for opening this PR.

I am trying to run the docker plugin install command and getting this error:

[root@fedora32 docker-lvm-plugin]# docker plugin install --alias lvm containers/docker-lvm-plugin/docker-lvm-plugin VOLUME_GROUP=vg0
Error response from daemon: pull access denied for containers/docker-lvm-plugin/docker-lvm-plugin, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

[olljanat]

@shishir-a412ed that install command will only works after image is build and published to Docker Hub by this project maintainer(s).

Before that you need build it using command which I added to CircleCI file.

[shishir-a412ed]

@olljanat Thanks for the tip! I will check and update soon.

[virtualdxs]

Any progress on this? I'd really like to avoid needing to make install if possible.

[shishir-a412ed]

@olljanat @virtualdxs Apologies for the delayed response! We were shut down the last 2 weeks (Christmas and New Year).
I will try to review it this week.

[shishir-a412ed]

@olljanat In the circleci logs, I see some errors messages in the make -f plugin/Makefile create step

docker rm --force --volumes rootfs || true
Error: No such container: rootfs
docker create --name rootfs containers/docker-lvm-plugin:rootfs
023f792d5fdb06c9b3a6c5afed42659f32ca5c30eab43b1f84bbd0ad61de7803
rm -rf plugin/rootfs
mkdir -p plugin/rootfs
docker export rootfs | tar -x -C plugin/rootfs
docker rm --force --volumes rootfs
rootfs
docker plugin rm --force containers/docker-lvm-plugin || true
Error: No such plugin: containers/docker-lvm-plugin
docker plugin create containers/docker-lvm-plugin plugin

Seems like the test is failing but just returning true

[shishir-a412ed]

@olljanat I was able to build the plugin shm32/docker-lvm-plugin:1.0 and push it to my docker hub. But when I try to install the plugin, I am getting the following error. Will try to dig more later.

[root@fedora32 plugin]# docker plugin install --alias lvm shm32/docker-lvm-plugin:1.0 VOLUME_GROUP=vg0
Error response from daemon: pull access denied for shm32/docker-lvm-plugin, the repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Just FYI: The repository (shm32/docker-lvm-plugin:1.0) is public

Also, there are no instructions in there. Can you give me the commands (and the order in which they need to be executed) in order for me to build the plugin and try it out?

[olljanat]

@shishir-a412ed this stuff looks that you have build is as normal container, not as plugin because docker pull shm32/docker-lvm-plugin:1.0 command accepts that image take when it should not.

So plz remove that tag and actually whole shm32/docker-lvm-plugin repository from Docker Hub side that and then rebuild and push it using these commands (first plugin push will create repository on correct format, thing which took me for a while to figure out):

docker plugin create shm32/docker-lvm-plugin:1.0
docker plugin push shm32/docker-lvm-plugin:1.0

then docker plugin install plugin command should accept it (and docker pull shouldn't).

Official documentation is available on https://docs.docker.com/engine/extend/#developing-a-plugin

[shishir-a412ed]

@olljanat Thank you for those instructions! Those were really helpful.

I was able to remove the existing plugin both from my docker-hub and from my local root filesystem docker plugin rm

Then as per your instructions, I created the plugin and pushed it to the docker hub.
However when I tried installing it, I am getting the following error:

[root@fedora32 docker-lvm-plugin]# docker plugin install shm32/docker-lvm-plugin:1.0
Plugin "shm32/docker-lvm-plugin:1.0" is requesting the following privileges:
 - network: [host]
 - mount: [/dev]
 - allow-all-devices: [true]
 - capabilities: [CAP_SYS_ADMIN]
Do you grant the above permissions? [y/N] y
1.0: Pulling from shm32/docker-lvm-plugin
b665ebfa4717: Download complete
Digest: sha256:d119d20b6b3e21a86c16bd2cd36c5163d6281f301c0ac4a692e8ef6c3109c300
Status: Downloaded newer image for shm32/docker-lvm-plugin:1.0
Error response from daemon: dial unix /run/docker/plugins/8d7abab47f726368b8e16565896bf7442b5f41855886dae7dbcc545058831171/lvm.sock: connect: no such file or directory

Just curious when you were testing it, did you had docker-lvm-plugin systemd service running on the host.
I explicitly stopped/removed the docker-lvm-plugin systemd service as I wanted to use the plugin.
Any ideas about the error?

[kale1d0code]

I've taken some interest in getting this working.

I've had a thought about the problem and I think it may be down to the "on-demand socket activated" nature of the plugin.
I believe docker is trying to access the socket before the plugin's container creates it.
However I will admit that I don't understand the differences in how the plugin is behaving when running locally or within its special plugin container (does the plugin create a socket when running on the host before docker ls is called?).
I'll identify and make the changes required after I setup my own docker hub account and practised building the plugin a few times. This is my first time working on a docker plugin

One note worthy thing
when you built the plugin against your personal docker hub account (shm32/docker-lvm-plugin)
the tag was made part of the name (shm32/docker-lvm-plugin:1.0:latest instead of shm32/docker-lvm-plugin:1.0)
I guess the issue will be in the makefile somewhere

I will try to sort this out with my own pull request,
but I don't know how to 'fork' another pull request and I have little go and docker plugin dev experience

[olljanat]

I will try to sort this out with my own pull request, but I don't know how to 'fork' another pull request and I have little go and docker plugin dev experience

Fork and clone it normally and then fetch my branch as starting point with:

git remote add olljanat https://github.com/olljanat/docker-lvm-plugin
git fetch olljanat
git checkout olljanat/docker-hub-support

But as there is just couple of files you can easily copy those also.

[kale1d0code]

I have found my first error,
I've jumped into the plugins container and tried running the binary directly to see what shows in stdout
this returns:
FATA[0000] Error initializing lvmDriver Unix syslog delivery error
I believe this might be why the plugin never creates the socket as it crashes at this point.

maybe some additional packages for syslog are required in the container,
I'll see if I can get any further.

[kale1d0code]

I have managed to get the plugin to start as a container after changing the logging library from log/syslog to log
I will create a Pull request soon after some testing

[kale1d0code]

moby/moby#33009 is holding me back at the moment,
I need to push my copy of the plugin to my own dockerhub for testing