nerdctl compose hangs in 2.0

[midnight-wonderer]

Edited: The Gist

nardctl compose can not run concurrently at the moment due to: #3543.
Dubbed global compose lock; this is due to some of the nerdctl compose functionalities not being safe for executing simultaneously yet.

---

Original:

Description

nerdctl compose ps never returns results; it simply hangs forever.

Steps to reproduce the issue

1. in a compose project, execute nerdctl compose up
2. in another terminal, execute nerdctl compose ps
3. it never returns

Additional info

• nerdctl compose --debug ps shows the following, then hangs

DEBU[0000] stateDir: /run/user/1000/containerd-rootless
DEBU[0000] RootlessKit detach-netns mode: true
DEBU[0000] rootless parent main: executing "/usr/bin/nsenter" with [-r/ -w/to/my/project --preserve-credentials -m -U -t 2342 -F nerdctl compose --debug ps]

• Ctrl + C (SIGINT) works properly. It looks like nerdctl is waiting for something.

Describe the results you received and expected

The list of containers in the compose project.

What version of nerdctl are you using?

Client:
 Version:	<unknown>
 OS/Arch:	linux/amd64
 Git commit:	<unknown>
 buildctl:
  Version:	0.17.1

Server:
 containerd:
  Version:	v1.7.23
  GitCommit:	57f17b0a6295a39009d861b89e3b3b87b005ca27
 runc:
  Version:	1.2.1
  GitCommit:	v1.2.1-0-gd7735e388ef5

Additional info

zypper info nerdctl returns

Information for package nerdctl:
--------------------------------
Repository     : Main Repository (OSS)
Name           : nerdctl
Version        : 2.0.0-1.1
Arch           : x86_64
Vendor         : openSUSE
Installed Size : 28.4 MiB
Installed      : Yes
Status         : up-to-date
Source package : nerdctl-2.0.0-1.1.src
Upstream URL   : https://github.com/containerd/nerdctl
Summary        : Docker-compatible CLI for containerd
Description    : 
    nerdctl is a Docker-compatible CLI for contained.

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

Host information

Client:
 Namespace:	default
 Debug Mode:	false

Server:
 Server Version: v1.7.23
 Storage Driver: overlayfs
 Logging Driver: json-file
  Cgroup Driver:  : systemd
  Cgroup Version: : 2
 Plugins:
  Log:     fluentd journald json-file none syslog
  Storage: btrfs native overlayfs
 Security Options:
  apparmor
  seccomp
   Profile:	builtin
  cgroupns
  rootless
 Kernel Version:   6.11.8-1-default
 Operating System: openSUSE Tumbleweed
 OSType:           linux
 Architecture:     x86_64
 CPUs:             8
 Total Memory:     30.97GiB
 Name:             NUC
 ID:               af29ba58-52af-430a-acf6-60e8f0b70de2

WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No cpu cfs period support
WARNING: No cpu cfs quota support
WARNING: No cpu shares support
WARNING: No cpuset support
WARNING: IPv4 forwarding is disabled
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

[midnight-wonderer]

Please note that when the compose is not running, the command returns properly.

$ nerdctl compose --debug ps
DEBU[0000] stateDir: /run/user/1000/containerd-rootless
DEBU[0000] RootlessKit detach-netns mode: true
DEBU[0000] rootless parent main: executing "/usr/bin/nsenter" with [-r/ -w/to/my/project --preserve-credentials -m -U -t 2242 -F nerdctl compose --debug ps]
DEBU[0000] filters: [labels."com.docker.compose.project"==myproject,labels."com.docker.compose.service"==database labels."com.docker.compose.project"==myproject,labels."com.docker.compose.service"==application]
NAME IMAGE COMMAND SERVICE STATUS PORTS

[midnight-wonderer]

Okay, sorry for the wrong description earlier. I scoped down the culprit.

It seems that during nerdctl compose logs -f application running, other compose commands (Like docker compose ps) will wait for it to finish. (Which never will.)

After I send SIGINT to compose logs, the command continues properly.
In my opinion, this still looks like a bug.

[apostasie]

Thanks @midnight-wonderer

The current implementation of compose in nerdctl is not safe to use concurrently - to alleviate that (short of a full rewrite), a global lock was introduced: #3543

This is definitely the reason for the behavior here ^^^.

Agreed with you that in the case of logs, it should not lock (or at least it should release early).

Currently, the lock is implemented once, inside pkg/cmd/composer.

To fix this we could either move the lock to every compose command (or at least have an explicit call to release the lock early in logs)

[midnight-wonderer]

Thanks for the details.
I am a bit surprised that virtually no one has reported/complained about this here since the 2.0 release.
Thank you for improving nerdctl, and I hope rough edges eventually get ironing out.

Note for those finding this thread later.

Now the issue became clear, a workaround is possible. It seems the lock only affects compose, so, in the mean time, we can use other commands.
As an example, for me:

• nerdctl container ls supports --filter. So, I can use nerdctl container ls --filter label=com.docker.compose.project=$(COMPOSE_PROJECT_NAME). Note: $() is the Make syntax.
• To do so, I have to set COMPOSE_PROJECT_NAME environment variable.
• My local development scripts became responsive again. 🎉 I hope they are applicable to your situations too.

[calltherain]

So according to #3543 that means I can't use compose to host multiple systems anymore.
To me this is a bit of strange, it is a common use case.
I use compose to run mediawiki, gitea, harbor on my server.
Any recommendations? Thanks

[midnight-wonderer]

Wait, is the lock not per project?
I have yet to hit that one, but the functionality sure is quite handy.

How so few people came here.
I will help rename the subject of the issue to be more generic.
I don't have a recommendation, though.

[calltherain]

No, the lock is not per project. If I start my mediawiki using nerdctl compose up , I can't run run other systems ( harbor, gitea, for example) anymore.
All compose commands hangs, like

nerdctl compose images
nerdctl compose pull
nerdctl compose top
...