x-initialSync should always copy files and directories when --watch is passed

[mvhatch]

Description

Running docker compose up --watch <service> does not copy files or directories when the file/directory is modified before image creation, even when setting x-initialSync: true. Why not just copy the files/directories without checking its modified timestamp?

[ndeloof]

x-initialSync is an experimental extension (as the tag name suggests). Actual behavior is subject to changes.
We only sync files which aren't up-to-date with image as such a copy can impact a large number of files and can slow down initial start for no reason.
Can you please clarify your usage? Why would you need to sync files which are already up-to-date in built image ?

[mvhatch]

I have a monorepo with dozens of services. I have a single compose.yml file with a top-level anchor the defines some common attributes for a service, including a docker image. This docker image only provides some common utilities and dependencies, but it does not copy any files from the monorepo. Instead, I'd like to specify a develop attribute for each service in my compose.yml, and watch a subset of files in my monorepo.

So to answer your question:

Why would you need to sync files which are already up-to-date in built image ?

Because there are no files in my built image. This seems to be a fairly common pattern (see thread here asking for the same thing)

Does docker compose cp not copy the files a user requests because it could be a large number of files? Can you allow end users to copy a large number of files w/ x-InitialSync in the same vein? Alternatively, could you allow docker compose --watch to watch a subset of files, even if some of those files are already being bind-mounted?

[ndeloof]

Because there are no files in my built image

Does this mean you can't run docker compose up with this image, without an extra step to copy content ?

Example on #11102 based on Traeffic demonstrate this misconception. watch should be used to make development iteration simpler and faster, but application should be able to run without. A workaround is to add a Dockerfile to add required files, so that image you run has required content:

services:
  traefik:
    build:
      dockerfile-inline: | 
        FROM: traefik:v2.10.7
        COPY ./traefik /etc/traefik
    develop:
      watch:
        - action: sync+restart
          path: ./traefik
          target: /etc/traefik

[mvhatch]

Does this mean you can't run docker compose up with this image, without an extra step to copy content ?

Not entirely the case. Our Dockerfile has a volume section that uses env var interpolation (e.g. ${APP_HOST_MOUNT}) that is properly set when executing make run. This bind mounts the developer's filesystem into the container. However, make watch sets the value to /dev/null (because docker compose watch is unable to watch anything under a directory that is bind mounted) and then copies the contents if the filesystem into the docker container.

It's just a little annoying to do this hack, and I was hoping to get this functionality directly from x-initialSync: true.

[mmx86]

In my case I'm syncing nginx configs directory from two different sources:

1. Service related.
2. Dev environment related (some conf.d entires, some templates).

Second one is not synced initially.

    develop:
      watch:
        - action: sync+restart
          path: ./my-service/nginx/
          target: /etc/nginx/
        - action: sync+restart
          path: ./dev/my-service/nginx/
          target: /etc/nginx/
          x-initialSync: true