Build and push manager in demo repo?

[Abby-Wheelis]

In part to reduce the time it takes to run the demo, we would like to just be able to pull the manager, rather than having to build it from scratch. EVerest ci-cd does build the manager, but it does not push it. Previous attempts to build the manager in github actions ourself has failed. Why does it work for EVerest but not for us? Could EVerest just push the manager after they build it and we could just pull it?

Starting this thread to investigate the existing GitHub Actions, how they work, and if we could expand them ourselves or just leverage the existing processes!

[Abby-Wheelis]

In everest-ci the "build and push docker images" action has several jobs, some of which have the same name? (everything depends on the env-setup)

• env-setup: Evaluate force rebuild and set env variables as outputs
• run-env-base: Build and push run-env-base docker image
• build-env-base: Build and push build-env-base docker image (depends on run-env-base)
• dev-env-base: Build and push dev-env-base docker image (depends on build-env-base)
• build-kit-base: Build and push build-kit docker image (depends on build-env-base)
• then a few jobs that are dedicated to backwards compat ..

Each of the "build and push" jobs use the deploy-single-docker-image.yml workflow which has two jobs - "Check for changes" and "Build and push". Check changes uses provided before and after github references to check for changes and see if a rebuild is needed - it is not set to force the rebuild. "Build and push" uses a docker-build-push github action which uses Buildx. One of the steps is to log into DockerHub, could this be why we can't build? Maybe there is some account type we don't have? In the job the option push is set to true, it looks like from the docs that it might be pushed to a registry? It really looks like its being pushed, but not exactly sure where

[Abby-Wheelis]

Just to see, I tried pulling each of the images that the everest-ci page says are deployed, and the ghcr.io/everest/everest-ci/run-env-base and ghcr.io/everest/everest-ci/dev-env-base pulled fine, but ghcr.io/everest/everest-ci/build-env-base gives an error:

request returned Internal Server Error for API route and version http://%2FUsers%2Fawheelis%2F.docker%2Frun%2Fdocker.sock/v1.24/images/create?fromImage=ghcr.io%2Feverest%2Feverest-ci%2Fbuild-env-base&tag=latest, check if the server supports the requested API version

I need to investigate that error, but I also want to see if I can run either of the images that pulled.

[shankari]

"Build and push" uses a docker-build-push github action which uses Buildx. One of the steps is to log into DockerHub, could this be why we can't build? Maybe there is some account type we don't have?

At least from everest-demo, we push to the GitHub builtin repo. The GITHUB_TOKEN, with the elevated permissions is able to push, and we don't need to create a new login step.

[Abby-Wheelis]

I have been experimenting with the workflow script for .github/workflows/cicd.yaml, uncommenting the out of use sections to see what errors are present.

I have been able to get the portion for mqtt to run, and pushed to my repo without errors.
The manager workflow is taking a long time (25 minutes and counting) to build and export to docker, no error yet.
The nodered workflow had a 403 forbidden error that I have not figured out yet.

[Abby-Wheelis]

Update: manager took about 30 minutes total, but worked without erroring out.

I pulled it and it took less than 10 minutes, which seems faster than normal, but I'm not sure what kind of speed we would expect if this work.

Got one warning that I still need to investigate, but it still seemed to work.

1 warning found (use docker --debug to expand):
 - FromPlatformFlagConstDisallowed: FROM --platform flag should not use constant value "linux/x86_64" (line 1)

[Abby-Wheelis]

It's still not pushing pre-built images it seems. Running the demo with what I pushed to my repo and what is already in everest-demo is not any faster. I need to dig into the workflow to see what is built and what is pushed. I don't think the manager is built in everest-ci because none of the actions have runtimes long enough if the 30 mins I saw is a decent benchmark.

I will bring this up in the FIT group call tomorrow, to see what others think about the problem. It looks like the docker images for EVerest were reworked about 6 months ago, #178, but the proposed changes to the manager were never enacted.

[Abby-Wheelis]

After the FIT meeting this morning I have a few new leads to try:

• adding build artifacts to current workflows
• use the workflow from everest-ci and extend it to upload artifacts such as the manager binary

It sounds like everest-framework could be the best place to do this, though similar steps may be also possible in everest-demo. When choosing how to implement this, I want to keep in mind the ongoing maintenance to the repository, and ensure that the workflow I choose is reusable and won't fall out of date with the rest of the organization.