feat: add support to build image from docker file #69

dineshkumar-j-genea · 2024-10-04T12:41:09Z

Description

Currently app-gen script has a limitation that only supports pulling images specified in the Compose file using docker pull, but it does not support building images locally using the build option with a Dockerfile.

It would be highly beneficial to have a feature that allows building Docker images locally during Artifact creation with the app-gen script. This would enable the user to define build instructions in the docker-compose.yml file, build the image on their local system, and then include that image in the Artifact for deployment. This approach would enhance flexibility and accommodate scenarios where local image building is necessary, such as incorporating local changes or when access to external registries is restricted.

Example of the current docker-compose.yml structure:

version: '3.8'

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    image: myapp:latest  # Specifies the name of the image to be created
    container_name: myapp-container  # Specifies the container name
    ports:
      - "8080:8080"
    environment:
      - NODE_ENV=development
    volumes:
      - ./:/app

This feature request comes from https://hub.mender.io/t/support-for-local-docker-image-builds-and-rollback-using-stored-images/7157

Mender Atlassian Ticket:

https://northerntech.atlassian.net/browse/MEN-7539

This commit introduces the ability to build images directly from a Dockerfile, enhancing the build process for containerized applications. Changelog: Added support for building images from Dockerfiles. Ticket: MEN-7539 Signed-off-by: dineshkumar-j-genea <[email protected]>

merlin-northern · 2024-10-07T12:49:23Z

@dineshkumar-j-genea thank you very much for this long-awaited feature. I find it very useful, and you have no idea how many times I wanted it.
what we need is:

testing around it I would say at least acceptance test
fix the failing tests as they fail now

dineshkumar-j-genea · 2024-10-10T12:41:47Z

@merlin-northern I have fixed the failing test cases. can you review the changes once

merlin-northern · 2024-10-11T05:47:46Z

@merlin-northern I have fixed the failing test cases. can you review the changes once

thank you very much. I would like to have my colleague Lluís opinion here as well.

This will modify the - pull images passed as args - image_not_present test case updated - ci config to install the yq package before testing - update the test script to use debian version instead of the alpine version due to the platform conflict. Signed-off-by: Dineshkumar J <[email protected]>

mender-test-bot · 2024-10-11T11:42:55Z

Merging these commits will result in the following changelog entries:

Changelogs

app-update-module (feature/build-with-dockerfile)

New changes in app-update-module since master:

Features

Added support for building images from Dockerfiles.
(MEN-7539)

dineshkumar-j-genea · 2024-10-14T07:42:03Z

@lluiscampos Did you get the chance to review this PR Request?

MuchoLucho · 2024-10-30T13:49:10Z

@lluiscampos just following up on here. Can you take a look?

merlin-northern · 2024-10-30T14:01:43Z

@lluiscampos just following up on here. Can you take a look?

there was a small re-assignment: I will take a look at it, this week for sure. sorry for the delay.

merlin-northern

sorry for not being clear on this one: we need a separate and new test scenario for this. let me know if you want me to write one.

merlin-northern · 2024-11-01T06:39:07Z

acceptance-tests/scenarios.d/03.basic.failed.images.not.present.sh

+        --application-name myapp0a # we expect a failure here now as we are using docker-compose to pull the images now
+
+    if [ $? -ne 0 ]; then
+        echo "images_not_present: app-gen artifact generation failed which is expected"
+        return 0
+    else
+        echo "Unexpected success, we were expecting failure"
+        return 1
+    fi
+    # echo "images_not_present: checking install rc"
+    # mender install "$artifact_file" && return 2 # we expect a failure
+    # echo "images_not_present: checking for running containers"
+    # docker ps -q | grep -q . && return 3 # and we expect nothing to be running
+    # return 0


we probably should not modify the existing tests, why do we need to do it?

in this test case, if you closely observe the existing flow, you will that we used image tag to pull image and then mender install it uses the docker compose to validate the image and expected a failure, as the image argument and docker compose totally defined with different image name.

but right now, as per the requirement we are using the docker-compose also to pull an image, which means image argument is essentially becomes the optional one.

with that, during the artifact generation itself, the script will throw an error in the new logic.

merlin-northern · 2024-11-01T06:40:25Z

acceptance-tests/scenarios.d/06.rollback.on.images.not.present.sh

+    # echo "images_not_present: checking install rc"
+    # mender install "$artifact_file" && return 2 # we expect a failure
+    # sleep $timeout_s
+    # echo "images_not_present: checking for running containers"
+    # docker ps
+    # diff "${temp_dir}/before-rollback-$$" <(docker ps --format '{{.Image}}')
+    # return 0


there is a bit of commented out code in this pull request, could you clean that?

dineshkumar-j-genea · 2024-11-04T12:11:11Z

@merlin-northern I am right now occupied with something else. it will be better, if you can take care of test case.

but in my opinion, there is no specific test case required, as all the existing test cases are actually verifying the new changes.

dineshkumar-j-genea marked this pull request as ready for review October 4, 2024 12:50

dineshkumar-j-genea force-pushed the feature/build-with-dockerfile branch from 1d1ff51 to daa3b15 Compare October 4, 2024 13:12

dineshkumar-j-genea force-pushed the feature/build-with-dockerfile branch 3 times, most recently from d0888cd to 13993b2 Compare October 4, 2024 13:31

lluiscampos requested review from merlin-northern and lluiscampos October 7, 2024 09:56

dineshkumar-j-genea force-pushed the feature/build-with-dockerfile branch 10 times, most recently from 3fd1f5e to d51cd04 Compare October 10, 2024 12:26

dineshkumar-j-genea force-pushed the feature/build-with-dockerfile branch from d51cd04 to 1760100 Compare October 11, 2024 05:45

dineshkumar-j-genea force-pushed the feature/build-with-dockerfile branch from 1760100 to a9b3515 Compare October 11, 2024 11:42

merlin-northern self-assigned this Oct 30, 2024

merlin-northern reviewed Nov 1, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support to build image from docker file #69

feat: add support to build image from docker file #69

dineshkumar-j-genea commented Oct 4, 2024

merlin-northern commented Oct 7, 2024

dineshkumar-j-genea commented Oct 10, 2024 •

edited

Loading

merlin-northern commented Oct 11, 2024

mender-test-bot commented Oct 11, 2024 •

edited by jira bot

Loading

dineshkumar-j-genea commented Oct 14, 2024

MuchoLucho commented Oct 30, 2024

merlin-northern commented Oct 30, 2024

merlin-northern left a comment

merlin-northern Nov 1, 2024

dineshkumar-j-genea Nov 4, 2024

merlin-northern Nov 1, 2024

dineshkumar-j-genea commented Nov 4, 2024 •

edited

Loading

feat: add support to build image from docker file #69

Are you sure you want to change the base?

feat: add support to build image from docker file #69

Conversation

dineshkumar-j-genea commented Oct 4, 2024

Description

Example of the current docker-compose.yml structure:

merlin-northern commented Oct 7, 2024

dineshkumar-j-genea commented Oct 10, 2024 • edited Loading

merlin-northern commented Oct 11, 2024

mender-test-bot commented Oct 11, 2024 • edited by jira bot Loading

Changelogs

app-update-module (feature/build-with-dockerfile)

Features

dineshkumar-j-genea commented Oct 14, 2024

MuchoLucho commented Oct 30, 2024

merlin-northern commented Oct 30, 2024

merlin-northern left a comment

Choose a reason for hiding this comment

merlin-northern Nov 1, 2024

Choose a reason for hiding this comment

dineshkumar-j-genea Nov 4, 2024

Choose a reason for hiding this comment

merlin-northern Nov 1, 2024

Choose a reason for hiding this comment

dineshkumar-j-genea commented Nov 4, 2024 • edited Loading

dineshkumar-j-genea commented Oct 10, 2024 •

edited

Loading

mender-test-bot commented Oct 11, 2024 •

edited by jira bot

Loading

dineshkumar-j-genea commented Nov 4, 2024 •

edited

Loading