Improve docker testing (#36)

* Changed .gitignore to ignore DS_Store * Update .travis.yml * Create migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Added the Unit Test stage to the new workflow * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update .gitignore * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml working on improved docker testing through GitHub actions (not complete) * Removal + Improvements Removes travis.yml from this branch as well. Started improving the Docker Hub test step on the workflow. * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * trying to get github actions work * Update Dockerfile * changes * Update migration-workflow.yml * Update migration-workflow.yml * Update Dockerfile * changes * Update migration-workflow.yml * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * changes * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * c * c2 * Update Dockerfile * c3 * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Changes after meeting with Jeremy * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * type=registry * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update Dockerfile * Update migration-workflow.yml * hope this works * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * naming changes and improvements * Update Dockerfile * Created .dockerignore * Update Dockerfile * empty commit * Update migration-workflow.yml * empty commit * empty commit 2 * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * hoping gradle will cache * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * trying copying into . * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * empty commit * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * Update migration-workflow.yml * empty commit * Update migration-workflow.yml * Update migration-workflow.yml * empty commit Co-authored-by: Jeremy Adams <[email protected]>
ga4gh · Jan 26, 2022 · 01254b5 · 01254b5
1 parent dafaa03
commit 01254b5
Show file tree

Hide file tree

Showing 6 changed files with 231 additions and 133 deletions.
diff --git a/.dockerignore b/.dockerignore
@@ -0,0 +1,5 @@
+#these files don't need to be sent to the docker build context
+.gradle/
+bin/
+build/
+gradle/
diff --git a/.github/workflows/migration-workflow.yml b/.github/workflows/migration-workflow.yml
@@ -5,49 +5,78 @@ jobs:
     runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v2
-    - name: Set up JDK 11
+
+    - name: Setting up JDK 11
       uses: actions/setup-java@v2
       with:
         java-version: '11'
         distribution: 'adopt' #using a specific distribution of jdk11 (AdoptOpenJDK)
-    - name: Setup Gradle
+
+    - name: Setting up Gradle
       uses: gradle/gradle-build-action@v2
-    - name: Setup Gradle Wrapper
+
+    - name: Setting up Gradle Wrapper
       run: "gradle wrapper"
-    - name: Validate Wrapper
+
+    - name: Validating Wrapper
       uses: gradle/wrapper-validation-action@v1
-    - name: Before Script
-      run: | #Setting up the SQLite database and Make
+
+    - name: Setting up the SQLite database and Make
+      run: | 
         sqlite3 -version
         make -version
         make clean-sqlite
         make sqlite-db-build
         make sqlite-db-populate
-    - name: Script
+
+    - name: Running Tests
       run: "./gradlew test --tests=org.*"
-    - name: Check Gradle Wrapper Tasks
-      run: "./gradlew tasks"
-    - name: After Success
+
+    - name: After Success Submitting Code Coverage
       run: | #jacocoTestReport is for testing code coverage, submits the last report to the link
         ./gradlew jacocoTestReport
         bash <(curl -s https://codecov.io/bash)
+
   Docker-Integration-Test:
     needs: Unit-Test
     runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v2
-    - name: Set up JDK 11
-      uses: actions/setup-java@v2
-      with:
-        java-version: '11'
-        distribution: 'adopt' #using a specific distribution of jdk11 (AdoptOpenJDK)
-    - name: Before Install
+
+    - name: Setting up Gradle
+      uses: gradle/gradle-build-action@v2
+
+    - name: Setting Up Docker Buildx #used for caching image layers, improves performance
+      id: buildx
+      uses: docker/setup-buildx-action@v1
+
+    - name: Login to Docker Hub
+      uses: docker/login-action@v1
+      with: 
+        username: ${{ secrets.DOCKER_HUB_USERNAME }}
+        password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
+
+    - name: Get and Set Version
       run: |
-        make docker-build
         source ci/set-docker-image-version.sh
-        echo ${DOCKER_IMG_VER}
-        docker run --rm -d --name starter-kit-common-test-default -p 4500:4500 -p 4501:4501 ga4gh/ga4gh-starter-kit-common:${DOCKER_IMG_VER}
-        docker run --rm -d --name starter-kit-common-test-custom -p 7000:7000 -p 7001:7001 ga4gh/ga4gh-starter-kit-common:${DOCKER_IMG_VER} --config ./src/test/resources/config/demo-config.yml
-    - name: Script
-      run: ./gradlew test --tests=integration.*
+        echo "version=${DOCKER_IMG_VER}" >> $GITHUB_ENV
+
+    - name: Build and push
+      uses: docker/build-push-action@v2
+      with:
+        context: .
+        builder: ${{ steps.buildx.outputs.name }} #didn't fix
+        file: ./Dockerfile
+        push: true #not pushing the docker image yet
+        tags: ga4gh/ga4gh-starter-kit-common:test
+        build-args: VERSION=${{ env.version }}
+        cache-from: type=gha #GitHub Actions Cache Exporter
+        cache-to: type=gha,mode=max
 
+    - name: Running Tests
+      run: |
+        docker run --rm -d --name starter-kit-common-test-default -p 4500:4500 -p 4501:4501 ga4gh/ga4gh-starter-kit-common:test
+        docker run --rm -d --name starter-kit-common-test-custom -p 7000:7000 -p 7001:7001 ga4gh/ga4gh-starter-kit-common:test --config ./src/test/resources/config/demo-config.yml
+    
+    - name: Gradle Integration Test
+      run: ./gradlew test --tests=integration.*
diff --git a/Dockerfile b/Dockerfile
@@ -9,9 +9,9 @@ USER root
 WORKDIR /usr/src/dependencies
 
 # INSTALL MAKE
-RUN apt update
-RUN apt install build-essential -y
-RUN apt install wget -y
+RUN apt update \
+    && apt install build-essential -y \
+    && apt install wget -y
 
 # INSTALL SQLITE3
 RUN wget https://www.sqlite.org/2021/sqlite-autoconf-3340100.tar.gz \
@@ -23,11 +23,26 @@ RUN wget https://www.sqlite.org/2021/sqlite-autoconf-3340100.tar.gz \
 
 # USER 'make' and 'sqlite3' to create the dev database
 COPY Makefile Makefile
-COPY settings.gradle settings.gradle
-COPY build.gradle build.gradle
 COPY database/sqlite database/sqlite
 RUN make sqlite-db-refresh
 
+##################################################
+# GRADLE CONTAINER
+##################################################
+
+FROM gradle:7.3.3-jdk11 as gradleimage
+
+WORKDIR /home/gradle/source
+
+COPY build.gradle build.gradle
+COPY gradlew gradlew
+COPY settings.gradle settings.gradle
+COPY src src
+
+RUN gradle wrapper
+
+RUN ./gradlew bootJar
+
 ##################################################
 # FINAL CONTAINER
 ##################################################
@@ -41,7 +56,7 @@ ARG VERSION
 WORKDIR /usr/src/app
 
 # copy jar, dev db, and dev resource files
-COPY build/libs/ga4gh-starter-kit-common-${VERSION}.jar ga4gh-starter-kit-common.jar
+COPY --from=gradleimage /home/gradle/source/build/libs/ga4gh-starter-kit-common-${VERSION}.jar ga4gh-starter-kit-common.jar
 COPY --from=builder /usr/src/dependencies/ga4gh-starter-kit.dev.db ga4gh-starter-kit.dev.db
 COPY src/test/resources/ src/test/resources/
 

diff --git a/gradle/wrapper/gradle-wrapper.jar b/gradle/wrapper/gradle-wrapper.jar
diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties
@@ -1,5 +1,5 @@
 distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-7.3.1-bin.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-7.3.3-bin.zip
 zipStoreBase=GRADLE_USER_HOME
 zipStorePath=wrapper/dists