Migrate from Travis to GitHub Actions for CI #1534

aliciapaz · 2021-08-24T01:07:01Z

Based on #1491 and publiclab/spectral-workbench#647

Jobs to perform in CI/CD Pipeline based on Travis.yml:

welcome · 2021-08-24T01:07:03Z

Thanks for opening this pull request! Dangerbot will test out your code and reply in a bit with some pointers and requests.
There may be some errors, but don't worry! We're here to help! 👍🎉😄

aliciapaz · 2021-08-24T01:36:45Z

Mysql2::Error::ConnectionError: Access denied for user 'runner'@'localhost' (using password: NO)

codeclimate · 2021-08-24T01:57:48Z

Code Climate has analyzed commit d83a53d and detected 0 issues on this pull request.

View more on Code Climate.

codecov · 2021-08-24T02:12:44Z

Codecov Report

Merging #1534 (2a496a9) into main (4331ef0) will decrease coverage by 0.64%.
The diff coverage is 61.53%.

@@            Coverage Diff             @@
##             main    #1534      +/-   ##
==========================================
- Coverage   72.98%   72.33%   -0.65%     
==========================================
  Files          40       40              
  Lines        1399     1406       +7     
==========================================
- Hits         1021     1017       -4     
- Misses        378      389      +11

Impacted Files	Coverage Δ
app/channels/concurrent_editing_channel.rb	`0.00% <0.00%> (-83.34%)`	⬇️
app/helpers/users_helper.rb	`21.73% <0.00%> (ø)`
app/models/map.rb	`92.25% <ø> (-1.30%)`	⬇️
app/helpers/application_helper.rb	`50.00% <20.00%> (ø)`
app/controllers/annotations_controller.rb	`59.25% <22.22%> (ø)`
app/controllers/sessions_controller.rb	`32.91% <22.22%> (ø)`
app/controllers/front_ui_controller.rb	`74.28% <33.33%> (ø)`
app/controllers/application_controller.rb	`88.88% <50.00%> (ø)`
app/controllers/images_controller.rb	`76.54% <61.90%> (ø)`
app/controllers/maps_controller.rb	`86.95% <70.00%> (ø)`
... and 14 more

aliciapaz · 2021-08-24T02:35:20Z

Ok, at least tests are running now, but with errors:

Minitest::UnexpectedError:         ArgumentError: You are passing an instance of ActiveRecord::Base to `find`. Please pass the id of the object by calling `.id`.
          app/channels/application_cable/connection.rb:12:in `find_verified_user'
          app/channels/application_cable/connection.rb:6:in `connect'
          test/channels/connection_test.rb:9:in `test_connection_with_user'

In unit tests:

FAIL ExporterTest#test_should_export_warpable_using_isolated_exporter_lib (7.71s)
       Expected false to be truthy.
       test/models/exporter_test.rb:38:in `block in <class:ExporterTest>'

In system tests:

Minitest::UnexpectedError:         Selenium::WebDriver::Error::UnknownError: unknown error: Chrome failed to start: exited abnormally.
       (unknown error: DevToolsActivePort file doesn't exist)
       (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
         test/system/searches_test.rb:5:in `block in <class:SearchesTest>'

aliciapaz · 2021-08-25T04:34:32Z

Hhhmmm I'm getting this error despite this gem not being inherited anywhere 🤔

Unable to find gem rubocop-discourse; is the gem installed? Gem::MissingSpecError

aliciapaz · 2021-08-25T04:47:58Z

Now I'm seeing:

Error: Ambiguous cop name `Style/VariableName` used in /home/runner/work/mapknitter/mapknitter/vendor/bundle/ruby/2.4.0/gems/geokit-1.13.1/.rubocop.yml needs department qualifier. Did you mean Naming/VariableName or RSpec/VariableName?

Weird...I can't see anything like "Style/VariableName" inside .rubocop.yml, nor inside the files from which it inherits (.rubocop_shopify_styleguide.yml and .rubocop_todo.yml)

aliciapaz · 2021-08-31T03:07:53Z

.rubocop.yml

@@ -15,7 +15,7 @@ AllCops:
    - '/Rakefile'
    - '/config.ru'
  Exclude:
-    - 'vendor/*'
+    - 'vendor/**/*'


This line did the trick!

Except that I know have errors in code climate 🌦️

oof wow that was obscure! Glad you're figuring it out though!

Hmm, i see:

We had trouble running the rubocop engine.
The engine's output shown below may indicate the cause of the error.
If not, please contact us so we can investigate further.

.rubocop.yml: `require` is concealed by duplicate /usr/local/bundle/gems/rubocop-0.70.0/lib/rubocop/config_loader_resolver.rb:196:in `rescue in gem_config_path': Unable to find gem rubocop-shopify; is the gem installed? Gem::MissingSpecError (Gem::LoadError) from /usr/local/bundle/gems/rubocop-0.70.0/lib/rubocop/config_loader_resolver.rb:192:in `gem_config_path' from /usr/local/bundle/gems/rubocop-0.70.0/lib/rubocop/config_loader_resolver.rb:48:in `block (2 levels) in resolve_inheritance_from_gems' from /usr/local/bundle/gems/rubocop-0.70.0/lib/rubocop/config_loader_resolver.rb:46:in `reverse_each'

Isn't CodeClimate just running rubocop? So maybe we just need one or the other. Rubocop's job ran fine!

32 files inspected, no offenses detected

Gosh surprising that this is a lot more complex seeming than our other Rails repos. I guess the setup is relatively different?

You're right, I removed the rubocop plugin from codeclimate as it is in GH Actions now.

aliciapaz · 2021-09-13T01:51:57Z

I get this error in step 6 of development dockerfile:

RUN apt-get update -qq && apt-get install -y \
  nodejs gdal-bin curl procps git imagemagick python-gdal zip
W: GPG error: http://packages.laboratoriopublico.org/publiclab stretch InRelease: The following signatures were invalid: EXPKEYSIG BF26EE05EA6A68F0 Public Lab (Tycho) <[email protected]>

E: There were unauthenticated packages and -y was used without --allow-unauthenticated

aliciapaz · 2021-09-13T15:05:46Z

@jywarren any idea on how to go about the codecov failures?

Apart from that, all the other jobs are successful 🎉

Thanks!!

jywarren · 2021-09-14T20:06:03Z

OK!!! So i think we could ignore the patch codecov because it's just saying that only 61% of your newly added/changed code is covered, which makes sense because you haven't really touched much testable code. The project one, well, we dropped below the 1% change (delta) threshold, which is supposed to ensure no PR lowers coverage by more than 1%. Again i think we can bypass that because it's clear we're not adding code that's really covered by a test suite.

So I'm going to change the job names a little bit for style/friendliness and to match plots2 if I can, but otherwise this looks good!

jywarren · 2021-09-14T20:16:35Z

app/channels/concurrent_editing_channel.rb

@@ -3,7 +3,7 @@ class ConcurrentEditingChannel < ApplicationCable::Channel

  def subscribed
    # Called first to connect user to the channel.
-    stream_from "concurrent_editing_channel:#{params[:mapSlug]}"
+    stream_from("concurrent_editing_channel:#{params[:mapSlug]}")


Were these changes from Rubocop telling you how it should be formatted? I think that's outside the scope of this PR ideally... it's fine that it's included but I'm sorry I should have caught this and made it clear you don't have to fix everyone else's syntax! Thank you though!!!

Yep, the reason for those changes is that while I was setting up the Rubocop job I ran into lots of errors as you can see by the commit history... In my attempts to make this work, I changed the setup of the linter a bit (like adding the style guide as a gem instead of requiring it as a static file). After I made those changes, I had to run Rubocop again so it wouldn't fail due to linter offenses.

jywarren · 2021-09-14T20:18:30Z

development

@@ -51,6 +51,7 @@ RUN git config --global url."https://".insteadOf git://
 COPY . /app/
 WORKDIR /app

+RUN gem install bundler


jywarren · 2021-09-14T20:18:39Z

production

@@ -41,6 +41,6 @@ WORKDIR /app
 RUN apt-get clean && \
    apt-get autoremove -y

-RUN bundle install
+RUN gem install bundler && bundle install


So, is it possible to move these development and production dockerfiles into a folder called dockerfiles or something? They're just a little mysterious sitting in the root folder. I don't recall we have them this way in plots2- but for comparison i think we have docker compose files in this directory:

https://github.com/publiclab/plots2/tree/main/containers/

I made some code suggestions for dockerfile/development etc below which you can accept, then we just need to move the files themselves. Almost there!!!

.github/workflows/tests.yml

jywarren · 2021-09-14T20:24:21Z

.github/workflows/tests.yml

+          name: system-test-screenshots
+          path: tmp/screenshots/*
+
+  docker-development:


Can we rename this? What is its role? It looks good but even familiar with MapKnitter and docker, it's not super clear what the purpose of this job is, so i wonder if we could explain it a bit using the name? Thanks @aliciapaz!!!

Hmm, could it be development-docker-build and production-docker-build?
From what I understand, these jobs ensure that changes made to the source code do not break those docker builds. Does that make sense?
Thank you @jywarren!

jywarren · 2021-09-14T20:24:33Z

.github/workflows/tests.yml

+          bundler-cache: true
+      - uses: ./.github/actions/setup-test-environment
+      - name: 'Development Docker Build'
+        run: docker build -t mapknitter -f production .


Suggested change

run: docker build -t mapknitter -f production .

run: docker build -t mapknitter -f dockerfiles/production .

Co-authored-by: Jeffrey Warren <[email protected]>

jywarren · 2021-09-14T22:14:47Z

Awesome. Even "docker-build-check-development"?

…

On Tue, Sep 14, 2021, 5:50 PM Alicia Paz ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In .github/workflows/tests.yml <#1534 (comment)>: > + RAILS_ENV: test + DB_PASSWORD: root + DB_PORT: ${{ job.services.mysql.ports[3306] }} + REDIS_HOST: localhost + REDIS_PORT: 6379 + run: | + export DISPLAY=:99 + chromedriver --url-base=/wd/hub & + bundle exec rails test test:system + - name: Archive system test screenshots + uses: ***@***.*** + with: + name: system-test-screenshots + path: tmp/screenshots/* + + docker-development: Hmm, could it be development-docker-build and production-docker-build? From what I understand, these jobs ensure that changes made to the source code do not break those docker builds. Does that make sense? Thank you @jywarren <https://github.com/jywarren>! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1534 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAF6J64XFZHJF77PPX3Q63UB67SZANCNFSM5CVYHSVA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

jywarren · 2021-09-20T22:50:40Z

Wooohoo!!!!!! Great work @aliciapaz !!!!!!

Try migrating from Travis to GitHub Actions for CI

65ad6f0

Update actions.yml and tests.yml

d83a53d

aliciapaz added 2 commits August 23, 2021 22:01

Add ruby version

e681180

Update config/database.yml.example

1e59560

aliciapaz added 4 commits August 24, 2021 23:53

Try rubocop and docker jobs

6650c8d

Fix indentation

b6d31d6

Update tests.yml

f922988

Update tests.yml

b4510df

Update gemfile

116f082

aliciapaz mentioned this pull request Aug 25, 2021

Convert all publiclab repositories' automated tests/checks from TravisCI to GitHub Actions publiclab/plots2#9641

Open

36 tasks

aliciapaz added 11 commits August 30, 2021 20:54

Update rubocop job and Gemfile

b73093c

Update .rubocop_shopify_styleguide.yml

9ac7c92

Update gemfile

10b4b10

Update .rubocop_todo.yml

00761d5

Update rubocop configuration and styleguides

e6a8f30

Fix rubocop offenses

0bd0169

Update .rubocop.yml

ff1e1e5

Update .rubocop.yml

998b294

Udate .codeclimate.yml

dd3d6cd

Update .codeclimate.yml

b2533ee

Update .codeclimate.yml

2373b44

aliciapaz commented Aug 31, 2021

View reviewed changes

Update .codeclimate.yml

33e96a0

aliciapaz added 2 commits September 12, 2021 22:27

Fix identifiers

ab16ea7

Fix path to action.yml

b3c36a7

aliciapaz added 9 commits September 13, 2021 10:25

Update development dockerfile

bf44440

Setup docker production build job

c157cec

Update tests.yml

c9e913b

Setup assets precompilation job

7892471

Setup production environment

9f069ab

Update tests.yml

38f3d36

Update action.yml for production

1af307c

Try with test setup

31d3d08

Remove action.yml for production

d6a0287

jywarren reviewed Sep 14, 2021

View reviewed changes

.github/workflows/tests.yml Outdated Show resolved Hide resolved

jywarren reviewed Sep 14, 2021

View reviewed changes

.github/workflows/tests.yml Outdated Show resolved Hide resolved

jywarren reviewed Sep 14, 2021

View reviewed changes

.github/workflows/tests.yml Outdated Show resolved Hide resolved

jywarren reviewed Sep 14, 2021

View reviewed changes

jywarren and others added 3 commits September 14, 2021 16:24

Change names of jobs for friendliness and clarity

8e6243a

Update .github/workflows/tests.yml

3858156

Co-authored-by: Jeffrey Warren <[email protected]>

Move dockerfiles to directory and change docker job names for clarity

d6e3711

Update docker job names

2a496a9

jywarren approved these changes Sep 20, 2021

View reviewed changes

jywarren merged commit d2d45db into main Sep 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate from Travis to GitHub Actions for CI #1534

Migrate from Travis to GitHub Actions for CI #1534

aliciapaz commented Aug 24, 2021 •

edited

Loading

welcome bot commented Aug 24, 2021

aliciapaz commented Aug 24, 2021

codeclimate bot commented Aug 24, 2021

codecov bot commented Aug 24, 2021 •

edited

Loading

aliciapaz commented Aug 24, 2021

aliciapaz commented Aug 25, 2021

aliciapaz commented Aug 25, 2021

aliciapaz Aug 31, 2021

aliciapaz Aug 31, 2021

jywarren Aug 31, 2021

jywarren Aug 31, 2021

jywarren Aug 31, 2021

aliciapaz Sep 1, 2021

aliciapaz commented Sep 13, 2021

aliciapaz commented Sep 13, 2021

jywarren commented Sep 14, 2021

jywarren Sep 14, 2021

aliciapaz Sep 14, 2021

jywarren Sep 14, 2021

jywarren Sep 14, 2021

jywarren Sep 14, 2021

jywarren Sep 14, 2021

jywarren Sep 14, 2021

aliciapaz Sep 14, 2021

jywarren Sep 14, 2021

jywarren commented Sep 14, 2021 via email

jywarren commented Sep 20, 2021

	run: docker build -t mapknitter -f production .
	run: docker build -t mapknitter -f dockerfiles/production .

Migrate from Travis to GitHub Actions for CI #1534

Migrate from Travis to GitHub Actions for CI #1534

Conversation

aliciapaz commented Aug 24, 2021 • edited Loading

welcome bot commented Aug 24, 2021

aliciapaz commented Aug 24, 2021

codeclimate bot commented Aug 24, 2021

codecov bot commented Aug 24, 2021 • edited Loading

Codecov Report

aliciapaz commented Aug 24, 2021

aliciapaz commented Aug 25, 2021

aliciapaz commented Aug 25, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aliciapaz commented Sep 13, 2021

aliciapaz commented Sep 13, 2021

jywarren commented Sep 14, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jywarren commented Sep 14, 2021 via email

jywarren commented Sep 20, 2021

aliciapaz commented Aug 24, 2021 •

edited

Loading

codecov bot commented Aug 24, 2021 •

edited

Loading