Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate from Travis to GitHub Actions for CI #1534

Merged
merged 50 commits into from
Sep 20, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
65ad6f0
Try migrating from Travis to GitHub Actions for CI
aliciapaz Aug 20, 2021
d83a53d
Update actions.yml and tests.yml
aliciapaz Aug 24, 2021
e681180
Add ruby version
aliciapaz Aug 24, 2021
1e59560
Update config/database.yml.example
aliciapaz Aug 24, 2021
6650c8d
Try rubocop and docker jobs
aliciapaz Aug 25, 2021
b6d31d6
Fix indentation
aliciapaz Aug 25, 2021
f922988
Update tests.yml
aliciapaz Aug 25, 2021
b4510df
Update tests.yml
aliciapaz Aug 25, 2021
116f082
Update gemfile
aliciapaz Aug 25, 2021
b73093c
Update rubocop job and Gemfile
aliciapaz Aug 31, 2021
9ac7c92
Update .rubocop_shopify_styleguide.yml
aliciapaz Aug 31, 2021
10b4b10
Update gemfile
aliciapaz Aug 31, 2021
00761d5
Update .rubocop_todo.yml
aliciapaz Aug 31, 2021
e6a8f30
Update rubocop configuration and styleguides
aliciapaz Aug 31, 2021
0bd0169
Fix rubocop offenses
aliciapaz Aug 31, 2021
ff1e1e5
Update .rubocop.yml
aliciapaz Aug 31, 2021
998b294
Update .rubocop.yml
aliciapaz Aug 31, 2021
dd3d6cd
Udate .codeclimate.yml
aliciapaz Aug 31, 2021
b2533ee
Update .codeclimate.yml
aliciapaz Aug 31, 2021
2373b44
Update .codeclimate.yml
aliciapaz Aug 31, 2021
33e96a0
Update .codeclimate.yml
aliciapaz Sep 1, 2021
4b76acc
Setup unit tests
aliciapaz Sep 1, 2021
39dce52
Update find_verified_user method in connection.rb
aliciapaz Sep 1, 2021
10cb4cd
Add yarn cache and install to setup
aliciapaz Sep 1, 2021
8eb9af7
install exporter dependencies; gdal/imagemagick
jywarren Sep 2, 2021
ce1621e
Introduce Gitpod to migration work (#1538)
aliciapaz Sep 3, 2021
0f35f7b
Merge branch 'main' into github_actions
jywarren Sep 3, 2021
5ff5712
Install gdal and use egordm for yarn cache
aliciapaz Sep 12, 2021
bd3fab8
Delete unnecessary query and fix layout
aliciapaz Sep 12, 2021
9a28b51
Run script with verbose output
aliciapaz Sep 12, 2021
7daffb1
Try installing package in action.yml
aliciapaz Sep 12, 2021
6698c28
Setup controllers job
aliciapaz Sep 12, 2021
2739b30
Fix indentation
aliciapaz Sep 12, 2021
d7df220
Setup system tests job
aliciapaz Sep 13, 2021
2e79956
Setup docker development build job
aliciapaz Sep 13, 2021
ab16ea7
Fix identifiers
aliciapaz Sep 13, 2021
b3c36a7
Fix path to action.yml
aliciapaz Sep 13, 2021
bf44440
Update development dockerfile
aliciapaz Sep 13, 2021
c157cec
Setup docker production build job
aliciapaz Sep 13, 2021
c9e913b
Update tests.yml
aliciapaz Sep 13, 2021
7892471
Setup assets precompilation job
aliciapaz Sep 13, 2021
9f069ab
Setup production environment
aliciapaz Sep 13, 2021
38f3d36
Update tests.yml
aliciapaz Sep 13, 2021
1af307c
Update action.yml for production
aliciapaz Sep 13, 2021
31d3d08
Try with test setup
aliciapaz Sep 13, 2021
d6a0287
Remove action.yml for production
aliciapaz Sep 13, 2021
8e6243a
Change names of jobs for friendliness and clarity
jywarren Sep 14, 2021
3858156
Update .github/workflows/tests.yml
aliciapaz Sep 14, 2021
d6e3711
Move dockerfiles to directory and change docker job names for clarity
aliciapaz Sep 14, 2021
2a496a9
Update docker job names
aliciapaz Sep 14, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .codeclimate.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
version: 2
plugins:
rubocop:
enabled: true
channel: rubocop-0-70
brakeman:
enabled: true
bundler-audit:
Expand Down
31 changes: 31 additions & 0 deletions .github/actions/setup-test-environment/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: 'Set up test environment'
description: 'Set up test environment for mapknitter'

services:
mysql:
image: mysql:5.7
env:
MYSQL_ROOT_PASSWORD: root
ports:
- 3306
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
runs:
using: 'composite'
steps:
- name: Setup database
shell: bash
env:
RAILS_ENV: test
DB_USER: root
DB_PASS: root
# tell Rails to use proper port for MySQL
DB_PORT: ${{ job.services.mysql.ports[3306] }}
DISABLE_BOOTSNAP: true
run: |
cp config/database.yml.example config/database.yml
cp db/schema.rb.example db/schema.rb
cp config/config.yml.example config/config.yml
sudo systemctl start mysql
mysql -uroot -proot -e "CREATE DATABASE mapknitter_test;"
bundle exec rake db:schema:load db:migrate --trace
sudo apt-get install -y gdal-bin
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, apparently, only this package is needed to run unit and integration tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's right - i think we do unit and integration tests which run map exporting processes, but we didn't bother to do that for other kinds of tests which are more focused on application logic rather than these big back-end processes.

166 changes: 166 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
name: CI/CD workflow
on: [pull_request]

jobs:
rubocop:
name: Code style suggestions
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Prepare Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: 2.4.6
bundler-cache: true
- name: Rubocop report
env:
FORCE_COLOR: 1
run: bundle exec rubocop --color --fail-fast

unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.4.6
bundler-cache: true
- uses: ./.github/actions/setup-test-environment
- uses: egordm/gha-yarn-node-cache@v1
- name: Install packages
run: yarn install
- name: 'Model Tests'
env:
RAILS_ENV: test
DB_PASSWORD: root
DB_PORT: ${{ job.services.mysql.ports[3306] }}
DISABLE_BOOTSNAP: true
run: |
bundle exec rails test test:models

integration-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.4.6
bundler-cache: true
- uses: ./.github/actions/setup-test-environment
- uses: egordm/gha-yarn-node-cache@v1
- name: Install JavaScript dependencies with Yarn
run: yarn check || yarn install;
- name: 'Integration Tests'
env:
RAILS_ENV: test
DB_PASSWORD: root
DB_PORT: ${{ job.services.mysql.ports[3306] }}
DISABLE_BOOTSNAP: true
run: bundle exec rails test test:integration

controller-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.4.6
bundler-cache: true
- uses: ./.github/actions/setup-test-environment
- uses: egordm/gha-yarn-node-cache@v1
- name: Install JavaScript dependencies with Yarn
run: yarn check || yarn install;
- name: 'Controller Tests'
env:
RAILS_ENV: test
DB_PASSWORD: root
DB_PORT: ${{ job.services.mysql.ports[3306] }}
DISABLE_BOOTSNAP: true
run: bundle exec rails test test:controllers

system-tests:
runs-on: ubuntu-latest
services:
redis:
image: redis
# Set health checks to wait until redis has started
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
# Maps port 6379 on service container to the host
- 6379:6379
steps:
- uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.4.6
bundler-cache: true
- uses: nanasess/[email protected]
- uses: ./.github/actions/setup-test-environment
- uses: egordm/gha-yarn-node-cache@v1
- name: Install JavaScript dependencies with Yarn
run: yarn check || yarn install;
- name: 'System Tests'
env:
RAILS_ENV: test
DB_PASSWORD: root
DB_PORT: ${{ job.services.mysql.ports[3306] }}
REDIS_HOST: localhost
REDIS_PORT: 6379
run: |
export DISPLAY=:99
chromedriver --url-base=/wd/hub &
bundle exec rails test test:system
- name: Archive system test screenshots
uses: actions/upload-artifact@v2
with:
name: system-test-screenshots
path: tmp/screenshots/*

docker-build-check-development:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.4.6
bundler-cache: true
- uses: ./.github/actions/setup-test-environment
- name: 'Development Docker Build'
run: docker build -t mapknitter -f dockerfiles/development .

docker-build-check-production:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.4.6
bundler-cache: true
- uses: ./.github/actions/setup-test-environment
- name: 'Development Docker Build'
run: docker build -t mapknitter -f dockerfiles/production .

assets-precompilation:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.4.6
bundler-cache: true
- uses: ./.github/actions/setup-test-environment
- uses: egordm/gha-yarn-node-cache@v1
- name: Install JavaScript dependencies with Yarn
run: yarn check || yarn install;
- name: 'Asset Precompilation'
env:
RAILS_ENV: production
DB_PASSWORD: root
DB_PORT: ${{ job.services.mysql.ports[3306] }}
DISABLE_BOOTSNAP: true
run: bundle exec rails assets:precompile
7 changes: 5 additions & 2 deletions .rubocop.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
require: rubocop-rails
require: rubocop-performance

# Start with Spotifys style guide as a base then customize from there
inherit_from:
- .rubocop_shopify_styleguide.yml
- .rubocop_todo.yml

inherit_gem:
rubocop-shopify: rubocop.yml

# Apply rule to all cops
AllCops:
Include:
- '*/**/*.rb'
- '/Rakefile'
- '/config.ru'
Exclude:
- 'vendor/*'
- 'vendor/**/*'
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line did the trick!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except that I know have errors in code climate 🌦️

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oof wow that was obscure! Glad you're figuring it out though!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, i see:

We had trouble running the rubocop engine.
The engine's output shown below may indicate the cause of the error.
If not, please contact us so we can investigate further.

.rubocop.yml: `require` is concealed by duplicate
/usr/local/bundle/gems/rubocop-0.70.0/lib/rubocop/config_loader_resolver.rb:196:in `rescue in gem_config_path': Unable to find gem rubocop-shopify; is the gem installed? Gem::MissingSpecError (Gem::LoadError)
	from /usr/local/bundle/gems/rubocop-0.70.0/lib/rubocop/config_loader_resolver.rb:192:in `gem_config_path'
	from /usr/local/bundle/gems/rubocop-0.70.0/lib/rubocop/config_loader_resolver.rb:48:in `block (2 levels) in resolve_inheritance_from_gems'
	from /usr/local/bundle/gems/rubocop-0.70.0/lib/rubocop/config_loader_resolver.rb:46:in `reverse_each'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't CodeClimate just running rubocop? So maybe we just need one or the other. Rubocop's job ran fine!

32 files inspected, no offenses detected

Gosh surprising that this is a lot more complex seeming than our other Rails repos. I guess the setup is relatively different?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I removed the rubocop plugin from codeclimate as it is in GH Actions now.

- 'node_modules/**/*'
- 'spec/**/*'
- 'bin/*'
Expand Down
Loading