Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add compatibility with arches 7.6 #18

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
[run]
source =
arches_templating/

omit =
*/python?.?/*
*/models/migrations/*
*/settings*.py
*/urls.py
*/wsgi.py
*/celery.py
*/__init__.py

data_file = coverage/python/.coverage

[report]
show_missing = true

exclude_lines =
pragma: no cover

[html]
directory = coverage/python/htmlcov

[xml]
output = coverage/python/coverage.xml

[json]
output = coverage/python/coverage.json
11 changes: 11 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto

# Known binary formats
*.png binary
*.jpg binary
*.ico binary
*.gif binary
*.ttf binary
*.woff binary
*.woff2 binary
99 changes: 99 additions & 0 deletions .github/actions/build-and-test-branch/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
name: 'Build and test branch'
description: 'Builds and tests a branch'
inputs:
branch-type:
description: 'String denoting either `target` or `feature` branch'
required: true
project-name:
description: 'String denoting the name of the project'
required: true
secrets:
description: 'Secrets from main.yml as JSON'
runs:
using: 'composite'
steps:
- name: Install Java, GDAL, and other system dependencies
run: |
sudo apt update
sudo apt-get install libxml2-dev libpq-dev openjdk-8-jdk libgdal-dev libxslt-dev
echo Postgres and ES dependencies installed
shell: bash

- name: Set up Elasticsearch
uses: ankane/setup-elasticsearch@v1
with:
elasticsearch-version: 8

- name: Install Python packages
run: |
python -m pip install --upgrade pip
pip install '.[dev]'
echo Python packages installed
shell: bash

- name: Install Arches applications
uses: ./.github/actions/install-arches-applications
with:
secrets: ${{ inputs.secrets }}

- name: Checkout into feature branch
if: inputs.branch-type == 'feature'
uses: actions/checkout@v4
with:
repository: ${{ github.repository }}
ref: ${{ github.ref }}
path: .

- name: Checkout into target branch
if: inputs.branch-type == 'target'
uses: actions/checkout@v4
with:
repository: ${{ github.repository }}
ref: ${{ github.event.pull_request.base.ref }}
path: .

- name: Check backend formatting with black
run: |
black . --check --exclude=node_modules
shell: bash

- name: Check line endings
run: |
! git ls-files --eol | grep 'w/crlf\|w/mixed'
shell: bash

- name: Check for missing migrations
run: |
python manage.py makemigrations arches_templating --check --settings="arches_templating.tests.test_settings"
shell: bash

- name: Ensure previous Python coverage data is erased
run: |
coverage erase
shell: bash

- name: Run Python unit tests
run: |
python -W default::DeprecationWarning -m coverage run manage.py test arches_templating.tests --settings="arches_templating.tests.test_settings"
shell: bash

- name: Generate Python report coverage
run: |
coverage report
coverage json
mv coverage/python/coverage.json ${{ inputs.branch-type }}_branch_python_coverage.json
shell: bash

- name: Upload frontend coverage report as artifact
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.branch-type }}-branch-frontend-coverage-report
path: ${{ inputs.branch-type }}_branch_frontend_coverage.xml
overwrite: true

- name: Upload Python coverage report as artifact
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.branch-type }}-branch-python-coverage-report
path: ${{ inputs.branch-type }}_branch_python_coverage.json
overwrite: true
29 changes: 29 additions & 0 deletions .github/actions/install-arches-applications/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: 'Install Arches Applications'
description: 'Manually edit this file to install all Arches Applications declared in settings.py, but not declared in `pyproject.toml`'
inputs:
secrets:
description: 'Secrets from main.yml as JSON'
runs:
using: 'composite'
steps:

# Manually add any ARCHES_APPLICATIONS to this file if not already declared in `pyproject.toml`.
# Below is a template for adding an application in a private repository.
# Be sure to delete the `no-op step` if adding when updating this file.

- name: No-op step to maintain workflow structure
run: echo "No-op step"
shell: bash

# - name: Checkout ${my_arches_application_name}
# uses: actions/checkout@v4
# with:
# repository: ${my_arches_application_repository}/${my_arches_application_name}
# token: ${{ fromJSON(inputs.secrets).${my_github_personal_access_token} }}
# path: ${my_arches_application_name}

# - name: Install ${my_arches_application_name}
# run: |
# pip install ./${my_arches_application_name}
# echo ${my_arches_application_name} installed
# shell: bash
113 changes: 113 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
name: CI

on:
# push: -- just run on PRs for now
pull_request:
workflow_dispatch:

jobs:
build_feature_branch:
runs-on: ubuntu-latest

services:
postgres:
image: postgis/postgis:13-3.0
env:
POSTGRES_PASSWORD: postgis
POSTGRES_DB: arches_templating
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
check-latest: true

- name: Build and test branch
uses: ./.github/actions/build-and-test-branch
with:
secrets: ${{ toJSON(secrets) }}
project-name: 'arches_templating'
branch-type: 'feature'

build_target_branch:
runs-on: ubuntu-latest

services:
postgres:
image: postgis/postgis:13-3.0
env:
POSTGRES_PASSWORD: postgis
POSTGRES_DB: arches_templating
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

strategy:
fail-fast: false
matrix:
python-version: ["3.12"]

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
check-latest: true

- name: Build and test branch
uses: ./.github/actions/build-and-test-branch
with:
secrets: ${{ toJSON(secrets) }}
project-name: 'arches_templating'
branch-type: 'target'

check_python_coverage:
runs-on: ubuntu-latest
needs: [build_feature_branch, build_target_branch]
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x' # Use the latest available version
check-latest: true

- name: Download feature branch Python coverage report artifact
uses: actions/download-artifact@v4
with:
name: feature-branch-python-coverage-report
path: .

- name: Download target branch Python coverage report artifact
uses: actions/download-artifact@v4
with:
name: target-branch-python-coverage-report
path: .

- name: Compare Python feature coverage with target coverage
if: github.event_name == 'pull_request'
run: |
feature_branch_python_coverage=$(cat feature_branch_python_coverage.json | grep -o '"totals": {[^}]*' | grep -o '"percent_covered": [0-9.]*' | awk -F ': ' '{print $2}')
target_branch_python_coverage=$(cat target_branch_python_coverage.json | grep -o '"totals": {[^}]*' | grep -o '"percent_covered": [0-9.]*' | awk -F ': ' '{print $2}')

# Compare feature coverage with target coverage using floating-point comparison
if awk -v feature="$feature_branch_python_coverage" -v target="$target_branch_python_coverage" 'BEGIN { exit (feature < target) ? 0 : 1 }'; then
echo "Coverage decreased from $target_branch_python_coverage% to $feature_branch_python_coverage%. Please add or update tests to increase coverage."
exit 1
else
echo "Feature branch coverage ($feature_branch_python_coverage%) >= Target branch coverage ($target_branch_python_coverage%)."
fi
21 changes: 17 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,17 @@
ENV/
arches_templating.egg-info
build/
__pycache__
*.pyc
*.log
node_modules
*.coverage
arches_templating/logs
arches_templating/export_deliverables
arches_templating/cantaloupe/*
arches_templating/staticfiles
arches_templating/media/packages
arches_templating/media/build/
arches_templating/uploadedfiles/*
arches_templating/settings_local.py
webpack-stats.json
.vscode/
*.egg-info
.DS_STORE
CACHE
9 changes: 9 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
repos:

- repo: https://github.com/psf/black
rev: 24.4.2
hooks:
- id: black
args: [--quiet]
exclude: node_modules
Loading
Loading