Skip to content

Commit

Permalink
Merge pull request #233 from qbic-pipelines/dev
Browse files Browse the repository at this point in the history
Release 2.3
  • Loading branch information
WackerO authored Jun 13, 2024
2 parents f5e54b6 + acd0dbd commit 062b03a
Show file tree
Hide file tree
Showing 78 changed files with 2,697 additions and 1,421 deletions.
11 changes: 2 additions & 9 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,15 @@
"name": "nfcore",
"image": "nfcore/gitpod:latest",
"remoteUser": "gitpod",
"runArgs": ["--privileged"],

// Configure tool-specific properties.
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Set *default* container specific settings.json values on container create.
"settings": {
"python.defaultInterpreterPath": "/opt/conda/bin/python",
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.formatting.autopep8Path": "/opt/conda/bin/autopep8",
"python.formatting.yapfPath": "/opt/conda/bin/yapf",
"python.linting.flake8Path": "/opt/conda/bin/flake8",
"python.linting.pycodestylePath": "/opt/conda/bin/pycodestyle",
"python.linting.pydocstylePath": "/opt/conda/bin/pydocstyle",
"python.linting.pylintPath": "/opt/conda/bin/pylint"
"python.defaultInterpreterPath": "/opt/conda/bin/python"
},

// Add the IDs of extensions you want installed when the container is created.
Expand Down
11 changes: 10 additions & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,16 @@ end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
indent_style = unset
indent_size = unset
[/subworkflows/nf-core/**]
charset = unset
end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
indent_style = unset

[/assets/email*]
indent_size = unset

# ignore python and markdown
[*.{py,md}]
indent_style = unset
2 changes: 1 addition & 1 deletion .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Once there, use `nf-core schema build` to add to `nextflow_schema.json`.

Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.

The process resources can be passed on to the tool dynamically within the process with the `${task.cpu}` and `${task.memory}` variables in the `script:` block.
The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block.

### Naming schemes

Expand Down
4 changes: 4 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
contact_links:
- name: Join nf-core
url: https://nf-co.re/join
about: Please join the nf-core community here
17 changes: 7 additions & 10 deletions .github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,13 @@ on:
types: [published]
workflow_dispatch:
jobs:
run-tower:
run-platform:
name: Run AWS full tests
if: github.repository == 'nf-core/rnadeseq'
if: github.repository == 'qbic-pipelines/rnadeseq'
runs-on: ubuntu-latest
steps:
- name: Launch workflow via tower
- name: Launch workflow via Seqera Platform
uses: seqeralabs/action-tower-launch@v2
# TODO nf-core: You can customise AWS full pipeline tests as required
# Add full size test data (but still relatively small datasets for few samples)
# on the `test_full.config` test runs with only one set of parameters
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
Expand All @@ -31,9 +28,9 @@ jobs:
}
profiles: test_full

- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: Tower debug log file
name: Seqera Platform debug log file
path: |
tower_action_*.log
tower_action_*.json
seqera_platform_action_*.log
seqera_platform_action_*.json
16 changes: 8 additions & 8 deletions .github/workflows/awstest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ name: nf-core AWS test
on:
workflow_dispatch:
jobs:
run-tower:
run-platform:
name: Run AWS tests
if: github.repository == 'nf-core/rnadeseq'
if: github.repository == 'qbic-pipelines/rnadeseq'
runs-on: ubuntu-latest
steps:
# Launch workflow using Tower CLI tool action
- name: Launch workflow via tower
# Launch workflow using Seqera Platform CLI tool action
- name: Launch workflow via Seqera Platform
uses: seqeralabs/action-tower-launch@v2
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
Expand All @@ -25,9 +25,9 @@ jobs:
}
profiles: test

- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: Tower debug log file
name: Seqera Platform debug log file
path: |
tower_action_*.log
tower_action_*.json
seqera_platform_action_*.log
seqera_platform_action_*.json
6 changes: 3 additions & 3 deletions .github/workflows/branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ jobs:
run: |
"{ [[ ${{github.event.pull_request.head.repo.full_name }} == qbic-pipelines/rnadeseq ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]"
# If the above check failed, post a comment on the PR explaining the failure
# If the above check failed, post a comment on the PR explaining the failure {%- raw %}
# NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
with:
message: |
## This PR is against the `master` branch :x:
Expand All @@ -41,4 +41,4 @@ jobs:
Thanks again for your contribution!
repo-token: ${{ secrets.GITHUB_TOKEN }}
allow-repeats: false
allow-repeats: false {%- endraw %}
9 changes: 4 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ on:

env:
NXF_ANSI_LOG: false
CAPSULE_LOG: none

concurrency:
group: "${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}"
Expand Down Expand Up @@ -40,14 +39,14 @@ jobs:
environment.yml
- name: Build new docker image
if: env.MATCHED_FILES
run: docker build --no-cache . -t ghcr.io/qbic-pipelines/rnadeseq:2.2
run: docker build --no-cache . -t ghcr.io/qbic-pipelines/rnadeseq:2.3

# Change the version above and the third version below before/after release
- name: Pull docker image
if: ${{ !env.MATCHED_FILES }}
run: |
docker pull ghcr.io/qbic-pipelines/rnadeseq:dev
docker tag ghcr.io/qbic-pipelines/rnadeseq:dev ghcr.io/qbic-pipelines/rnadeseq:2.2
docker tag ghcr.io/qbic-pipelines/rnadeseq:dev ghcr.io/qbic-pipelines/rnadeseq:2.3
- name: Install Nextflow
uses: nf-core/setup-nextflow@v1
Expand Down Expand Up @@ -94,14 +93,14 @@ jobs:
environment.yml
- name: Build new docker image
if: env.MATCHED_FILES
run: docker build --no-cache . -t ghcr.io/qbic-pipelines/rnadeseq:2.2
run: docker build --no-cache . -t ghcr.io/qbic-pipelines/rnadeseq:2.3

# Change the version above and the third version below before/after release
- name: Pull docker image
if: ${{ !env.MATCHED_FILES }}
run: |
docker pull ghcr.io/qbic-pipelines/rnadeseq:dev
docker tag ghcr.io/qbic-pipelines/rnadeseq:dev ghcr.io/qbic-pipelines/rnadeseq:2.2
docker tag ghcr.io/qbic-pipelines/rnadeseq:dev ghcr.io/qbic-pipelines/rnadeseq:2.3
- name: Install Nextflow
uses: nf-core/setup-nextflow@v1
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/clean-up.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v7
- uses: actions/stale@28ca1036281a5e5922ead5184a1bbf96e5fc984e # v9
with:
stale-issue-message: "This issue has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment otherwise this issue will be closed in 20 days."
stale-pr-message: "This PR has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment if it is still useful."
Expand Down
86 changes: 86 additions & 0 deletions .github/workflows/download_pipeline.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
name: Test successful pipeline download with 'nf-core download'

# Run the workflow when:
# - dispatched manually
# - when a PR is opened or reopened to master branch
# - the head branch of the pull request is updated, i.e. if fixes for a release are pushed last minute to dev.
on:
workflow_dispatch:
inputs:
testbranch:
description: "The specific branch you wish to utilize for the test execution of nf-core download."
required: true
default: "dev"
pull_request:
types:
- opened
- edited
- synchronize
branches:
- master
pull_request_target:
branches:
- master

env:
NXF_ANSI_LOG: false

jobs:
download:
runs-on: ubuntu-latest
steps:
- name: Install Nextflow
uses: nf-core/setup-nextflow@v2

- name: Disk space cleanup
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1

- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
with:
python-version: "3.12"
architecture: "x64"
- uses: eWaterCycle/setup-singularity@931d4e31109e875b13309ae1d07c70ca8fbc8537 # v7
with:
singularity-version: 3.8.3

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install git+https://github.com/nf-core/tools.git@dev
- name: Get the repository name and current branch set as environment variable
run: |
echo "REPO_LOWERCASE=${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
echo "REPOTITLE_LOWERCASE=$(basename ${GITHUB_REPOSITORY,,})" >> ${GITHUB_ENV}
echo "REPO_BRANCH=${{ github.event.inputs.testbranch || 'dev' }}" >> ${GITHUB_ENV}
- name: Download the pipeline
env:
NXF_SINGULARITY_CACHEDIR: ./
run: |
nf-core download ${{ env.REPO_LOWERCASE }} \
--revision ${{ env.REPO_BRANCH }} \
--outdir ./${{ env.REPOTITLE_LOWERCASE }} \
--compress "none" \
--container-system 'singularity' \
--container-library "quay.io" -l "docker.io" -l "ghcr.io" \
--container-cache-utilisation 'amend' \
--download-configuration
- name: Inspect download
run: tree ./${{ env.REPOTITLE_LOWERCASE }}

- name: Run the downloaded pipeline (stub)
id: stub_run_pipeline
continue-on-error: true
env:
NXF_SINGULARITY_CACHEDIR: ./
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -stub -profile test,singularity --outdir ./results
- name: Run the downloaded pipeline (stub run not supported)
id: run_pipeline
if: ${{ job.steps.stub_run_pipeline.status == failure() }}
env:
NXF_SINGULARITY_CACHEDIR: ./
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -profile test,singularity --outdir ./results
74 changes: 54 additions & 20 deletions .github/workflows/fix-linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,52 +4,86 @@ on:
types: [created]

jobs:
deploy:
fix-linting:
# Only run if comment is on a PR with the main repo, and if it contains the magic keywords
if: >
contains(github.event.comment.html_url, '/pull/') &&
contains(github.event.comment.body, '@nf-core-bot fix linting') &&
github.repository == 'nf-core/rnadeseq'
github.repository == 'qbic-pipelines/rnadeseq'
runs-on: ubuntu-latest
steps:
# Use the @nf-core-bot token to check out so we can push later
- uses: actions/checkout@v3
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
with:
token: ${{ secrets.nf_core_bot_auth_token }}

# indication that the linting is being fixed
- name: React on comment
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
with:
comment-id: ${{ github.event.comment.id }}
reactions: eyes

# Action runs on the issue comment, so we don't get the PR by default
# Use the gh cli to check out the PR
- name: Checkout Pull Request
run: gh pr checkout ${{ github.event.issue.number }}
env:
GITHUB_TOKEN: ${{ secrets.nf_core_bot_auth_token }}

- uses: actions/setup-node@v3
# Install and run pre-commit
- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
with:
python-version: "3.12"

- name: Install Prettier
run: npm install -g prettier @prettier/plugin-php
- name: Install pre-commit
run: pip install pre-commit

# Check that we actually need to fix something
- name: Run 'prettier --check'
id: prettier_status
run: |
if prettier --check ${GITHUB_WORKSPACE}; then
echo "result=pass" >> $GITHUB_OUTPUT
else
echo "result=fail" >> $GITHUB_OUTPUT
fi
- name: Run pre-commit
id: pre-commit
run: pre-commit run --all-files
continue-on-error: true

- name: Run 'prettier --write'
if: steps.prettier_status.outputs.result == 'fail'
run: prettier --write ${GITHUB_WORKSPACE}
# indication that the linting has finished
- name: react if linting finished succesfully
if: steps.pre-commit.outcome == 'success'
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
with:
comment-id: ${{ github.event.comment.id }}
reactions: "+1"

- name: Commit & push changes
if: steps.prettier_status.outputs.result == 'fail'
id: commit-and-push
if: steps.pre-commit.outcome == 'failure'
run: |
git config user.email "[email protected]"
git config user.name "nf-core-bot"
git config push.default upstream
git add .
git status
git commit -m "[automated] Fix linting with Prettier"
git commit -m "[automated] Fix code linting"
git push
- name: react if linting errors were fixed
id: react-if-fixed
if: steps.commit-and-push.outcome == 'success'
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
with:
comment-id: ${{ github.event.comment.id }}
reactions: hooray

- name: react if linting errors were not fixed
if: steps.commit-and-push.outcome == 'failure'
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
with:
comment-id: ${{ github.event.comment.id }}
reactions: confused

- name: react if linting errors were not fixed
if: steps.commit-and-push.outcome == 'failure'
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
with:
issue-number: ${{ github.event.issue.number }}
body: |
@${{ github.actor }} I tried to fix the linting errors, but it didn't work. Please fix them manually.
See [CI log](https://github.com/qbic-pipelines/rnadeseq/actions/runs/${{ github.run_id }}) for more details.
Loading

0 comments on commit 062b03a

Please sign in to comment.