Merge pull request #5 from nf-core/dev

Push for first release!

.github/CONTRIBUTING.md

@@ -42,4 +42,4 @@ If there are any failures then the automated tests fail.
 These tests are run both with the latest available version of Nextflow and also the minimum required version that is stated in the pipeline code.
 
 ## Getting help
-For further information/help, please consult the [nf-core/nascent documentation](https://github.com/nf-core/nascent#documentation) and don't hesitate to get in touch on [Gitter](https://gitter.im/nf-core/Lobby)
+For further information/help, please consult the [nf-core/nascent documentation](https://github.com/nf-core/nascent#documentation) and don't hesitate to get in touch on the pipeline channel on [Slack](https://nf-core-invite.herokuapp.com/).

.github/markdownlint.yml

@@ -0,0 +1,9 @@
+# Markdownlint configuration file
+default: true,
+line-length: false
+no-multiple-blanks: 0
+blanks-around-headers: false
+blanks-around-lists: false
+header-increment: false
+no-duplicate-header:
+    siblings_only: true

.gitignore

@@ -4,3 +4,4 @@ data/
 results/
 .DS_Store
 tests/test_data
+*.pyc

.travis.yml

@@ -11,20 +11,23 @@ before_install:
   # PRs to master are only ok if coming from dev branch
   - '[ $TRAVIS_PULL_REQUEST = "false" ] || [ $TRAVIS_BRANCH != "master" ] || ([ $TRAVIS_PULL_REQUEST_SLUG = $TRAVIS_REPO_SLUG ] && [ $TRAVIS_PULL_REQUEST_BRANCH = "dev" ])'
   # Pull the docker image first so the test doesn't wait for this
-  - docker pull ignaciot/nascent
+  - docker pull nfcore/nascent:dev
   # Fake the tag locally so that the pipeline runs properly
-  - docker tag ignaciot/nascent ignaciot/nascent:latest
+  # Looks weird when this is :dev to :dev, but makes sense when testing code for a release (:dev to :1.0.1)
+  - docker tag nfcore/nascent:dev nfcore/nascent:1.0
 
 install:
   # Install Nextflow
   - mkdir /tmp/nextflow && cd /tmp/nextflow
   - wget -qO- get.nextflow.io | bash
   - sudo ln -s /tmp/nextflow/nextflow /usr/local/bin/nextflow
   # Install nf-core/tools
-  - pip install --upgrade pip 
+  - pip install --upgrade pip
   - pip install nf-core
   # Reset
   - mkdir ${TRAVIS_BUILD_DIR}/tests && cd ${TRAVIS_BUILD_DIR}/tests
+  # Install markdownlint-cli
+  - sudo apt-get install npm && npm install -g markdownlint-cli
 
 env:
   - NXF_VER='0.32.0' # Specify a minimum NF version that should be tested and work
@@ -33,5 +36,7 @@ env:
 script:
   # Lint the pipeline code
   - nf-core lint ${TRAVIS_BUILD_DIR}
+  # Lint the documentation
+  - markdownlint ${TRAVIS_BUILD_DIR} -c ${TRAVIS_BUILD_DIR}/.github/markdownlint.yml
   # Run the pipeline with the test profile
   - nextflow run ${TRAVIS_BUILD_DIR} -profile test,docker

CHANGELOG.md

@@ -1,4 +1,4 @@
 # nf-core/nascent: Changelog
 
-## v1.0dev - <date>
+## v1.0 - 2019-04-16
 Initial release of nf-core/nascent, created with the [nf-core](http://nf-co.re/) template.

CODE_OF_CONDUCT.md

@@ -34,7 +34,7 @@ This Code of Conduct applies both within project spaces and in public spaces whe
 
 ## Enforcement
 
-Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team on the [Gitter channel](https://gitter.im/nf-core/Lobby). The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team on [Slack](https://nf-core-invite.herokuapp.com/). The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
 
 Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
 

README.md

@@ -1,9 +1,9 @@
 # nf-core/nascent
-**Nascent Transcription Processing Pipeline**
 
-[![Build Status](https://travis-ci.org/nf-core/nascent.svg?branch=master)](https://travis-ci.org/nf-core/nascent)
-[![Nextflow](https://img.shields.io/badge/nextflow-%E2%89%A50.32.0-brightgreen.svg)](https://www.nextflow.io/)
+**Nascent Transcription Processing Pipeline**.
 
+[![Build Status](https://travis-ci.com/nf-core/nascent.svg?branch=master)](https://travis-ci.com/nf-core/nascent)
+[![Nextflow](https://img.shields.io/badge/nextflow-%E2%89%A50.32.0-brightgreen.svg)](https://www.nextflow.io/)
 [![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg)](http://bioconda.github.io/)
 [![Docker](https://img.shields.io/docker/automated/nfcore/nascent.svg)](https://hub.docker.com/r/nfcore/nascent)
 ![Singularity Container available](
@@ -13,18 +13,70 @@ https://img.shields.io/badge/singularity-available-7E4C74.svg)
 The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker / singularity containers making installation trivial and results highly reproducible.
 
 
+#### Reference
+
+If you've used this pipeline in your research, you can cite this pipeline using DOI 10.17605/OSF.IO/SV4UB ([OSF project](https://osf.io/sv4ub/)).
+
 ### Documentation
 The nf-core/nascent pipeline comes with documentation about the pipeline, found in the `docs/` directory:
-
-1. [Installation](docs/installation.md)
+1. [Installation](https://nf-co.re/usage/installation)
 2. Pipeline configuration
-    * [Local installation](docs/configuration/local.md)
-    * [Adding your own system](docs/configuration/adding_your_own.md)
+    * [Local installation](https://nf-co.re/usage/local_installation)
+    * [Adding your own system config](https://nf-co.re/usage/adding_own_config)
+    * [Reference genomes](https://nf-co.re/usage/reference_genomes)
 3. [Running the pipeline](docs/usage.md)
 4. [Output and how to interpret the results](docs/output.md)
-5. [Troubleshooting](docs/troubleshooting.md)
+5. [Troubleshooting](https://nf-co.re/usage/troubleshooting)
+
+This pipeline is designed to process the sequencing output of nascent transcription assays, like GRO-seq or PRO-seq. It produces bedGraph- and bigWig-fomatted outputs after mapping strand-specific reads, as well as other useful outputs like quality control reports or IGV-ready (Integrative Genomics Viewer) tdf files.
+
+### Quick start
+
+Edit the appropriate config file, e.g. `conf/slurm_grch38.config`, to ensure the proper paths are set for genome reference files and other executables (look for all mentions of `COMPLETE_*`). Variable names should hopefully be self-explanatory. You can specify the Nextflow working directory and output directory with flags. Note you must also now specify the email to which the report will be sent for the run.
+
+    nextflow run nf-core/nascent --reads '*_R{1,2}.fastq.gz' -profile standard,docker
+
+## Arguments
+
+### Required Arguments
+| Argument  | Usage                            | Description                                                          |
+|-----------|----------------------------------|----------------------------------------------------------------------|
+| -profile  | \<base,slurm\>                    | Configuration profile to use.                                       |
+| --fastqs  | \</project/\*\_{R1,R2}\*.fastq\> | Directory pattern for fastq files.                                   |
+| --sras    | \</project/\*.sra\>              | Directory pattern for sra files.                                     |
+| --genome_id | \<'hg38'>                      | Genome ID to which the samples will be mapped (e.g. hg38, mm10, rn6).|
+| --workdir | \</project/tmp/\>                | Nextflow working directory where all intermediate files are saved.   |
+| --email   | \<EMAIL\>                        | Where to send workflow report email.                                 |
+
+### Save Options
+| Arguments  | Usage         | Description                                               |
+|------------|---------------|-----------------------------------------------------------|
+| --outdir   | \</project/\> | Specifies where to save the output from the nextflow run. |
+| --savefq   |               | Compresses and saves raw fastq reads.                     |
+| --saveTrim |               | Compresses and saves trimmed fastq reads.                 |
+| --saveAll  |               | Compresses and saves all fastq reads.                     |
+| --skipBAM  |               | Skips saving BAM files (only save CRAM). Default=False    |
+
+### Input File Options
+| Arguments    | Usage       | Description                                                                  |
+|--------------|-------------|------------------------------------------------------------------------------|
+| --singleEnd  |             | Specifies that the input files are not paired reads (default is paired-end). |
+| --flip       |             | Reverse complements each strand. Necessary for some library preps.           |
+
+### Performance Options
+
+| Arguments       | Usage       | Description                                             |
+|-----------------|-------------|---------------------------------------------------------|
+| --threadfqdump  |             | Runs multi-threading for fastq-dump for sra processing. |
+
+### QC Options
+
+| Arguments       | Usage       | Description                                             |
+|-----------------|-------------|---------------------------------------------------------|
+| --skipMultiQC   |             | Skip running MultiQC.                                   |
+| --skipRSeQC     |             | Skip running RSeQC.                                     |
 
-<!-- TODO nf-core: Add a brief overview of what the pipeline does and how it works -->
+## Credits
+nf-core/nascent was originally written by Ignacio Tripodi ([@ignaciot](https://github.com/ignaciot)) and Margaret Gruca ([@magruca](https://github.com/magruca)).
 
-### Credits
-nf-core/nascent was originally written by Ignacio Tripodi, Margaret Gruca.
+Many thanks to the nf-core team and all who provided invaluable feedback and assistance along the way, particularly to [@apeltzer](https://github.com/apeltzer), [@ewels](https://github.com/ewels), [@drpatelh](https://github.com/drpatelh), and [@pditommaso](https://github.com/pditommaso).

assets/email_template.txt

@@ -17,23 +17,6 @@ ${errorReport}
 } %>
 
 
-<% if (!success){
-    out << """####################################################
-## nf-core/nascent execution completed unsuccessfully! ##
-####################################################
-The exit status of the task that caused the workflow execution to fail was: $exitStatus.
-The full error message was:
-
-${errorReport}
-"""
-} else {
-    out << "## nf-core/nascent execution completed successfully! ##"
-}
-%>
-
-
-
-
 The workflow was completed at $dateComplete (duration: $duration)
 
 The command used to launch the workflow was as follows:

assets/multiqc_config.yaml

@@ -0,0 +1,9 @@
+report_comment: >
+    This report has been generated by the <a href="https://github.com/nf-core/nascent" target="_blank">nf-core/nascent</a>
+    analysis pipeline. For information about how to interpret these results, please see the
+    <a href="https://github.com/nf-core/nascent" target="_blank">documentation</a>.
+report_section_order:
+    nf-core/nascent-software-versions:
+        order: -1000
+
+export_plots: true

assets/sendmail_template.txt

@@ -1,11 +1,36 @@
 To: $email
 Subject: $subject
 Mime-Version: 1.0
-Content-Type: multipart/related;boundary="nfmimeboundary"
+Content-Type: multipart/related;boundary="nfcoremimeboundary"
 
---nfmimeboundary
+--nfcoremimeboundary
 Content-Type: text/html; charset=utf-8
 
 $email_html
 
---nfmimeboundary--
+<%
+if (mqcFile){
+def mqcFileObj = new File("$mqcFile")
+if (mqcFileObj.length() < mqcMaxSize){
+out << """
+--nfcoremimeboundary
+Content-Type: text/html; name=\"multiqc_report\"
+Content-Transfer-Encoding: base64
+Content-ID: <mqcreport>
+Content-Disposition: attachment; filename=\"${mqcFileObj.getName()}\"
+
+${mqcFileObj.
+  bytes.
+  encodeBase64().
+  toString().
+  tokenize( '\n' )*.
+  toList()*.
+  collate( 76 )*.
+  collect { it.join() }.
+  flatten().
+  join( '\n' )}
+"""
+}}
+%>
+
+--nfcoremimeboundary--

bin/scrape_software_versions.py

@@ -3,7 +3,6 @@
 from collections import OrderedDict
 import re
 
-# TODO nf-core: Add additional regexes for new tools in process get_software_versions
 regexes = {
     'nf-core/nascent': ['v_pipeline.txt', r"(\S+)"],
     'Nextflow': ['v_nextflow.txt', r"(\S+)"],
@@ -24,9 +23,14 @@
         if match:
             results[k] = "v{}".format(match.group(1))
 
+# Remove software set to false in results
+for k in results:
+    if not results[k]:
+        del(results[k])
+
 # Dump to YAML
 print ('''
-id: 'nf-core/nascent-software-versions'
+id: 'software_versions'
 section_name: 'nf-core/nascent Software Versions'
 section_href: 'https://github.com/nf-core/nascent'
 plot_type: 'html'
@@ -35,5 +39,10 @@
     <dl class="dl-horizontal">
 ''')
 for k,v in results.items():
-    print("        <dt>{}</dt><dd>{}</dd>".format(k,v))
+    print("        <dt>{}</dt><dd><samp>{}</samp></dd>".format(k,v))
 print ("    </dl>")
+
+# Write out regexes as csv file:
+with open('software_versions.csv', 'w') as f:
+    for k,v in results.items():
+        f.write("{}\t{}\n".format(k,v))

conf/awsbatch.config

@@ -1,10 +1,15 @@
 /*
  * -------------------------------------------------
- *  Nextflow config file for AWS Batch
+ *  Nextflow config file for running on AWS batch
  * -------------------------------------------------
- * Imported under the 'awsbatch' Nextflow profile in nextflow.config
- * Uses docker for software depedencies automagically, so not specified here.
+ * Base config needed for running with -profile awsbatch
  */
+params {
+  config_profile_name = 'AWSBATCH'
+  config_profile_description = 'AWSBATCH Cloud Profile'
+  config_profile_contact = 'Alexander Peltzer (@apeltzer)'
+  config_profile_url = 'https://aws.amazon.com/de/batch/'
+}
 
 aws.region = params.awsregion
 process.executor = 'awsbatch'

conf/base.config

@@ -11,14 +11,11 @@
 
 process {
 
-  container = params.container
-
-  // TODO nf-core: Check the defaults for all processes
   cpus = { check_max( 2 * task.attempt, 'cpus' ) }
   memory = { check_max( 8.GB * task.attempt, 'memory' ) }
   time = { check_max( 2.h * task.attempt, 'time' ) }
 
-  errorStrategy = { task.exitStatus in [143,137] ? 'retry' : 'finish' }
+  errorStrategy = { task.exitStatus in [143,137,104,134,139] ? 'retry' : 'finish' }
   maxRetries = 1
   maxErrors = '-1'
 

conf/igenomes.config

@@ -9,7 +9,6 @@
 
 params {
   // illumina iGenomes reference file paths
-  // TODO nf-core: Add new reference types and strip out those that are not needed
   genomes {
     'GRCh37' {
       bed12   = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed"