initial refactoring to match SMaHT portal changes

.github/workflows/main.yml

@@ -1,4 +1,4 @@
-# portal-pipeline-utils GA Workflow
+# smaht-pipeline-utils GA Workflow
 
 name: CI
 
@@ -17,7 +17,7 @@ on:
 jobs:
   # This workflow contains a single job called "test"
   test:
-    name: Test portal-pipeline-utils with Python ${{ matrix.python_version }}
+    name: Test smaht-pipeline-utils with Python ${{ matrix.python_version }}
 
     # The type of runner that the job will run on
     runs-on: ubuntu-22.04
@@ -33,7 +33,7 @@ jobs:
         with:
           python-version: ${{ matrix.python_version }}
 
-      - name: Test portal-pipeline-utils
+      - name: Test smaht-pipeline-utils
         run: |
           make configure
           make build

README.md

@@ -2,21 +2,21 @@
 
 Utilities for deploying pipelines and interfacing with portal infrastructure.
 
-For more information on available commands and how to contribute and deploy pipelines within the infrastructure check the extended [*documentation*](https://portal-pipeline-utils.readthedocs.io/en/latest/ "portal-pipeline-utils documentation").
+For more information on available commands and how to contribute and deploy pipelines within the infrastructure check the extended [*documentation*](https://smaht-pipeline-utils.readthedocs.io/en/latest/ "smaht-pipeline-utils documentation").
 
 ## Install
 
 The software is python based. To install the software and the required packages, we recommend using a fresh virtual environment.
 Please refer to `pyproject.toml` for the supported Python versions.
 
-The package is available on [*pypi*](https://pypi.org/project/portal-pipeline-utils "portal-pipeline-utils pypi"):
+The package is available on [*pypi*](https://pypi.org/project/smaht-pipeline-utils "smaht-pipeline-utils pypi"):
 
-    pip install portal-pipeline-utils
+    pip install smaht-pipeline-utils
 
 To install from source:
 
-    git clone https://github.com/dbmi-bgm/portal-pipeline-utils.git
-    cd portal-pipeline-utils
+    git clone https://github.com/smaht-dac/smaht-pipeline-utils.git
+    cd smaht-pipeline-utils
     make configure
     make update
     make build

docs/conf.py

@@ -17,9 +17,9 @@
 
 # -- Project information -----------------------------------------------------
 
-project = 'portal-pipeline-utils'
+project = 'smaht-pipeline-utils'
 copyright = '2021, HMS DBMI'
-author = 'Michele Berselli, CGAP & SMaHT Team'
+author = 'Michele Berselli, SMaHT Team'
 
 # The full version, including alpha/beta/rc tags
 release = '2.1.0'

docs/contribute_pipeline.rst

@@ -177,4 +177,4 @@ Example
 Examples
 --------
 
-Real examples of implemented pipeline modules can be found linked as submodules in our main pipeline repository for the CGAP project here: https://github.com/dbmi-bgm/cgap-pipeline-main.
+Real examples of implemented pipeline modules can be found linked as submodules in our main pipeline repository for the SMaHT project here: https://github.com/smaht-dac/main-pipelines.

docs/deploy_pipeline.rst

@@ -55,37 +55,37 @@ by running a test command:
 More information on how to setup Docker can be found in the
 `AWS Documentation <https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html>`_.
 
-We now need to install the ``pipeline_utils`` software to deploy the pipeline components.
+We now need to install the ``smaht_pipeline_utils`` software to deploy the pipeline components.
 
-Install pipeline_utils
-======================
+Install smaht_pipeline_utils
+============================
 
 The software is Python-based.
 To install the software and the required packages, we recommend
 using a fresh virtual environment.
-Please refer to `pyproject.toml <https://github.com/dbmi-bgm/portal-pipeline-utils/blob/main/pyproject.toml>`_ for the supported Python version.
+Please refer to `pyproject.toml <https://github.com/smaht-dac/smaht-pipeline-utils/blob/main/pyproject.toml>`_ for the supported Python version.
 
 We recommend using pyenv to manage virtual environments.
 Instructions for installing and using
 pyenv can be found `here <https://realpython.com/intro-to-pyenv/>`_.
 
-Once the virtual environment is set up and activated, we can proceed to :ref:`install <install>` portal-pipeline-utils software.
+Once the virtual environment is set up and activated, we can proceed to :ref:`install <install>` smaht-pipeline-utils software.
 
 .. code-block:: bash
 
   # Install from source
-  git clone https://github.com/dbmi-bgm/portal-pipeline-utils.git
-  cd portal-pipeline-utils
+  git clone https://github.com/smaht-dac/smaht-pipeline-utils.git
+  cd smaht-pipeline-utils
   make configure
   make update
   make build
   cd ..
 
   # Install from pypi
-  pip install portal-pipeline-utils
+  pip install smaht-pipeline-utils
 
-To check that the software is correctly installed, try to run ``pipeline_utils``.
-If installed from source, this command may fail with a bash "command not found" error, try ``poetry run pipeline_utils`` instead.
+To check that the software is correctly installed, try to run ``smaht_pipeline_utils``.
+If installed from source, this command may fail with a bash "command not found" error, try ``poetry run smaht_pipeline_utils`` instead.
 
 Set Up Credentials and Environmental Variables
 ==============================================
@@ -188,7 +188,7 @@ by the ``--repos`` argument.
 
 .. code-block:: bash
 
-  pipeline_utils pipeline_deploy \
+  smaht_pipeline_utils pipeline_deploy \
     --ff-env ${ENV_NAME} \
     --keydicts-json ${KEYDICTS_JSON} \
     --wfl-bucket ${WFL_BUCKET} \
@@ -214,18 +214,18 @@ The default is set to the ``main`` branch. The ``--local-build`` flag will preve
 
 *Note: we are working to enable more builders with a command line argument for which builder to use to deploy modules from different repositories through AWS CodeBuild.*
 
-Deploying CGAP Pipelines
+Deploying SMaHT Pipelines
 ========================
 
-CGAP pipelines are released as a complete package with a customized set up for automated deployment to the desired environment.
+SMaHT pipelines are released as a complete package with a customized set up for automated deployment to the desired environment.
 To deploy the pipelines run the following steps:
 
 1. Clone the main pipeline repository.
 The submodules will be empty and set to the current commits saved for the main branch.
 
 .. code-block:: bash
 
-  git clone https://github.com/dbmi-bgm/cgap-pipeline-main.git
+  git clone https://github.com/smaht-dac/main-pipelines.git
 
 2. Check out the desired version.
 This will set the submodules to the commits saved for that pipeline release.
@@ -241,7 +241,7 @@ The submodules will be set in detached state on their current commit.
 
   make pull
 
-4. Build pipeline_utils (optional).
+4. Build smaht_pipeline_utils (optional).
 This will build from source the latest version linked for the current release.
 
 .. code-block:: bash

docs/index.rst

@@ -2,9 +2,9 @@
 Portal Pipeline Utilities
 =========================
 
-Documentation for portal-pipeline-utils_, a collection of utilities for deploying pipelines and interfacing with portal infrastructure.
+Documentation for smaht-pipeline-utils_, a collection of utilities for deploying pipelines and interfacing with portal infrastructure.
 
-.. _portal-pipeline-utils: https://github.com/dbmi-bgm/portal-pipeline-utils
+.. _smaht-pipeline-utils: https://github.com/smaht-dac/smaht-pipeline-utils
 
 Contents
 ########

docs/install.rst

@@ -9,11 +9,11 @@ PyPI
 
 The package is available on pypi_:
 
-.. _pypi: https://pypi.org/project/portal-pipeline-utils
+.. _pypi: https://pypi.org/project/smaht-pipeline-utils
 
 .. code-block:: bash
 
-    pip install portal-pipeline-utils
+    pip install smaht-pipeline-utils
 
 Source
 ^^^^^^
@@ -23,10 +23,10 @@ To install the latest version from source:
 
 .. code-block:: bash
 
-    git clone https://github.com/dbmi-bgm/portal-pipeline-utils.git
-    cd portal-pipeline-utils
+    git clone https://github.com/smaht-dac/smaht-pipeline-utils.git
+    cd smaht-pipeline-utils
     make configure
     make update
     make build
 
-Please refer to `pyproject.toml <https://github.com/dbmi-bgm/portal-pipeline-utils/blob/main/pyproject.toml>`_ for the supported Python version.
+Please refer to `pyproject.toml <https://github.com/smaht-dac/smaht-pipeline-utils/blob/main/pyproject.toml>`_ for the supported Python version.

docs/pipeline_utils.rst

@@ -1,6 +1,6 @@
-==============
-pipeline_utils
-==============
+====================
+smaht_pipeline_utils
+====================
 
 This is the entry point for a collection of utilities available as commands:
 
@@ -10,7 +10,7 @@ Usage:
 
 .. code-block:: bash
 
-    pipeline_utils [COMMAND] [ARGS]
+    smaht_pipeline_utils [COMMAND] [ARGS]
 
 .. _pipeline_deploy:
 
@@ -24,7 +24,7 @@ Usage:
 
 .. code-block:: bash
 
-    pipeline_utils pipeline_deploy --ff-env FF_ENV --repos REPO [REPO ...] [OPTIONAL ARGS]
+    smaht_pipeline_utils pipeline_deploy --ff-env FF_ENV --repos REPO [REPO ...] [OPTIONAL ARGS]
 
 **Arguments:**
 

docs/repo.rst

@@ -56,4 +56,4 @@ Example ``foo_bar`` pipeline:
     ├── PIPELINE
     └── VERSION
 
-Real examples can be found linked as submodules in our pipelines repository for CGAP project here: https://github.com/dbmi-bgm/cgap-pipeline-main.
+Real examples can be found linked as submodules in our pipelines repository for SMaHT project here: https://github.com/smaht-dac/main-pipelines.

docs/yaml_metaworkflow.rst

@@ -20,11 +20,6 @@ Template
     name: <string>
     description: <string>
 
-    # All the following fields are optional and provided as example,
-    #   can be expanded to anything accepted by the schema
-    #   https://github.com/dbmi-bgm/cgap-portal/tree/master/src/encoded/schemas
-    proband_only: <boolean>
-
     ## General arguments ########################################
     #     Pipeline input, reference files, and general arguments
     #       define all arguments for the pipeline here

pipeline_utils/__main__.py

@@ -20,8 +20,8 @@
 
 # Variables
 PIPELINE_DEPLOY = 'pipeline_deploy'
-CGAP_ALIAS = 'cgap-core'
-DBMI_ALIAS = 'hms-dbmi'
+CONSORTIA_ALIAS = 'CONSORTIUM'
+SUBMISSION_CENTERS_ALIAS = 'SUBMISSION_CENTER'
 KEYS_ALIAS = '~/.cgap-keys.json'
 MAIN_ALIAS = 'main'
 BUILDER_ALIAS = '<ff-env>-pipeline-builder'
@@ -32,7 +32,7 @@ def main(args=None):
     '''Command line wrapper around available commands.
     '''
     # Adding parser and subparsers
-    parser = argparse.ArgumentParser(prog='pipeline_utils', description='Collection of utilities for deploying pipelines and interfacing with portal infrastructure')
+    parser = argparse.ArgumentParser(prog='smaht_pipeline_utils', description='Collection of utilities for deploying pipelines and interfacing with portal infrastructure')
     subparsers = parser.add_subparsers(dest='func', metavar="<command>")
 
     # Add pipeline_deploy to subparsers
@@ -50,10 +50,10 @@ def main(args=None):
     pipeline_deploy_parser.add_argument('--wfl-bucket', required=False, help='Bucket to use for upload of Workflow Description files')
     pipeline_deploy_parser.add_argument('--account', required=False, help='AWS account to use for deployment')
     pipeline_deploy_parser.add_argument('--region', required=False, help='AWS account region to use for deployment')
-    pipeline_deploy_parser.add_argument('--project', required=False, help=f'Project to use for deployment [{CGAP_ALIAS}]',
-                                                          default=CGAP_ALIAS)
-    pipeline_deploy_parser.add_argument('--institution', required=False, help=f'Institution to use for deployment [{DBMI_ALIAS}]',
-                                                              default=DBMI_ALIAS)
+    pipeline_deploy_parser.add_argument('--consortia', required=False, nargs='+', help='List of consortia to use for deployment',
+                                                       default=[CONSORTIA_ALIAS])
+    pipeline_deploy_parser.add_argument('--submission-centers', required=False, nargs='+', help='List of centers to use for deployment',
+                                                                default=[SUBMISSION_CENTERS_ALIAS])
 
     pipeline_deploy_parser.add_argument('--post-software', action='store_true', help='POST|PATCH Software objects')
     pipeline_deploy_parser.add_argument('--post-file-format', action='store_true', help='POST|PATCH FileFormat objects')