Release v2.8.0

docs/upgrade-guides/2-7-1-to-2-8-0.md

@@ -0,0 +1,131 @@
+# 2.7.1 to 2.8.0
+
+This document describes the upgrade instructions from `2.7.1` to `2.8.0`.
+
+Aerie v2.8.0 contains significant changes to our PostgreSQL database structure, along with changes to our required environment variables and configuration. It therefore requires a different upgrade process than usual, especially if you plan to migrate your data forward to v2.8.0. Make sure to read all instructions below carefully before continuing, and please open a Github issue or ask for help in our Slack room if you have any questions.
+
+
+## For Aerie End-Users
+(ie. those running Aerie from the released `Deployment.zip` or the `deployment` folder in the repository)
+
+### Changes to docker-compose configuration
+Several changes have been made to the variables used in our `docker-compose.yml` config file. If you are using the supplied config as-is, you don't need to do anything - but if you are using **modified version of this file** for your own deployment, you'll need to make some updates. You can either:
+
+* Take a fresh copy of our `docker-compose.yml` file and recreate any modifications you made
+* Or see the full detailed list of changes to the config variables at the bottom of this document, and manually apply these changes to your own version of the file.
+
+
+## For Aerie Developers
+(ie. anyone running Aerie from a cloned version of the `aerie` repository)
+
+* Run `./gradlew clean` **before** checking out `develop` or `v2.8.0` in order to clean up old SQL files.
+  - Or if you have already checked it out, run `rm -rf deployment/postgres-init-db/sql && git restore deployment/postgres-init-db/sql`
+* Several new environment variables are now required. Compare the new version of `env.template` to your existing `.env` file and copy over any new variables which are missing in your `.env`.
+  - existing values, especially AERIE_USERNAME, AERIE_PASSWORD, POSTGRES_USER and POSTGRES_PASSWORD, must remain unchanged from your previous version
+  - You must fill in (arbitrary) values for the password variables, which are left blank in the template
+* If you plan migrate your data, follow migration instructions below before running Aerie v2.8.0 for the first time
+* When running v2.8.0 for the first time, make sure to rebuild the backend, and build/pull all containers, ie.:
+  - `docker compose down --remove-orphans && ./gradlew assemble && docker compose up --pull always -d --build`
+
+
+## For Anyone Migrating Data to v2.8.0
+These instructions apply if you wish to save your data and **migrate it forward** to work with v2.8.0, whether you're a user or a developer.
+
+### Migration Prerequisites
+* Your database must already be up-to-date with the migrations from the prior release, 2.7.1. If you are upgrading from an earlier version, you should upgrade and migrate to v2.7.1 **first** before continuing.
+* You should have already followed one set of instructions above to modify either your `docker-compose` file or your `.env` file.
+* Make sure you have the `psql` command installed and you can run it from your shell. If not:
+  - Mac users can run `brew install libpq`
+  - For other platforms, [download the Postgres installer](https://www.postgresql.org/download), and uncheck "PostgreSQL Server" when installing
+* From the deployment folder, run `python -m pip install -r requirements.txt` to make sure you have all necessary Python packages installed
+* If you have ever changed the default names for the Aerie databases (`aerie_merlin`, `aerie_scheduler`, `aerie_sequencing`, `aerie_ui`) - this is no longer allowed, and you **must** rename them to their default names.
+
+### Migration Process
+1. Starting from the deployment directory, run `cd merge_aerie_db`
+2. Run `./merge_db.sh -h` to list the available command line flags and see if any apply to your deployment - ie. if you are using non-default config file locations or connecting to a non-local database
+3. Run `./merge_db.sh`, adding any necessary command line flags, to start the migration
+4. As a precaution, your **old** databases have been preserved at this point. If the last step failed for any reason, you can safely drop the `aerie` database, troubleshoot the problem and re-run the `merge_db.sh` script
+5. After the migration has run successfully, run `drop_old_dbs.sh` to drop your old databases.
+
+Aerie developers should also rebuild their backend following the migration (`./gradlew assemble`) - not required for end-users.
+
+Your data should now be fully migrated and ready to run Aerie v2.8.0! Note that, due to breaking changes to the mission model framework in v2.7.0, you may need to **recompile and re-upload** your mission models if you are upgrading from a version **before v2.7.0**.
+
+
+## Appendix
+### Full list of `docker-compose.yml` variable changes
+Below are all of the changes to the variables in our `docker-compose.yml` file introduced in v2.8.0, if you plan to modify your version of this file manually:
+
+- aerie_gateway:
+  - removed envvars:
+    - `POSTGRES_AERIE_MERLIN_DB`
+    - `POSTGRES_USER`
+    - `POSTGRES_PASSWORD`
+  - renamed envvars:
+    - `POSTGRES_HOST` -> `AERIE_DB_HOST`
+    - `POSTGRES_PORT` -> `AERIE_DB_PORT`
+  - added envvars:
+    - `GATEWAY_DB_USER: "${GATEWAY_USERNAME}"`
+    - `GATEWAY_DB_PASSWORD: "${GATEWAY_PASSWORD}"`
+- aerie_merlin:
+  - removed envvars:  `MERLIN_DB`
+  - renamed envvars:
+    - `MERLIN_DB_SERVER` -> `AERIE_DB_HOST`
+    - `MERLIN_DB_PORT` -> `AERIE_DB_PORT`
+  - `MERLIN_DB_USER` should now be mapped at the envvar `"${MERLIN_USERNAME}"`
+  - `MERLIN_DB_PASSWORD` should now be mapped at the envvar `"${MERLIN_PASSWORD}"`
+- aerie_merlin_workers: (for each worker)
+  - removed envvars:
+    - `MERLIN_WORKER_DB`
+    - `MERLIN_WORKER_DB_USER`
+    - `MERLIN_WORKER_DB_PASSWORD`
+  - renamed envvars:
+    - `MERLIN_WORKER_DB_SERVER` -> `AERIE_DB_HOST`
+    - `MERLIN_WORKER_DB_PORT` -> `AERIE_DB_PORT`
+  - added envvars:
+    - `MERLIN_DB_USER: "${MERLIN_USERNAME}"`
+    - `MERLIN_DB_PASSWORD: "${MERLIN_PASSWORD}"`
+- aerie_scheduler:
+  - removed envvars:  `SCHEDULER_DB`
+  - renamed envvars:
+    - `SCHEDULER_DB_SERVER` -> `AERIE_DB_HOST`
+    - `SCHEDULER_DB_PORT` -> `AERIE_DB_PORT`
+  - `SCHEDULER_DB_USER` should now be mapped at the envvar `"${SCHEDULER_USERNAME}"`
+  - `SCHEDULER_DB_PASSWORD` should now be mapped at the envvar `"${SCHEDULER_PASSWORD}"`
+- aerie_scheduler_workers: (for each worker)
+  - removed envvars:
+    - `SCHEDULER_WORKER_DB`
+    - `SCHEDULER_WORKER_DB_USER`
+    - `SCHEDULER_WORKER_DB_PASSWORD`
+  - renamed envvars:
+    - `SCHEDULER_WORKER_DB_SERVER` -> `AERIE_DB_HOST`
+    - `SCHEDULER_WORKER_DB_PORT` -> `AERIE_DB_PORT`
+  - added envvars:
+    - `SCHEDULER_DB_USER: "${SCHEDULER_USERNAME}"`
+    - `SCHEDULER_DB_PASSWORD: "${SCHEDULER_PASSWORD}"`
+- aerie_sequencing:
+  - removed envvars:  `SEQUENCING_DB`
+  - renamed envvars:
+    - `SEQUENCING_DB_SERVER` -> `AERIE_DB_HOST`
+    - `SEQUENCING_DB_PORT` -> `AERIE_DB_PORT`
+  - `SEQUENCING_DB_USER` should now be mapped at the envvar `"${SEQUENCING_USERNAME}"`
+  - `SEQUENCING_DB_PASSWORD` should now be mapped at the envvar `"${SEQUENCING_PASSWORD}"`
+- hasura:
+  - removed envvars:
+    - `AERIE_MERLIN_DATABASE_URL`
+    - `AERIE_SCHEDULER_DATABASE_URL`
+    - `AERIE_SEQUENCING_DATABASE_URL`
+    - `AERIE_UI_DATABASE_URL`
+  - added envvars:
+    - `AERIE_DATABASE_URL: "postgres://${AERIE_USERNAME}:${AERIE_PASSWORD}@postgres:5432/aerie?options=-c%20search_path%3Dutil_functions%2Chasura%2Cpermissions%2Ctags%2Cmerlin%2Cscheduler%2Csequencing%2Cpublic"`
+- postgres:
+  - added envvars:
+    - `GATEWAY_DB_USER: "${GATEWAY_USERNAME}"`
+    - `GATEWAY_DB_PASSWORD: "${GATEWAY_PASSWORD}"`
+    - `MERLIN_DB_USER: "${MERLIN_USERNAME}"`
+    - `MERLIN_DB_PASSWORD: "${MERLIN_PASSWORD}"`
+    - `SCHEDULER_DB_USER: "${SCHEDULER_USERNAME}"`
+    - `SCHEDULER_DB_PASSWORD: "${SCHEDULER_PASSWORD}"`
+    - `SEQUENCING_DB_USER: "${SEQUENCING_USERNAME}"`
+    - `SEQUENCING_DB_PASSWORD: "${SEQUENCING_PASSWORD}"`
+

sidebars.js

@@ -324,6 +324,7 @@ const sidebars = {
     'glossary',
   ],
   upgradeGuides: [
+    'upgrade-guides/2-7-1-to-2-8-0',
     'upgrade-guides/2-7-0-to-2-7-1',
     'upgrade-guides/2-6-1-to-2-7-0',
     'upgrade-guides/2-6-0-to-2-6-1',