diff --git a/docs/upgrade-guides/2-7-1-to-2-8-0.md b/docs/upgrade-guides/2-7-1-to-2-8-0.md new file mode 100644 index 0000000..cfa20a9 --- /dev/null +++ b/docs/upgrade-guides/2-7-1-to-2-8-0.md @@ -0,0 +1,133 @@ +# 2.7.1 to 2.8.0 + +This document describes the upgrade instructions from `2.7.1` to `2.8.0`. + +Aerie v2.8.0 contains significant changes to our PostgreSQL database structure, along with changes to our required environment variables and configuration. It therefore requires a different upgrade process than usual, especially if you plan to migrate your data forward to v2.8.0. Make sure to read all instructions below carefully before continuing, and please open a Github issue or ask for help in our Slack room if you have any questions. + + +## For Aerie End-Users +(ie. those running Aerie from the released `Deployment.zip` or the `deployment` folder in the repository) + +### Changes to docker-compose configuration +Several changes have been made to the variables used in our `docker-compose.yml` config file. If you are using the supplied config as-is, you don't need to do anything - but if you are using a **modified version of this file** for your own deployment, you'll need to make some updates. You can either: + +* Take a fresh copy of our `docker-compose.yml` file and recreate any modifications you made +* Or see the [full detailed list of changes](#full-list-of-docker-composeyml-variable-changes) to the config variables in the appendix of this document, and manually apply these changes to your own version of the file. + +### Changes to .env file +Several new environment variables are now required. Compare the new version of the `.env` file in the deployment directory to your existing `.env` file and copy over any new variables which are missing in your `.env`. +- existing values, especially AERIE_USERNAME, AERIE_PASSWORD, POSTGRES_USER and POSTGRES_PASSWORD, **must** remain unchanged from your previous version if you plan to migrate your data +- You must fill in (arbitrary) values for the password variables, which are left blank in the released `.env` file + +If you plan to migrate your data, follow the [migration instructions below](#for-anyone-migrating-data-to-v280) before running Aerie v2.8.0 for the first time. + +## For Aerie Developers +(ie. anyone running Aerie from a cloned version of the `aerie` repository) + +* Run `./gradlew clean` **before** checking out `develop` or `v2.8.0` in order to clean up old SQL files. + - Or if you have already checked it out, run `rm -rf deployment/postgres-init-db/sql && git restore deployment/postgres-init-db/sql` +* Several new environment variables are now required. Compare the new version of `env.template` in the repository root to your existing `.env` file and copy over any new variables which are missing in your `.env`. Keep any of your existing values, and fill in (arbitrary) values for the password variables which are blank in the template. +* If you plan to migrate your data, follow the [migration instructions below](#for-anyone-migrating-data-to-v280) before running Aerie v2.8.0 for the first time +* When running v2.8.0 for the first time, make sure to rebuild the backend, and build/pull all containers, ie.: + - `docker compose down --remove-orphans && ./gradlew assemble && docker compose up --pull always -d --build` + + +## For Anyone Migrating Data to v2.8.0 +These instructions apply if you wish to save your data and **migrate it forward** to work with v2.8.0, whether you're a user or a developer. + +### Migration Prerequisites +* Your database must already be up-to-date with the migrations from the prior release, 2.7.1. If you are upgrading from an earlier version, you should upgrade and migrate to v2.7.1 **first** before continuing. +* You should have already followed one set of instructions above to modify either your `docker-compose` file or your `.env` file. +* Make sure you have the `psql` command installed and you can run it from your shell. If not: + - Mac users can run `brew install libpq` + - For other platforms, [download the Postgres installer](https://www.postgresql.org/download), and uncheck "PostgreSQL Server" when installing +* From the deployment folder, run `python -m pip install -r requirements.txt` to make sure you have all necessary Python packages installed +* If you have ever changed the default names for the Aerie databases (`aerie_merlin`, `aerie_scheduler`, `aerie_sequencing`, `aerie_ui`) - this is no longer allowed, and you **must** rename them to their default names. + +### Migration Process +1. Starting from the deployment directory, run `cd merge_aerie_db` +2. Run `./merge_db.sh -h` to list the available command line flags and see if any apply to your deployment - ie. if you are using non-default config file locations or connecting to a non-local database +3. Run `./merge_db.sh`, adding any necessary command line flags, to start the migration +4. As a precaution, your **old** databases have been preserved at this point. If the last step failed for any reason, you can safely drop the `aerie` database, troubleshoot the problem and re-run the `merge_db.sh` script +5. After the migration has run successfully, run `drop_old_dbs.sh` to drop your old databases. + +Your data should now be fully migrated and ready to run Aerie v2.8.0! Note that, due to breaking changes to the mission model framework in v2.7.0, you may need to **recompile and re-upload** your mission models if you are upgrading from a version **before v2.7.0**. + + +## Appendix +### Full list of `docker-compose.yml` variable changes +Below are all of the changes to the variables in our `docker-compose.yml` file introduced in v2.8.0, if you plan to modify your version of this file manually: + +- aerie_gateway: + - removed envvars: + - `POSTGRES_AERIE_MERLIN_DB` + - `POSTGRES_USER` + - `POSTGRES_PASSWORD` + - renamed envvars: + - `POSTGRES_HOST` -> `AERIE_DB_HOST` + - `POSTGRES_PORT` -> `AERIE_DB_PORT` + - added envvars: + - `GATEWAY_DB_USER: "${GATEWAY_USERNAME}"` + - `GATEWAY_DB_PASSWORD: "${GATEWAY_PASSWORD}"` +- aerie_merlin: + - removed envvars: `MERLIN_DB` + - renamed envvars: + - `MERLIN_DB_SERVER` -> `AERIE_DB_HOST` + - `MERLIN_DB_PORT` -> `AERIE_DB_PORT` + - `MERLIN_DB_USER` should now be mapped at the envvar `"${MERLIN_USERNAME}"` + - `MERLIN_DB_PASSWORD` should now be mapped at the envvar `"${MERLIN_PASSWORD}"` +- aerie_merlin_workers: (for each worker) + - removed envvars: + - `MERLIN_WORKER_DB` + - `MERLIN_WORKER_DB_USER` + - `MERLIN_WORKER_DB_PASSWORD` + - renamed envvars: + - `MERLIN_WORKER_DB_SERVER` -> `AERIE_DB_HOST` + - `MERLIN_WORKER_DB_PORT` -> `AERIE_DB_PORT` + - added envvars: + - `MERLIN_DB_USER: "${MERLIN_USERNAME}"` + - `MERLIN_DB_PASSWORD: "${MERLIN_PASSWORD}"` +- aerie_scheduler: + - removed envvars: `SCHEDULER_DB` + - renamed envvars: + - `SCHEDULER_DB_SERVER` -> `AERIE_DB_HOST` + - `SCHEDULER_DB_PORT` -> `AERIE_DB_PORT` + - `SCHEDULER_DB_USER` should now be mapped at the envvar `"${SCHEDULER_USERNAME}"` + - `SCHEDULER_DB_PASSWORD` should now be mapped at the envvar `"${SCHEDULER_PASSWORD}"` +- aerie_scheduler_workers: (for each worker) + - removed envvars: + - `SCHEDULER_WORKER_DB` + - `SCHEDULER_WORKER_DB_USER` + - `SCHEDULER_WORKER_DB_PASSWORD` + - renamed envvars: + - `SCHEDULER_WORKER_DB_SERVER` -> `AERIE_DB_HOST` + - `SCHEDULER_WORKER_DB_PORT` -> `AERIE_DB_PORT` + - added envvars: + - `SCHEDULER_DB_USER: "${SCHEDULER_USERNAME}"` + - `SCHEDULER_DB_PASSWORD: "${SCHEDULER_PASSWORD}"` +- aerie_sequencing: + - removed envvars: `SEQUENCING_DB` + - renamed envvars: + - `SEQUENCING_DB_SERVER` -> `AERIE_DB_HOST` + - `SEQUENCING_DB_PORT` -> `AERIE_DB_PORT` + - `SEQUENCING_DB_USER` should now be mapped at the envvar `"${SEQUENCING_USERNAME}"` + - `SEQUENCING_DB_PASSWORD` should now be mapped at the envvar `"${SEQUENCING_PASSWORD}"` +- hasura: + - removed envvars: + - `AERIE_MERLIN_DATABASE_URL` + - `AERIE_SCHEDULER_DATABASE_URL` + - `AERIE_SEQUENCING_DATABASE_URL` + - `AERIE_UI_DATABASE_URL` + - added envvars: + - `AERIE_DATABASE_URL: "postgres://${AERIE_USERNAME}:${AERIE_PASSWORD}@postgres:5432/aerie?options=-c%20search_path%3Dutil_functions%2Chasura%2Cpermissions%2Ctags%2Cmerlin%2Cscheduler%2Csequencing%2Cpublic"` +- postgres: + - added envvars: + - `GATEWAY_DB_USER: "${GATEWAY_USERNAME}"` + - `GATEWAY_DB_PASSWORD: "${GATEWAY_PASSWORD}"` + - `MERLIN_DB_USER: "${MERLIN_USERNAME}"` + - `MERLIN_DB_PASSWORD: "${MERLIN_PASSWORD}"` + - `SCHEDULER_DB_USER: "${SCHEDULER_USERNAME}"` + - `SCHEDULER_DB_PASSWORD: "${SCHEDULER_PASSWORD}"` + - `SEQUENCING_DB_USER: "${SEQUENCING_USERNAME}"` + - `SEQUENCING_DB_PASSWORD: "${SEQUENCING_PASSWORD}"` + diff --git a/package.json b/package.json index be3ae4a..965f346 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "aerie-docs", - "version": "1.0.1", + "version": "2.8.0", "private": true, "engines": { "node": ">=16.14" diff --git a/sidebars.js b/sidebars.js index 0a8f07c..f430246 100644 --- a/sidebars.js +++ b/sidebars.js @@ -324,6 +324,7 @@ const sidebars = { 'glossary', ], upgradeGuides: [ + 'upgrade-guides/2-7-1-to-2-8-0', 'upgrade-guides/2-7-0-to-2-7-1', 'upgrade-guides/2-6-1-to-2-7-0', 'upgrade-guides/2-6-0-to-2-6-1',