Tweak the EDD doc (#184)

bitwarden · Sep 21, 2023 · f420880 · f420880
1 parent e4f5218
commit f420880
Show file tree

Hide file tree

Showing 5 changed files with 119 additions and 60 deletions.
diff --git a/docs/contributing/database-migrations/edd.mdx b/docs/contributing/database-migrations/edd.mdx
@@ -1,16 +1,13 @@
 import Tabs from "@theme/Tabs";
 import TabItem from "@theme/TabItem";
 
-import refactoringPhases from "./refactoring_phases.jpg";
-
 # Evolutionary database design
 
-At Bitwarden we follow
-[Evolutionary Database Design (EDD)](https://en.wikipedia.org/wiki/Evolutionary_database_design).
-EDD describes a process where the database schema is continuously updated while still ensuring
-compatibility with older releases by using database transition phases.
+At Bitwarden we follow [Evolutionary Database Design (EDD)][edd-wiki]. EDD describes a process where
+the database schema is continuously updated while still ensuring compatibility with older releases
+by defining a database transition phases.
 
-Additional requirements include:
+Bitwarden also needs to support:
 
 - **Zero-downtime deployments**: Which means that multiple versions of the application will be
   running concurrently during the deployment window.
@@ -32,73 +29,76 @@ For background on this decision please see the [Evolutionary Database Design RFD
 
 ## Design
 
-Database changes can be categorized into two categories: destructive and non-destructive
+Database changes can be categorized into two categories: destructive and non-destructive changes
 \[[1](./edd#further-reading)\]. A destructive change prevents existing functionality from working as
-expected without an accompanying code change. A non-destructive change is one that does not require
-a code change to allow the application to continue working as expected.
+expected without an accompanying code change. A non-destructive change is the opposite: a database
+change that does not require a code change to allow the non-application to continue working as
+expected.
 
-### Non-destructive database changes
+### Non-destructive changes
 
-Non-destructive changes almost always use either nullable fields or default values in the database
-tables, views, and stored procedures. This ensures that the stored procedures can be called without
-the new columns which allows it to run with both the old and new code.
+Many database changes can be designed in a backwards compatible manner by using a mix of nullable
+fields and default values in the database tables, views, and stored procedures. This ensures that
+the stored procedures can be called without the new columns and allow them to run with both the old
+and new code.
 
 ### Destructive changes
 
-Destructive database changes are handled elegantly by breaking them up into three phases: _Start_,
-_Transition_ and _End_.
-
-<div style={{ margin: "1em" }}>
-  <img src={refactoringPhases} alt="Refactoring Phases" />
-  <div style={{ fontSize: 12, textAlign: "center" }}>
-    Refactoring Phases:{" "}
-    <a href="https://martinfowler.com/articles/evodb.html">
-      {" "}
-      https://martinfowler.com/articles/evodb.html{" "}
-    </a>
-  </div>
-</div>
-
-To add terminology to compliment the [above diagram](./edd#destructive-changes), migrations that are
-a part of "Deploy new changes, migrate data, put in scaffolding code" are considered _Initial_
-migrations. Migrations that are run during the Transition Phase are considered _Transition_
-migrations. And the migrations that run as a part of "Remove old schema, scaffolding code" are
-considered _Finalization_ migrations. The definitions of each are helpful when discussing the type
-of migration in relation to orchestrating them during a deployment.
-
-### Initial migrations
-
-- Compatible with _X.1.0_ **and** _X.2.0_ application code changes
-- Represents the beginning of a database change
-- Updates the database schema to support any new functionality while also maintaining old
-  functionality
-- Supports both the previous version of code and the one being upgraded to
-- Run during upgrade
-- Must execute quickly to minimize downtime.
-
-### Transition migrations
-
-- Compatible with _X.1.0_ **and** _X.2.0_ application code changes
-- The time between initial migration and finalization migration
-- Exists to provide an opportunity to rollback server to _X.1.0_ version prior to breaking changes
+Any change that cannot be done in a non-destructive manner is a destructive change. This can be as
+simple as adding a non nullable column where the value needs to be computed from existing fields, or
+renaming an existing column. To handle destructive changes it's necessary to break them up into
+three phases: _Start_, _Transition_, and _End_ as shown in the diagram below.
+
+<figure>
+
+![Refactoring Stages](./transitions.png)
+
+<figcaption>Refactoring Phases</figcaption>
+
+</figure>
+
+It's worth noting that the _Refactoring Phases_ are usually rolling, and the _End phase_ of one
+refactor is the _Transition phase_ of another. The table below details which application releases
+needs to be supported during which database phase.
+
+| Database Phase | Release X | Release X+1 | Release X+2 |
+| -------------- | --------- | ----------- | ----------- |
+| Start          | ✅        | ❌          | ❌          |
+| Transition     | ✅        | ✅          | ❌          |
+| End            | ❌        | ✅          | ✅          |
+
+### Migrations
+
+The three different migrations described in the diagram above are, _Initial migration_, _Transition
+migration_ and _ Finalization migration_.
+
+#### Initial migration
+
+The initial migration runs before the code deployment, and its purpose is to add support for
+_Release X+1_ without breaking support of _Release X_. The migration should execute quickly and not
+contain any costly operations to ensure zero downtime.
+
+#### Transition migration
+
+The transition migration are run sometime during the transition phase, and provides an optional data
+migration should it be too slow or put too much load on the database, or otherwise make it
+unsuitable for the _Initial migration_.
+
+- Compatible with _Release X_ **and** _Release X+1_ application.
 - Only data population migrations may be run at this time, if they are needed
-  - Optional step, required only when migrating data would be too slow to execute during the initial
-    migration. This might be a column population, index creation, anything to prepare the database
-    for the _X.2.0_ version
   - Must be run as a background task during the Transition phase.
-  - These MUST run in a way where the database stays responsive during the full migration
+  - Operation is batched or otherwise optimized to ensure the database stays responsive.
 - Schema changes are NOT to be run during this phase.
 
-### Finalization migrations
+#### Finalization migration
 
-- Only compatible with _X.2.0_ application code; represents the point of no return for this
-  migration
-- Removes columns, data, and fallback code required to support _X.1.0_ version
-- Should be run as a typical migration either during a subsequent upgrade
+The finalization migration removes the temporary measurements that were needed to retain backwards
+compatibility with _Release X_, and the database schema henceforth only supports _Release X+1_.
+These migrations are run as part of the deployment of _Release X+2_.
 
 ### Example
 
-Let’s look at an example, the rename column refactor is shown in the image below.
+Let's look at an example, the rename column refactor is shown in the image below.
 
 ![Rename Column Refactor](./rename-column.gif)
 
@@ -318,5 +318,6 @@ ensure their database changes run correctly against the currently released versi
 4. Refactoring Databases: Evolutionary Database Design (Addison-Wesley Signature Series (Fowler))
    ISBN-10: 0321774515
 
+[edd-wiki]: https://en.wikipedia.org/wiki/Evolutionary_database_design
 [edd-rfd]:
   https://bitwarden.atlassian.net/wiki/spaces/PIQ/pages/177701412/Adopt+Evolutionary+database+design
diff --git a/docs/contributing/database-migrations/refactoring_phases.jpg b/docs/contributing/database-migrations/refactoring_phases.jpg
diff --git a/docs/contributing/database-migrations/transitions.drawio b/docs/contributing/database-migrations/transitions.drawio
@@ -0,0 +1,53 @@
+<mxfile host="app.diagrams.net" modified="2023-08-22T09:26:06.965Z" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/116.0" etag="hovSHvbrOD95EHPObRd7" version="21.6.8" type="device">
+  <diagram name="Page-1" id="lcHUyxeu1cxOqkDhXFud">
+    <mxGraphModel dx="1802" dy="393" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
+      <root>
+        <mxCell id="0" />
+        <mxCell id="1" parent="0" />
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-3" value="Start" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#ff2603;fontColor=#ffffff;strokeColor=#000000;perimeterSpacing=0;" vertex="1" parent="1">
+          <mxGeometry x="-640" y="280" width="120" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-4" value="Transition" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#fffc02;fontColor=#000000;strokeColor=#000000;" vertex="1" parent="1">
+          <mxGeometry x="-500" y="280" width="120" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-6" value="End" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#73fa79;" vertex="1" parent="1">
+          <mxGeometry x="-360" y="280" width="120" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-26" style="rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.75;exitY=1;exitDx=0;exitDy=0;dashed=1;" edge="1" parent="1" source="PMLj0W4xVDKLJgSlkqyW-17">
+          <mxGeometry relative="1" as="geometry">
+            <mxPoint x="-510" y="280" as="targetPoint" />
+          </mxGeometry>
+        </mxCell>
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-17" value="&lt;div&gt;Deploy X + 1&lt;br&gt;&lt;/div&gt;&lt;div&gt;Initial migration&lt;/div&gt;" style="rounded=1;whiteSpace=wrap;html=1;dashed=1;glass=0;" vertex="1" parent="1">
+          <mxGeometry x="-620" y="180" width="120" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-27" style="rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.25;exitY=1;exitDx=0;exitDy=0;dashed=1;" edge="1" parent="1" source="PMLj0W4xVDKLJgSlkqyW-20">
+          <mxGeometry relative="1" as="geometry">
+            <mxPoint x="-370" y="280" as="targetPoint" />
+          </mxGeometry>
+        </mxCell>
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-20" value="&lt;div&gt;Deploy X + 2&lt;br&gt;&lt;/div&gt;&lt;div&gt;Finalization migration&lt;/div&gt;" style="rounded=1;whiteSpace=wrap;html=1;dashed=1;glass=0;" vertex="1" parent="1">
+          <mxGeometry x="-380" y="180" width="120" height="60" as="geometry" />
+        </mxCell>
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-23" value="" style="endArrow=none;html=1;rounded=0;strokeWidth=3;targetPerimeterSpacing=0;" edge="1" parent="1">
+          <mxGeometry width="50" height="50" relative="1" as="geometry">
+            <mxPoint x="-510" y="280" as="sourcePoint" />
+            <mxPoint x="-510" y="340" as="targetPoint" />
+          </mxGeometry>
+        </mxCell>
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-24" value="" style="endArrow=none;html=1;rounded=0;strokeWidth=3;" edge="1" parent="1">
+          <mxGeometry width="50" height="50" relative="1" as="geometry">
+            <mxPoint x="-370" y="280" as="sourcePoint" />
+            <mxPoint x="-370" y="340" as="targetPoint" />
+          </mxGeometry>
+        </mxCell>
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-29" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;dashed=1;" edge="1" parent="1" source="PMLj0W4xVDKLJgSlkqyW-28" target="PMLj0W4xVDKLJgSlkqyW-4">
+          <mxGeometry relative="1" as="geometry" />
+        </mxCell>
+        <mxCell id="PMLj0W4xVDKLJgSlkqyW-28" value="Transition migration" style="rounded=1;whiteSpace=wrap;html=1;dashed=1;glass=0;" vertex="1" parent="1">
+          <mxGeometry x="-500" y="370" width="120" height="60" as="geometry" />
+        </mxCell>
+      </root>
+    </mxGraphModel>
+  </diagram>
+</mxfile>
diff --git a/docs/contributing/database-migrations/transitions.png b/docs/contributing/database-migrations/transitions.png
diff --git a/src/css/custom.css b/src/css/custom.css
@@ -45,3 +45,8 @@
 .menu__link--active {
   font-weight: bold;
 }
+
+figcaption {
+  text-align: center;
+  font-style: italic;
+}