- We are taking an app-by-app approach to migrating from Bootstrap 3 to 5. This process is summarized by the following:
+ Bootstrap is a third-party UI library used throughout
+ HQ. It's essentially a set of stylesheets and javascript widgets. This means that migrating from one version to the
+ next involves the following kinds of work:
+
+
+ Because the HQ codebase is large, we are taking an incremental approach to this migration.
+ For organizational purposes, we are executing and tracking this migration primarily on an app-by-app basis.
+
+
+ This means that files that are required by both migrated and unmigrated pages need to be "split" into two
+ versions: one to include on Bootstrap 3 pages and another to include on Bootstrap 5 pages. This applies largely to code that
+ is fundamental to HQ's UI and used throughout the platform, such as the hqwebapp app, the
+ analytics JavaScript libraries, etc.
+
+
+ It can also be helpful to "split" files in this way while migrating an app. This allows
+ Bootstrap 5 changes to be developed, reviewed, and merged in small pieces, before switching the Bootstrap
+ version used in production. It is a more complex process that adds more code during the migration,
+ although most of this code is automatically generated by the migration tool.
+ In these cases, once the app is fully migrated, the Bootstrap 3 files can be deleted.
+
+
+ To support this approach, we have tooling that facilitates "splitting" and "unsplitting" files:
+ copying files, moving them, and updating references to them. This tooling also performs the simpler migration
+ updates, such as renaming CSS classes. For updates that it cannot handle automatically, it adds a TODO comment
+ to the relevant code and writes out a changelog.
+
+
+ Once the entire platform is fully migrated to Bootstrap 5, any remaining Bootstrap 3 code can be removed:
+ the Bootstrap 3 versions of files in hqwebapp and similar apps, the @use_bootstrap5
+ decorator, any code related to request.use_bootstrap5 or window.USE_BOOTSTRAP_5,
+ the Bootstrap 3 stylesheets and JavaScript library, and so forth.
-
If you haven't done so, you will need to install Dart Sass to compile
@@ -108,24 +130,83 @@
- Before you begin the migration of an app, please announce your intention to do so in #gtd-dev. It is
- important to communicate timelines for how long you expect the "split files" stage to last. Additionally, please
- ask that people refrain from any front-end development in that app until changes from the split files step have
- been merged.
-
-
-
Important:
- After announcing your intent to migrate the app, please update the app's migration status
in this table. This is also a good place to see what apps have not been migrated yet, if
- you are in need of ideas.
+ target="_blank">This spreadsheet is a high-level tracker of which apps have been migrated.
+ It contains notes on the approximate size of different apps (in terms of how many views they contain) and on
+ inter-app dependencies.
+
+
+ The migration tool supports two general workflows for migrating an app: one based on splitting all files into
+ Bootstrap 3 and Bootstrap 5 versions, and one based on migrating files "in place." The splitting files approach
+ is necessary for apps that are dependencies for other apps, which will need to support both
+ Bootstrap 3 and Bootstrap 5 for a while. It is optional for small, independent apps.
+
+
+ Before you begin the migration of an app, please announce your intention to do so in #gtd-dev.
+ It is helpful to be aware of other front-end development in the app. If the app has ongoing front-end
+ development, consider delaying either that development or the migration, to reduce code conflicts.
+ After announcing your intent to migrate the app, please update
+ the app's migration status.
+
+
+
+ Step 2 (small apps): The "No Split" Process
+
+
+ For small apps that are not depended on by other apps, the migration tool supports a streamlined process that
+ does not require splitting files into a Bootstrap 3 and Bootstrap 5 version. This is a good option for apps
+ with a small number of views, as it involves fewer pull requests and avoids the overhead of splitting files.
+ This approach does the entire migration in a single pull request, which
+ makes it unlikely to be a good choice for apps with 10+ views.
+
+
+ To begin a "no split" migration, first
+ lint the app's JavaScript
+ and make a pull request for any linting changes. Then, to begin the actual migration, run:
+
+
+
+
./manage.py migrate_app_to_bootstrap5 <app_name> --no-split
+
+
+
+ This will iterate through all the templates and javascript files in <app_name>,
+ applying all automated fixes and adding TODO comments for fixes that aren't automated. After
+ staging the changes to each file, you will have the option to auto-commit those changes.
+
+
+ For each migrated template, the command will wait for you to apply the @use_bootstrap5 decorator
+ to the relevant views (see Migrating Views) before moving on to the next template.
+
+
+ Once all the files are migrated in-place, go through the TODO comments, which reference helper documents in the
+ changes_guide directory.
+
+
+ After all TODOs are addressed, test your work: load the page and
+ test out any interactivity. Most migrations should also go through QA. Migrations can skip QA if they make
+ relatively superficial changes and are in low-risk areas.
+
+
+ Also mark the migration as "complete" as part of your pull request. This updates a status file where we
+ programmatically track the status of the overall platform migration. To do so:
+
+
+
+
./manage.py complete_bootstrap5_migration <app_name>
+
+
+ You're done! Ignore the rest of this document.
+
- Step 2: Auto-Migrate & Split Files
+ Step 2 (large apps): Auto-Migrate & Split Files
- The next step is to split affected template and javascript files into bootstrap3 and
+ For large migrations, the first step is to split affected template and javascript files into bootstrap3 and
bootstrap5 versions. This is an automated process that uses a management command to find and
replace the straightforward changes, while flagging more complex changes to be addressed later.
@@ -187,7 +268,8 @@
The option --skip-all is optional, but recommended unless you have a particularly tricky app
as it speeds up the migration process by auto-committing the split files changes for each template and
- javascript file in the app.
+ javascript file in the app. If you use --skip-all, you are still responsible for changes made by
+ the tool, so it's a good idea to review the auto-generated commits in GitHub or your favorite diff tool.
If --skip-all is not used, the management command will iterate through all template and javascript
@@ -195,15 +277,19 @@
the diff for each file at the end.
- Regardless of what route you take, a changelog for each file will be saved to a markdown file in
+ Regardless of what route you take, code will also be updated with TODO comments noting areas that need
+ attention and could not automatically be updated. These comments reference helper documents in the
+ changes_guide directory.
+
+
+ In addition, a changelog for each file will be saved to a markdown file in
BOOTSTRAP_MIGRATION_LOGS_DIR (ouside of repository).
- If you want to work on migrating an application with multiple people, you can zip these markdown files and
- share with your collaborators for future reference.
- It is possible that you might have to run a migration multiple times on an app. It might be the case that templates
- or javascript files have several nested dependencies.
+ If templates or javascript files have several nested dependencies, you may need to run this split files step
+ multiple times.
For instance, let's suppose one template called webapps_base.html is
@@ -307,36 +393,6 @@
build_bootstrap5_diffs to rebuild the diffs.
-
- Step 2.1B: Migrating Smaller Apps
-
-
- Some apps only have a handful of templates and javascript files, and it's possible to run the auto-migration
- on the templates in-place and migrate the views to Bootstrap 5 all within the same Pull Request. In this case,
- you can make use of the --no-split option for migrate_app_to_bootstrap5.
-
-
-
-
./manage.py migrate_app_to_bootstrap5 <app_name> --no-split
-
-
-
- This option will iterate through all the templates and javascript files in <app_name>. After
- staging the changes to each file, you will have the option to auto-commit those changes.
-
-
- For each migrated template, the command will wait for you to apply the @use_bootstrap5 decorator
- to the relevant views (see Migrating Views) before moving on to the next template.
-
-
- Once all the files are migrated in-place, you can then make additional manual changes to each template and view.
- When the views are fully migrated, you can then create your pull request.
-
-
- This option is useful for smaller apps, because it avoids the split-files and reference renames. Please only
- choose this option if you are confident you can migrate all the views and javascript files within 1 to 2 weeks.
-
-
Step 2B: Migrate Stylesheets
@@ -440,8 +496,8 @@
part accurately.
- It is important to create a separate Pull Request for stylesheet changes. This PR should come after the "split files"
- changes in the previous step.
+ Updating stylesheets is in large part a mechanical process. As always, please commit large mechanical changes
+ separately from changes that need closer review.
You will have to update bootstrap5_diff_config.json manually when working with stylesheet files.
@@ -474,11 +530,12 @@
Finally, you should make sure any javascript files needed to render that view also inherit from bootstrap5
- dependencies.
+ dependencies. The migration tool should have handled updating any bootstrap3 dependencies to their
+ bootstrap5 versions.
- Once a view's template and javascript files updated, it is now time to load that view make visual observations and
- make adjustments to get the view to render without javascript or visual errors. The view should be as similar
+ Once a view's template and javascript files are updated, it is now time to load that view and test for
+ any javascript or visual errors. The view should be as similar
as possible to its Bootstrap 3 version, which you can compare with on production.
@@ -531,15 +588,14 @@
Deploy to Staging
-
- IMPORTANT: Do not skip this step! Please deploy to staging to avoid any fires. You have been warned.
-
- Once you have your migrated view working locally, it is important to deploy your view's branch to staging to ensure
- that the requirejs build doesn't encounter any issues and that there are no other staticfiles or javascript
- issues with this view. It is often the case that the production environment introduces front-end issues that are
- often not discovered when testing locally, due to the complexity of the requirejs build, django compressor, caching,
- and content distribution of static files.
+ Once you have your migrated view working locally, it is also helpful to deploy to staging and smoke test there.
+ This is particularly important for migrations that involve switching a JavaScript library from a Bootstrap 3 to
+ Bootstrap 5 version, such as switching the Bootstrap 3 date picker with Botostrap 5-compatible tempus dominus.
+
+
+ Most view migrations should also go through QA. Migrations can skip QA if they make
+ relatively superficial changes and are in low-risk areas.
@@ -618,7 +674,7 @@
Finally, LESS files can be removed from all parts of HQ and COMPRESS_PRECOMPILERS in
settings.py can be updated to only precompile SCSS. Any LESS-related dependencies can be removed from
- packange.json and commcare-cloud, and new developer setup guides like DEV_SETUP.md can
+ package.json and commcare-cloud, and new developer setup guides like DEV_SETUP.md can
be updated to remove references to LESS.