diff --git a/docs/reference/hot_upgrde.md b/docs/reference/hot_upgrde.md new file mode 100644 index 0000000..159f8b0 --- /dev/null +++ b/docs/reference/hot_upgrde.md @@ -0,0 +1,158 @@ + ## Caution + + Please never try to hot upgrade a running application without + having first a good understand of how a hot upgrade is performed, + its limitations and steps required. + + See next sections for an overview of the hot upgrade process. + + ## Hot upgrading a running application + + Hot upgrade is the process that allows to seamlessly change + the code of a running application without the need to + stopping and restarting it, i.e. mantaining active the + service in production. + + This is one of the most interesting capabilities of Erlang/OTP, + but it is a very complex process that *cannot* be fully + automated, i.e. require a good knowledge of the tecnologies + involved and the configurations files needed and their locations + at every stage of the process. You have also to know how to + recognize when a hot upgrade isn't an advisable action, + because it could have some severe limitations + and unwanted consequences in some circumstances. + + Therefore it is strongly advised you read the official + documentation about the hot upgrade stuff on the Erlang/OTP + website, and how Distillery, the technolgy underlying + the Bootleg task, accomplished that. + + Here it is a selected - but not exaustive - list of important + pieces of documentation to read: + + # OTP Design Principles - Releases + http://erlang.org/doc/design_principles/release_structure.html + + # OTP Design Principles - Release Handling + http://erlang.org/doc/design_principles/release_handling.html + + # System Architecture Support Libraries - appup + http://erlang.org/doc/man/appup.html + + # Distillery - Hot upgrades and downgrades + https://hexdocs.pm/distillery/guides/upgrades_and_downgrades.html + + # Distillery - Appups + https://hexdocs.pm/distillery/guides/appups.html + + ### Bootleg hot upgrade task + + In the following description we assume that the development + enviroinment is organized in this way (the build and + the production places can be the same machine): + + * the development machine - where you edit and + test locally your app source files; + + * the build machine - the computer where you will transfer to + and compile the committed source code; + + * the production server - the server where you will deploy + (transfer to and run) the code previously compiled on + the build machine. + + Bootleg helps you in the hot upgrade process providing + some specific tasks: + + * mix bootleg.build_upgrade + will tranfer the last committed source code of your application + from the development machine to the build directory of + your build machine (for example `~/build/myapp/`), then + it will clean the directory from the previous code deleting + every file but the `_build` directory, it will generate the + `appup` file and compile the newest app release. + + Please note that before you can use this task for the first time, + you have to deploy your _first version_ of your app using + `bootleg.build`, `bootleg.deploy` and `bootleg.start` + (or `bootleg.update`); + + * mix bootleg.deploy_upgrade + will transfer the tarball of the compiled app from the + build machine to the production directory of the production + machine (e.g. `~/production/myapp/`), then it will extract + and setting up the needed files; + + * mix bootleg.hot_upgrade + will call `mix distillery upgrade ` that + will upgrade the running app to the last version. Notice that + you *cannot* use this task if the app is not running, or + if it there is a mismatch in the version numbers of the + deployed versions. + + * mix bootleg.upgrade + Call in sequences the above tasks in just one command. + + ### A step-by-step example + + Given you have configured the first version of your app with all + the needed and appropriately customized Bootleg configuration files, + you can go through the following steps to release and run the + first version, and subsequentely hot upgrade it to the newest + versions: + + First version of your app + + # Step 1 - deploy the first version of your app + edit the version number of your in the mix.exs file + (or in the file if you use an external reference), + to the first version (e.g. 0.1.0); + + # Step 2 - Commit + commit the changes you've made in step 1; + + # Step 3 - Build the first version + use `mix bootleg.build` (not bootleg.build_upgrade!) to build + your first version; + + # Step 4 - Deploy the first version + use `mix bootleg.deploy` (not bootleg.build_upgrade!) to deploy + your first version; + + # Step 5 - Run the first version + use `mix bootleg.start` to run the app + + now your first version is up and running. To upgrade it + to the future version, you have to follow these steps instead: + + Following versions + + # Step 1 - update the version number + e.g. 0.2.0 + + # Step 2 - Commit + + # Step 3 - Build the new version + use `mix bootleg.build_upgrade` + + # Step 4 - Deploy the new version + use `mix bootleg.deploy_upgrade` + + # Step 5 - Hot upgrade the new version + use `mix bootleg.hot_upgrade` + + (or you can execute just the `bootleg.upgrade` + that packs the previous tasks together if you don't need to + manually adjust the created `appup` file) + + Now you have an upgraded version running. But if you stop + and restart it, the previous version will be launched instead + of the most recent. This is useful because if your new version + has some blocking bug, you can easily restart the service to the last + working release. + + If you are shure that you want to having the last version restarted, + just delete the folder `~/production/myapp/var`. This folder contains + the file `start_erl.data` that lists the version number to start with. + Deleting the `var` folder will automatically create it next time the app + is started, with the last version number. diff --git a/lib/bootleg/tasks/build/remote.exs b/lib/bootleg/tasks/build/remote.exs index 4723da7..41b693c 100644 --- a/lib/bootleg/tasks/build/remote.exs +++ b/lib/bootleg/tasks/build/remote.exs @@ -294,3 +294,72 @@ task :pull_remote do end before_task(:pull_remote, :verify_repo_config) + +# Hot Upgrade support +task :remote_build_upgrade do + build_role = Config.get_role(:build) + invoke(:init) + invoke(:clean_for_upgrade) + invoke(:remote_scm_update) + invoke(:compile) + invoke(:remote_generate_release_upgrade) + + if build_role.options[:release_workspace] do + invoke(:copy_build_release) + else + invoke(:download_release) + end +end + +task :clean_for_upgrade do + remote :build do + "ls" + end + |> Enum.map(fn result -> + with {:ok, stdout_list, _code, _host} when stdout_list != [] <- result do + locations = + stdout_list + |> Keyword.get(:stdout) + |> String.split("\n") + |> Enum.drop(-1) + |> Enum.filter(fn el -> el != "_build" end) + |> Enum.join(" ") + + if locations != "" do + remote :build do + "rm -rvf #{locations}" + end + end + end + end) +end + +task :generate_upgrade_release do + invoke(:remote_generate_release_upgrade) +end + +task :remote_generate_release_upgrade do + mix_env = config({:mix_env, "prod"}) + source_path = config({:ex_path, ""}) + + release_args = + {:release_args, ["--quiet"]} + |> config() + |> Enum.join(" ") + + UI.info("Generating upgrade release...") + + remote :build, cd: source_path do + "MIX_ENV=#{mix_env} mix distillery.release --upgrade #{release_args}" + end +end + +task :remote_hot_upgrade do + app_name = "#{Config.app()}" + + UI.info("Upgrading #{app_name} to version: #{Config.version()}") + + remote :app do + "bin/#{app_name} upgrade #{Config.version()}" + end +end diff --git a/lib/bootleg/tasks/build_upgrade.exs b/lib/bootleg/tasks/build_upgrade.exs new file mode 100644 index 0000000..2edd4da --- /dev/null +++ b/lib/bootleg/tasks/build_upgrade.exs @@ -0,0 +1,12 @@ +alias Bootleg.{UI, Config, DSL} +use Bootleg.DSL + +task :build_upgrade do + build_type = config({:build_type, "remote"}) + bootleg_env = config(:env) + UI.info("Starting #{build_type} build for #{bootleg_env} environment") + invoke(:"#{build_type}_verify_config") + invoke(:"#{build_type}_build_upgrade") +end + +before_task(:build, :verify_config) diff --git a/lib/bootleg/tasks/deploy_upgrade.exs b/lib/bootleg/tasks/deploy_upgrade.exs new file mode 100644 index 0000000..68c7193 --- /dev/null +++ b/lib/bootleg/tasks/deploy_upgrade.exs @@ -0,0 +1,50 @@ +alias Bootleg.{UI, Config, DSL} +use Bootleg.DSL + +task :deploy_upgrade do + app_role = Config.get_role(:app) + + if app_role.options[:release_workspace] do + invoke(:copy_deploy_release_upgrade) + else + invoke(:upload_release_upgrade) + end + + invoke(:unpack_release_upgrade) +end + +task :copy_deploy_release_upgrade do + app_role = Config.get_role(:app) + release_workspace = app_role.options[:release_workspace] + release = "#{Config.version()}.tar.gz" + source_path = Path.join(release_workspace, release) + dest_path = "#{Config.app()}.tar.gz" + + UI.info("Copying release archive from release upgrade workspace") + + remote :app do + "cp #{source_path} #{dest_path}" + end +end + +task :upload_release_upgrade do + remote_path = "#{Config.app()}.tar.gz" + local_archive_folder = "#{File.cwd!()}/releases" + local_path = Path.join(local_archive_folder, "#{Config.version()}.tar.gz") + UI.info("Uploading release upgrade archive") + upload(:app, local_path, remote_path) +end + +task :unpack_release_upgrade do + remote_path = "#{Config.app()}.tar.gz" + remote_dest = "#{Config.version()}/#{Config.app()}.tar.gz" + + UI.info("Unpacking release upgrade archive") + + remote :app do + "tar -zxvf #{remote_path}" + "cp #{remote_path} releases/#{remote_dest}" + end + + UI.info("Unpacked release upgrade archive") +end diff --git a/lib/bootleg/tasks/hot_downgrade.exs b/lib/bootleg/tasks/hot_downgrade.exs new file mode 100644 index 0000000..d1096cb --- /dev/null +++ b/lib/bootleg/tasks/hot_downgrade.exs @@ -0,0 +1,11 @@ +alias Bootleg.{UI, Config, DSL} +use Bootleg.DSL + +task :hot_downgrade do + build_type = config({:build_type, "remote"}) + bootleg_env = config(:env) + UI.info("Starting #{build_type} hot downgrade for #{bootleg_env} environment") + invoke(:"#{build_type}_hot_downgrade") +end + +before_task(:build, :verify_config) diff --git a/lib/bootleg/tasks/hot_upgrade.exs b/lib/bootleg/tasks/hot_upgrade.exs new file mode 100644 index 0000000..bb46e67 --- /dev/null +++ b/lib/bootleg/tasks/hot_upgrade.exs @@ -0,0 +1,11 @@ +alias Bootleg.{UI, Config, DSL} +use Bootleg.DSL + +task :hot_upgrade do + build_type = config({:build_type, "remote"}) + bootleg_env = config(:env) + UI.info("Starting #{build_type} hot upgrade for #{bootleg_env} environment") + invoke(:"#{build_type}_hot_upgrade") +end + +before_task(:build, :verify_config) diff --git a/lib/bootleg/tasks/upgrade.exs b/lib/bootleg/tasks/upgrade.exs new file mode 100644 index 0000000..44c5440 --- /dev/null +++ b/lib/bootleg/tasks/upgrade.exs @@ -0,0 +1,8 @@ +alias Bootleg.{Config, DSL} +use DSL + +task :upgrade do + invoke(:build_upgrade) + invoke(:deploy_upgrade) + invoke(:hot_upgrade) +end diff --git a/lib/mix/tasks/build_upgrade.ex b/lib/mix/tasks/build_upgrade.ex new file mode 100644 index 0000000..9e4283a --- /dev/null +++ b/lib/mix/tasks/build_upgrade.ex @@ -0,0 +1,27 @@ +defmodule Mix.Tasks.Bootleg.BuildUpgrade do + use Bootleg.MixTask, :build_upgrade + + @shortdoc "Build a release for upgrade" + + @moduledoc """ + Build a release for upgrade + + ## Usage + + mix bootleg.build + + ## Caution + + Please never try to hot upgrade a running application without + having first a good understand of how a hot upgrade is performed, + its limitations and steps required. + + ## Documentation + + Please see the "Hot upgrading a running application" section + of `bootleg.upgrade` documentation for an overview of + the hot upgrade process: + + mix help bootleg.upgrade + """ +end diff --git a/lib/mix/tasks/deploy_upgrade.ex b/lib/mix/tasks/deploy_upgrade.ex new file mode 100644 index 0000000..9a510a1 --- /dev/null +++ b/lib/mix/tasks/deploy_upgrade.ex @@ -0,0 +1,27 @@ +defmodule Mix.Tasks.Bootleg.DeployUpgrade do + use Bootleg.MixTask, :deploy_upgrade + + @shortdoc "Deploy an upgrade release" + + @moduledoc """ + Deploy an upgrade release + + ## Usage + + mix bootleg.deploy_upgrade + + ## Caution + + Please never try to hot upgrade a running application without + having first a good understand of how a hot upgrade is performed, + its limitations and steps required. + + ## Documentation + + Please see the "Hot upgrading a running application" section + of `bootleg.upgrade` documentation for an overview of + the hot upgrade process: + + mix help bootleg.upgrade + """ +end diff --git a/lib/mix/tasks/hot_downgrade.ex b/lib/mix/tasks/hot_downgrade.ex new file mode 100644 index 0000000..a5e0091 --- /dev/null +++ b/lib/mix/tasks/hot_downgrade.ex @@ -0,0 +1,27 @@ +defmodule Mix.Tasks.Bootleg.HotDowngrade do + use Bootleg.MixTask, :hot_downgrade + + @shortdoc "Downgrade a running release with the last release" + + @moduledoc """ + Downgrade a running release with the last release + + ## Usage + + mix bootleg.hot_downgrade + + ## Caution + + Please never try to hot upgrade a running application without + having first a good understand of how a hot upgrade is performed, + its limitations and steps required. + + ## Documentation + + Please see the "Hot upgrading a running application" section + of `bootleg.upgrade` documentation for an overview of + the hot upgrade process: + + mix help bootleg.upgrade + """ +end diff --git a/lib/mix/tasks/hot_upgrade.ex b/lib/mix/tasks/hot_upgrade.ex new file mode 100644 index 0000000..cb6c033 --- /dev/null +++ b/lib/mix/tasks/hot_upgrade.ex @@ -0,0 +1,23 @@ +defmodule Mix.Tasks.Bootleg.HotUpgrade do + use Bootleg.MixTask, :hot_upgrade + + @shortdoc "Upgrade a running release with the last release" + + @moduledoc """ + Upgrade a running release with the last release + + ## Caution + + Please never try to hot upgrade a running application without + having first a good understand of how a hot upgrade is performed, + its limitations and steps required. + + ## Documentation + + Please see the "Hot upgrading a running application" section + of `bootleg.upgrade` documentation for an overview of + the hot upgrade process: + + mix help bootleg.upgrade + """ +end diff --git a/lib/mix/tasks/upgrade.ex b/lib/mix/tasks/upgrade.ex new file mode 100644 index 0000000..17b6353 --- /dev/null +++ b/lib/mix/tasks/upgrade.ex @@ -0,0 +1,175 @@ +defmodule Mix.Tasks.Bootleg.Upgrade do + use Bootleg.MixTask, :upgrade + + @shortdoc "Build, deploy, and hot upgrade a release all in one command." + + @moduledoc """ + Build, deploy, and hot upgrade a new release all in one command. + + Note that this comand will _not_ do an Ecto migration. + + ## Usage: + + * mix bootleg.upgrade + + ## Caution + + Please never try to hot upgrade a running application without + having first a good understand of how a hot upgrade is performed, + its limitations and steps required. + + See next sections for an overview of the hot upgrade process. + + ## Hot upgrading a running application + + Hot upgrade is the process that allows to seamlessly change + the code of a running application without the need to + stopping and restarting it, i.e. mantaining active the + service in production. + + This is one of the most interesting capabilities of Erlang/OTP, + but it is a very complex process that *cannot* be fully + automated, i.e. require a good knowledge of the tecnologies + involved and the configurations files needed and their locations + at every stage of the process. You have also to know how to + recognize when a hot upgrade isn't an advisable action, + because it could have some severe limitations + and unwanted consequences in some circumstances. + + Therefore it is strongly advised you read the official + documentation about the hot upgrade stuff on the Erlang/OTP + website, and how Distillery, the technolgy underlying + the Bootleg task, accomplished that. + + Here it is a selected - but not exaustive - list of important + pieces of documentation to read: + + # OTP Design Principles - Releases + http://erlang.org/doc/design_principles/release_structure.html + + # OTP Design Principles - Release Handling + http://erlang.org/doc/design_principles/release_handling.html + + # System Architecture Support Libraries - appup + http://erlang.org/doc/man/appup.html + + # Distillery - Hot upgrades and downgrades + https://hexdocs.pm/distillery/guides/upgrades_and_downgrades.html + + # Distillery - Appups + https://hexdocs.pm/distillery/guides/appups.html + + ### Bootleg hot upgrade task + + In the following description we assume that the development + enviroinment is organized in this way (the build and + the production places can be the same machine): + + * the development machine - where you edit and + test locally your app source files; + + * the build machine - the computer where you will transfer to + and compile the committed source code; + + * the production server - the server where you will deploy + (transfer to and run) the code previously compiled on + the build machine. + + Bootleg helps you in the hot upgrade process providing + some specific tasks: + + * mix bootleg.build_upgrade + will tranfer the last committed source code of your application + from the development machine to the build directory of + your build machine (for example `~/build/myapp/`), then + it will clean the directory from the previous code deleting + every file but the `_build` directory, it will generate the + `appup` file and compile the newest app release. + + Please note that before you can use this task for the first time, + you have to deploy your _first version_ of your app using + `bootleg.build`, `bootleg.deploy` and `bootleg.start` + (or `bootleg.update`); + + * mix bootleg.deploy_upgrade + will transfer the tarball of the compiled app from the + build machine to the production directory of the production + machine (e.g. `~/production/myapp/`), then it will extract + and setting up the needed files; + + * mix bootleg.hot_upgrade + will call `mix distillery upgrade ` that + will upgrade the running app to the last version. Notice that + you *cannot* use this task if the app is not running, or + if it there is a mismatch in the version numbers of the + deployed versions. + + * mix bootleg.upgrade + Call in sequences the above tasks in just one command. + + ### A step-by-step example + + Given you have configured the first version of your app with all + the needed and appropriately customized Bootleg configuration files, + you can go through the following steps to release and run the + first version, and subsequentely hot upgrade it to the newest + versions: + + First version of your app + + # Step 1 - deploy the first version of your app + edit the version number of your in the mix.exs file + (or in the file if you use an external reference), + to the first version (e.g. 0.1.0); + + # Step 2 - Commit + commit the changes you've made in step 1; + + # Step 3 - Build the first version + use `mix bootleg.build` (not bootleg.build_upgrade!) to build + your first version; + + # Step 4 - Deploy the first version + use `mix bootleg.deploy` (not bootleg.build_upgrade!) to deploy + your first version; + + # Step 5 - Run the first version + use `mix bootleg.start` to run the app + + now your first version is up and running. To upgrade it + to the future version, you have to follow these steps instead: + + Following versions + + # Step 1 - update the version number + e.g. 0.2.0 + + # Step 2 - Commit + + # Step 3 - Build the new version + use `mix bootleg.build_upgrade` + + # Step 4 - Deploy the new version + use `mix bootleg.deploy_upgrade` + + # Step 5 - Hot upgrade the new version + use `mix bootleg.hot_upgrade` + + (or you can execute just the `bootleg.upgrade` + that packs the previous tasks together if you don't need to + manually adjust the created `appup` file) + + Now you have an upgraded version running. But if you stop + and restart it, the previous version will be launched instead + of the most recent. This is useful because if your new version + has some blocking bug, you can easily restart the service to the last + working release. + + If you are shure that you want to having the last version restarted, + just delete the folder `~/production/myapp/var`. This folder contains + the file `start_erl.data` that lists the version number to start with. + Deleting the `var` folder will automatically create it next time the app + is started, with the last version number. + + """ +end diff --git a/mkdocs.yml b/mkdocs.yml index d7162af..539020a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -60,6 +60,7 @@ nav: - 'Bootleg Task Providers': 'reference/task_providers.md' - 'Continuous Integration': 'reference/ci.md' - 'Customization': 'reference/customization.md' + - 'Hot Upgrade': 'reference/hot_upgrade.md' - 'More Resources': 'resources.md' - 'API Reference': 'api-reference.html'