Support for hot upgrades

docs/reference/hot_upgrde.md

@@ -0,0 +1,158 @@
+  ## Caution
+
+  Please never try to hot upgrade a running application without 
+  having first a good understand of how a hot upgrade is performed, 
+  its limitations and steps required.
+
+  See next sections for an overview of the hot upgrade process.
+
+  ## Hot upgrading a running application
+
+  Hot upgrade is the process that allows to seamlessly change 
+  the code of a running application without the need to 
+  stopping and restarting it, i.e. mantaining active the 
+  service in production.
+
+  This is one of the most interesting capabilities of Erlang/OTP,
+  but it is a very complex process that *cannot* be fully 
+  automated, i.e. require a good knowledge of the tecnologies 
+  involved and the configurations files needed and their locations
+  at every stage of the process. You have also to know how to
+  recognize when a hot upgrade isn't an advisable action,
+  because it could have some severe limitations 
+  and unwanted consequences in some circumstances.
+
+  Therefore it is strongly advised you read the official 
+  documentation about the hot upgrade stuff on the Erlang/OTP
+  website, and how Distillery, the technolgy underlying
+  the Bootleg task, accomplished that.
+
+  Here it is a selected - but not exaustive - list of important 
+  pieces of documentation to read:
+
+  # OTP Design Principles - Releases
+    http://erlang.org/doc/design_principles/release_structure.html
+
+  # OTP Design Principles - Release Handling
+    http://erlang.org/doc/design_principles/release_handling.html
+
+  # System Architecture Support Libraries - appup
+    http://erlang.org/doc/man/appup.html
+
+  # Distillery - Hot upgrades and downgrades
+    https://hexdocs.pm/distillery/guides/upgrades_and_downgrades.html
+
+  # Distillery - Appups
+    https://hexdocs.pm/distillery/guides/appups.html
+
+  ### Bootleg hot upgrade task
+
+  In the following description we assume that the development 
+  enviroinment is organized in this way (the build and 
+  the production places can be the same machine):
+
+    * the development machine - where you edit and 
+      test locally your app source files;
+
+    * the build machine - the computer where you will transfer to
+      and compile the committed source code;
+
+    * the production server - the server where you will deploy 
+      (transfer to and run) the code previously compiled on 
+      the build machine.
+
+  Bootleg helps you in the hot upgrade process providing 
+  some specific tasks:
+
+    * mix bootleg.build_upgrade
+      will tranfer the last committed source code of your application
+      from the development machine to the build directory of 
+      your build machine (for example `~/build/myapp/`), then
+      it will clean the directory from the previous code deleting
+      every file but the `_build` directory, it will generate the
+      `appup` file and compile the newest app release.
+
+      Please note that before you can use this task for the first time, 
+      you have to deploy your _first version_ of your app using 
+      `bootleg.build`, `bootleg.deploy` and `bootleg.start` 
+      (or `bootleg.update`);
+
+    * mix bootleg.deploy_upgrade
+      will transfer the tarball of the compiled app from the 
+      build machine to the production directory of the production
+      machine (e.g. `~/production/myapp/`), then it will extract 
+      and setting up the needed files;
+
+    * mix bootleg.hot_upgrade
+      will call `mix distillery <myapp> upgrade <version>` that
+      will upgrade the running app to the last version. Notice that
+      you *cannot* use this task if the app is not running, or 
+      if it there is a mismatch in the version numbers of the
+      deployed versions.
+
+    * mix bootleg.upgrade
+      Call in sequences the above tasks in just one command.
+
+  ### A step-by-step example
+
+  Given you have configured the first version of your app with all
+  the needed and appropriately customized Bootleg configuration files, 
+  you can go through the following steps to release and run the 
+  first version, and subsequentely hot upgrade it to the newest
+  versions:
+
+  First version of your app
+
+    # Step 1 - deploy the first version of your app
+      edit the version number of your in the mix.exs file 
+      (or in the file if you use an external reference),
+      to the first version (e.g. 0.1.0);
+
+    # Step 2 - Commit
+      commit the changes you've made in step 1;
+
+    # Step 3 - Build the first version
+      use `mix bootleg.build` (not bootleg.build_upgrade!) to build 
+      your first version;
+
+    # Step 4 - Deploy the first version
+      use `mix bootleg.deploy` (not bootleg.build_upgrade!) to deploy 
+      your first version;
+
+    # Step 5 - Run the first version
+      use `mix bootleg.start` to run the app
+
+    now your first version is up and running. To upgrade it 
+    to the future version, you have to follow these steps instead:
+
+  Following versions
+
+    # Step 1 - update the version number
+      e.g. 0.2.0
+
+    # Step 2 - Commit
+
+    # Step 3 - Build the new version
+      use `mix bootleg.build_upgrade`
+
+    # Step 4 - Deploy the new version
+      use `mix bootleg.deploy_upgrade`
+
+    # Step 5 - Hot upgrade the new version
+      use `mix bootleg.hot_upgrade`
+
+    (or you can execute just the `bootleg.upgrade` 
+    that packs the previous tasks together if you don't need to
+    manually adjust the created `appup` file)
+
+    Now you have an upgraded version running. But if you stop
+    and restart it, the previous version will be launched instead
+    of the most recent. This is useful because if your new version 
+    has some blocking bug, you can easily restart the service to the last 
+    working release.
+
+    If you are shure that you want to having the last version restarted,
+    just delete the folder `~/production/myapp/var`. This folder contains
+    the file `start_erl.data` that lists the version number to start with.
+    Deleting the `var` folder will automatically create it next time the app
+    is started, with the last version number.

lib/bootleg/tasks/build/remote.exs

@@ -294,3 +294,72 @@ task :pull_remote do
 end
 
 before_task(:pull_remote, :verify_repo_config)
+
+# Hot Upgrade support
+task :remote_build_upgrade do
+  build_role = Config.get_role(:build)
+  invoke(:init)
+  invoke(:clean_for_upgrade)
+  invoke(:remote_scm_update)
+  invoke(:compile)
+  invoke(:remote_generate_release_upgrade)
+
+  if build_role.options[:release_workspace] do
+    invoke(:copy_build_release)
+  else
+    invoke(:download_release)
+  end
+end
+
+task :clean_for_upgrade do
+  remote :build do
+    "ls"
+  end
+  |> Enum.map(fn result ->
+    with {:ok, stdout_list, _code, _host} when stdout_list != [] <- result do
+      locations =
+        stdout_list
+        |> Keyword.get(:stdout)
+        |> String.split("\n")
+        |> Enum.drop(-1)
+        |> Enum.filter(fn el -> el != "_build" end)
+        |> Enum.join(" ")
+
+      if locations != "" do
+        remote :build do
+          "rm -rvf #{locations}"
+        end
+      end
+    end
+  end)
+end
+
+task :generate_upgrade_release do
+  invoke(:remote_generate_release_upgrade)
+end
+
+task :remote_generate_release_upgrade do
+  mix_env = config({:mix_env, "prod"})
+  source_path = config({:ex_path, ""})
+
+  release_args =
+    {:release_args, ["--quiet"]}
+    |> config()
+    |> Enum.join(" ")
+
+  UI.info("Generating upgrade release...")
+
+  remote :build, cd: source_path do
+    "MIX_ENV=#{mix_env} mix distillery.release --upgrade #{release_args}"
+  end
+end
+
+task :remote_hot_upgrade do
+  app_name = "#{Config.app()}"
+
+  UI.info("Upgrading #{app_name} to version: #{Config.version()}")
+
+  remote :app do
+    "bin/#{app_name} upgrade #{Config.version()}"
+  end
+end

lib/bootleg/tasks/build_upgrade.exs

@@ -0,0 +1,12 @@
+alias Bootleg.{UI, Config, DSL}
+use Bootleg.DSL
+
+task :build_upgrade do
+  build_type = config({:build_type, "remote"})
+  bootleg_env = config(:env)
+  UI.info("Starting #{build_type} build for #{bootleg_env} environment")
+  invoke(:"#{build_type}_verify_config")
+  invoke(:"#{build_type}_build_upgrade")
+end
+
+before_task(:build, :verify_config)

lib/bootleg/tasks/deploy_upgrade.exs

@@ -0,0 +1,50 @@
+alias Bootleg.{UI, Config, DSL}
+use Bootleg.DSL
+
+task :deploy_upgrade do
+  app_role = Config.get_role(:app)
+
+  if app_role.options[:release_workspace] do
+    invoke(:copy_deploy_release_upgrade)
+  else
+    invoke(:upload_release_upgrade)
+  end
+
+  invoke(:unpack_release_upgrade)
+end
+
+task :copy_deploy_release_upgrade do
+  app_role = Config.get_role(:app)
+  release_workspace = app_role.options[:release_workspace]
+  release = "#{Config.version()}.tar.gz"
+  source_path = Path.join(release_workspace, release)
+  dest_path = "#{Config.app()}.tar.gz"
+
+  UI.info("Copying release archive from release upgrade workspace")
+
+  remote :app do
+    "cp #{source_path} #{dest_path}"
+  end
+end
+
+task :upload_release_upgrade do
+  remote_path = "#{Config.app()}.tar.gz"
+  local_archive_folder = "#{File.cwd!()}/releases"
+  local_path = Path.join(local_archive_folder, "#{Config.version()}.tar.gz")
+  UI.info("Uploading release upgrade archive")
+  upload(:app, local_path, remote_path)
+end
+
+task :unpack_release_upgrade do
+  remote_path = "#{Config.app()}.tar.gz"
+  remote_dest = "#{Config.version()}/#{Config.app()}.tar.gz"
+
+  UI.info("Unpacking release upgrade archive")
+
+  remote :app do
+    "tar -zxvf #{remote_path}"
+    "cp #{remote_path} releases/#{remote_dest}"
+  end
+
+  UI.info("Unpacked release upgrade archive")
+end

lib/bootleg/tasks/hot_downgrade.exs

@@ -0,0 +1,11 @@
+alias Bootleg.{UI, Config, DSL}
+use Bootleg.DSL
+
+task :hot_downgrade do
+  build_type = config({:build_type, "remote"})
+  bootleg_env = config(:env)
+  UI.info("Starting #{build_type} hot downgrade for #{bootleg_env} environment")
+  invoke(:"#{build_type}_hot_downgrade")
+end
+
+before_task(:build, :verify_config)

lib/bootleg/tasks/hot_upgrade.exs

@@ -0,0 +1,11 @@
+alias Bootleg.{UI, Config, DSL}
+use Bootleg.DSL
+
+task :hot_upgrade do
+  build_type = config({:build_type, "remote"})
+  bootleg_env = config(:env)
+  UI.info("Starting #{build_type} hot upgrade for #{bootleg_env} environment")
+  invoke(:"#{build_type}_hot_upgrade")
+end
+
+before_task(:build, :verify_config)

lib/bootleg/tasks/upgrade.exs

@@ -0,0 +1,8 @@
+alias Bootleg.{Config, DSL}
+use DSL
+
+task :upgrade do
+  invoke(:build_upgrade)
+  invoke(:deploy_upgrade)
+  invoke(:hot_upgrade)
+end

lib/mix/tasks/build_upgrade.ex

@@ -0,0 +1,27 @@
+defmodule Mix.Tasks.Bootleg.BuildUpgrade do
+  use Bootleg.MixTask, :build_upgrade
+
+  @shortdoc "Build a release for upgrade"
+
+  @moduledoc """
+  Build a release for upgrade
+
+  ## Usage
+
+      mix bootleg.build
+
+  ## Caution
+
+  Please never try to hot upgrade a running application without 
+  having first a good understand of how a hot upgrade is performed, 
+  its limitations and steps required.
+
+  ## Documentation
+
+  Please see the "Hot upgrading a running application" section 
+  of `bootleg.upgrade` documentation for an overview of 
+  the hot upgrade process:
+
+      mix help bootleg.upgrade
+  """
+end

lib/mix/tasks/deploy_upgrade.ex

@@ -0,0 +1,27 @@
+defmodule Mix.Tasks.Bootleg.DeployUpgrade do
+  use Bootleg.MixTask, :deploy_upgrade
+
+  @shortdoc "Deploy an upgrade release"
+
+  @moduledoc """
+  Deploy an upgrade release
+
+  ## Usage
+
+      mix bootleg.deploy_upgrade
+
+  ## Caution
+
+  Please never try to hot upgrade a running application without 
+  having first a good understand of how a hot upgrade is performed, 
+  its limitations and steps required.
+
+  ## Documentation
+
+  Please see the "Hot upgrading a running application" section 
+  of `bootleg.upgrade` documentation for an overview of 
+  the hot upgrade process:
+
+      mix help bootleg.upgrade
+  """
+end