Temporal Release Orchestration Demo App

This is a simple Temporal demo in Python that orchestrates a release process, made up of one to many deploys, to one to many environments.

Releasing is a Troublesome Process

A release is a complex process: orchestrating multiple failure-possible subprocesses, with possible human interaction and approval. Conceptually, you want to "do a release", which is often a series of manual steps:

1. Gather and document what is to be released, load it into the brains of the people doing the release
2. Verify that things are approved and good to go
3. For each thing to deploy in each environment we're releasing to:
   1. Do builds (if not build yet)
   2. Gather artifacts
   3. Gather configs
   4. Deploy artifacts
   5. Update configuration
   6. Verify deploy
4. Summarize Release success

Output: environments are updated with new artifacts, and the process was successful and well-understood.

Manual deployment is often error-prone, as people aren't really great at doing the repetitive complex tasks perfectly over and over under pressure.

Automation: Make the Robots Do It

All of these steps could be automated, and they often are at least partially automated. Automation improves success rates, repeatability, and speed, but reduces visibility. If the automation fails it can be hard to figure out why or what happened vs. a manual process, assuming manual operators are perfect at validation and detecting their errors.

Making The Process More Visible and Durable

Temporal's Durable Execution model adds two interesting pieces to a release & deploy process:

1. Insight
2. Durability

Insight

Developing in the Temporal Workflow & Activity model applies a natural organization to code: Workflow: The orchestration goes here. Order and control goes here. The repeatable, deterministic process model goes here. Nothing that can break goes here. Activity: The stuff that does stuff. Breakable steps go here. Should be idempotent. Can call other existing methods/services/functions.

Writing code in this model leads to a natural simplicity and understandability: you can follow the workflows as they execute in the Temporal UI: (todo add screenshot here)

Durability

This demo demonstrates the durability of a process implemented in Temporal:

1. Crashing the process doesn't kill it. Upon resume it picks up right where it left off.
2. Errors are recovered without thought or work
3. At-Least-Once execution: activities succeed at least once per the workflow

These capabilities are great to develop with and change the way I think about doing development.

As a developer I can focus just on what I want to do, and Temporal manages what happens when things don't work out.

While working on this project, I created many bugs in my activities, and all I had to do to fix my in-flight releases was fix the code bugs and restart the worker process. The errors went away and none of the workflows failed, they all succeeded.

Zero workflow processes failed in the building of this application.

Process Results

This demo will create "artifacts" created by a "build" system. For the demo, these are just empty files in the environments folder. The artifacts are built according to the rules of the release process. The demo will also deploy these artifacts to the environments, and orchestrate a release made up of a set of related deployments.

This release process executes durably, and is guaranteed to finish once it begins per the rules specified in the workflows.

Getting Started

See Setup Instructions.

First Demo

After the setup is done, you can do the basic demo described in the setup instructions. You can see an order get processed, maybe fail randomly.

Next Steps

1. Check out the ways to demonstrate that this works nicely
2. Play around with the code in new ways, try to break Temporal, maybe try some retry policies
3. Feel free to fork and contribute!
4. Implement your release process with Temporal and enjoy Durable Releases.