Managing Branches in GitHub

NOTE: This page is an early draft. Pay it no attention until this notice is removed.

Introduction

Following the release of perfSONAR 4.1, the development team adopted a more formal scheme for version control during development. The primary goal of this scheme is to facilitate development on multiple software versions simultaneously without adding too much overhead for developers. We aim to accomplish this goal with the following:

• A clearly defined branching strategy that cleanly separates code from different versions to prevent the accidental "leaking" of features and incompatible code
• A set of tools for automating much of the process to reduce developer overhead and ensure the defined procedures are followed

This document will look at the high-level branching strategy, how to make a change to the code and how to implement the strategy when performing releases.

Branching Strategy

Basics

The following are the basic tenants of the branching strategy.

• Each release version currently under development has a dedicated branch named using the version number (e.g. 4.1.3, 4.2.0, 5.0.0). These branches will be referred to as release branches.
• Any changes that need to be made will be performed in a separate branch created from one of the release branches and named for the issue or feature being implemented (e.g. issue-1234). These will be referred to as feature branches
• When it is time for release, the release branch will be merged into master and any later release branches. Master will be tagged and the release branch will be closed. No further changes should be made to the closed release branch. A new release branch will be created for the next version and the process will repeat.

Naming Conventions

The following naming conventions should be followed when naming branches:

• Release Branches - Named using the version they represent in the form of MAJOR.MINOR.PATCH. All three parts MUST be included. Examples: 4.1.0, 4.2.1, 5.0.0
• Feature Branches - Feature branches should be named for the issue they represent. They should start with the prefix issue- followed by the issue number. For larger changes that will spend significant time in a branch the it is valid to reference an epic issue that will consist of many smaller issues. It is not required each sub-issue of the epic get its own feature branch when working in this manner. Examples: issue-1, issue-2

The following naming conventions should be used when creating tags on master:

• Release Tags - When a release branch is merged into master, the commit of the merger should be tagged with the letter "v" followed by the version number. The "v" is added to prevent conflicts that can occur when tags and branches have the same name in git. Examples: v4.1.0, v4.2.1, v5.0.0

Making a Change

All of the above can seem like a lot, but in practice it shouldn't be too difficult for a developer making changes to the code. The following are generally the steps for making a change:

1. Create an issue for the change you wish to make if it does not already exist.
2. Identify the earliest release for which the change needs to be applied. This information should be in the issue. If it is not, you may want to consult with the development team. Alternatively, a good rule of thumb is to apply it to the latest open release branch since the team is generally less conservative about changes going into later releases.
3. Checkout the source code of the component that needs a change and create a new branch from the target version of the code. Examples commands are below where the component is psconfig, target version is X.Y.Z and issue number is 1:

   git clone https://github.com/perfsonar/psconfig
   cd psconfig
   git fetch
   git checkout X.Y.Z
   git checkout -b issue-1
   git push origin issue-1
   

4. Make changes to your new branch
5. Check if the target version has had any changes since you began working on your change. If it has run git merge X.Y.Z on your feature branch. This helps avoid conflicts at pull request time.
6. Submit a pull request to merge your change back into the target version.

From there your pull requests can be reviewed by the development team and merged back into the code.

Handling Community Pull Requests

It is unreasonable to assume a community member that is not part of the day-to-day development team submitting a pull request will have intimate knowledge of our branching strategy or which version their change applies. The good news is that they do not have to have that knowledge and GitHub provides tools to get changes directed in the right place.

It seems likely that most pull requests from the community at large will be based off of master since that will be the latest released code. It also seems likely that pull requests may try to merge into master, which we never want. Luckily GitHub allows maintainers of the project to change the target branch of a pull request by selecting Edit next the pull request title and then selecting the target branch from the pulldown. If the change will not merge into the desired branch, the reviewer should share that feedback with the community member.

In summary, community generated pull requests should be merged in the same as any other pull requests, but the reviewer may need to pay extra attention to the target branch and adjust it accordingly on behalf of the submitter.

When to Merge

The process clearly defines that a branch is merged into both master and future release branches at release time. It is also highly recommended that future release branches periodically merge in changes from earlier open release branches to help catch conflicts early. Failure to do so can lead to surprises at release time including source code conflicts or bugs. No timetable is explicitly defined, but in general it seems good practice to never go more than a week without merging in changes from earlier releases while both are under active development.

Release Procedure and Tools

Overview

When it is time to perform a release the following procedure must be followed:

1. Identify which git repositories contain changes in this release using the prepare-release command. If a repository does not contain changes from the previous release then it will NOT be tagged and built.
2. Create pull requests and merge each release branch with updates into master
3. Tag each master branch according the tagging naming conventions defined in this document
4. Create pull requests and merge each release branch with updates into any open future release branches.
5. Run the finalize-release command to close all release branches for the given version (including those that have no changes) and to create the required future release branches.

The sections that follow explore these steps in greater detail.

Installing the Tools

If you have not done so already, you should install the perfsonar-dev-tools from GitHub that contain a set of utilities for performing release actions. You can clone these tools with the following command:

git clone https://github.com/perfsonar/perfsonar-dev-tools

You may have everything you need at this point and can skip to preparing the release. The downloaded repository does however contain a Vagrant environment for running the scripts. Using the vagrant environment is optional as the tools should run on any system with Git and a shell including Mac and Windows (using the Git bash shell). You can access the Vagrant environment with the following commands:

cd perfsonar-dev-tools
vagrant up
vagrant ssh

Once logged-in to the vagrant environment, you will need to configure git with your access credentials so you can push to the repository( e.g. add you username and password to /home/vagrant/.netrc). All the scripts in the remainder of this document can be found in the shared directory /vagrant/bin.

Identify repositories changed in the release

In this step you will use a script to identify which projects changed from the previous release and need to be merged and tagged. The script does not make any changes to the repository, it simply compares the provided release branch name to master. Below assumes you are preparing version X.Y.Z and using the vagrant environment, so if not using vagrant your path may be slightly different:

/vagrant/bin/prepare-release X.Y.Z

The output should contain a list of repositories that have been updated.

Merging Release Branches

Now begins the manual portion of the process where you visit each listed project to create a pull request to merge into master AND any already open future release branches. It is critical you complete the pull requests and merge prior to finalizing the release as a failure to do so will result in the merger of files that indicate the branch is closed (which we do not want).

Tagging master

Once merged you should go to each master branch where a merge occurred and tag according to the naming conventions. For example:

git tag vX.Y.Z

Finalizing the release

Once everything has been merged and tagged, we can now close the existing release branches and create the new ones. This can be done with a single command below:

/vagrant/bin/finalize-release X.Y.Z

The command will iterate through each project, create a BRANCH-CLOSED file and create the next release branches as necessary. This will always include the next patch release and may include the next minor release if the branch does not already exist. If you have made it this far, congratulations you have successfully released the perfSONAR source code!

Optional: Enforcing closed-state on Client

See the perfsonar-dev-tools README for information on how to install client-side git hooks on your local machine. These are not strictly required but do prevent accidental commits to a closed branch.

Optional: Enforcing closed-state on GitHub

We may make this a requirement at some point, but to make sure the closed state gets enforced you may perform the following manual steps (if we are able to automate this in the future, it will likely become a hard requirement)

1. Log into GitHub using an administrator account.
2. Go to the repository that is to be closed.
3. Select the Settings tab.
4. Select Branches in the left-side pane.
5. Click Add Rule.
6. Apply the rule to the name of the branch being closed (e.g., 9.1). Note that wildcards are possible, so the number of rules can be kept down by using them. (E.g., if all of the 9.x releases are closed, a single 9* rule would cover all of them.)
7. Select Include administrators.
8. Select Restrict who can push to matching branches. Put only the release administrator, arlake228 on the list.
9. Click Add. Note that the change may take a few minutes to take effect.

Example Branch Workflow

The diagram above shows an example of the life-cycle of the various branches. Starting from the top of the diagram you'll see we begin with two branches: the master branch and the 4.1.0 release branch. We also want to create a 4.2.0 release branch so the next major release can be developed in parallel. This branch is created from the most recent release branch, 4.1.0, as shown by the black-dotted line at the top going between the two branches.

Next, an issue is identified that we want to go in versions 4.1.0 and later of the software. A feature branch is created called issue-1 from the 4.1.0 branch since it is the lowest version we want to have the feature. The solid blue lines in the diagram shows the life-cycle with the first blue arrow being the branch creation, the second being a commit in the issue-1 branch with the change desired and the final blue arrow being a merge back into the 4.1.0 branch. At that point, the issue-1 branch is closed and no further changes are made to the branch. This process is followed by three dotted blue lines representing merges of this change to other branches. For now we will just look at the first of these lines. This first dotted blue line is a merge of the change into the 4.2.0 branch since we want this fix in that version as well. It is not strictly required for the merger to happen at this point, but it is highly recommended as doing this often will help identify any potential conflicts early in the development process. Speaking of the 4.2.0 branch, let's look at what was happening over there while we worked on issue-1

Simultaneous to the creation of the issue-1 branch, a second issue that we only want in version 4.2.0 is identified. A feature branch called issue-2 is created. The solid red lines follow the lifecycle of this branch with the first being the creation of the branch from 4.2.0 and the second being a commit to the issue branch. With the issue now resolved we want to merge back into the 4.2.0 release branch...but there have been some changes to that branch since we originally created issue-2! In order for there to be a clean merge we first need to merge the latest changes from the 4.2.0 branch into issue-2 as shown by the dotted blue line. When that is completed, we can cleanly do the reverse direction and merge issue-2 back into 4.2.0

Back on branch 4.1.0, we decide we are now ready for release. First we merge our changes into master and create a tag called v4.1.0. We then create a new branch called 4.1.1 to hold changes for the next minor revision. The 4.1.0 branch is then closed and no further changes are committed to that branch.

The life-cycle for 4.1.1 is very similar to that of its a predecessor. Another issue is identified and spun into a branch called issue-3 and then merged back in to the 4.1.1 release branch. When 4.1.1 is ready for release, it is merged into master, tagged v4.1.1, and merged into 4.2.0. At that point 4.1.1 is considered closed as well.

After 4.1.1 it is identified that we are ready to release 4.2.0 As such it is merged into master and tagged. Not shown is the fact that 4.2.1 and 4.3.0 released branches are created so the process can continue.