EPICS V4 Branched Development with hg

EPICS V4 Working Group, Working Draft, 04-Nov-2013

Latest version:
using_hgflow.html
This version:
using_hgflow_20131104.html
Previous version:
using_hgflow_20130828.html
Editors:
Ralph Lange, Helmholtz-Zentrum Berlin für Materialien und Energie / BESSY II
License:
Creative Commons BY-SA

Abstract

The Mercurial plug-in hg flow implements a generalized Driessen branching model. This document describes the basic feature of the model and how hg flow and its high-level commands can be applied to EPICS V4 development repositories.

This document is accompanied by a shell script that executes all procedures mentioned. The script itself can be used as a reference, the sample repository is a nice disposable sandbox.

Status of this Document

This document describes the suggested hg flow procedures for EPICS V4 development as of 2013-08-28. It is not complete, and is intended to be a background for informed discussion and decision.

TODO

describe working with hotfix branches.

Describe working with support branches.

Describe --dirty option.

Describe procedures for multiple master trunks, i.e., parallel development of several versions.

Table of Contents

Introduction

Since modern distributed revision control systems have gained popularity within software developers, many branching and workflow models have been developed, which differ in features and grade of adoption.

A particular model that has gained wide-spread popularity is the workflow developed and described 2010 by Vincent Driessen, originally for the Git version control system, referred to as the Driessen branching model.

For the Mercurial community, Yujie Wu develops and supports a Mercurial plugin called hg flow that implements a generalized version of the Driessen model. It offers convenient high-level commands for operations on branches, avoiding the complex and mechanical low-level operations needed to achieve the required operations.

This document uses the original illustrations by Vincent Driessen.

Why Branched Development?

For larger projects with many developers, a model based on one development line in a central repository is not working well. Processes like large feature developments (with multiple developers involved) and the preparation of a new release (going through phases of pre-releases and release candidates) can take weeks and would block the development of other features or important bugfixes. Support of older versions is hard. Tracking bugfixes across releases is tedious.

Features of the Generalized Driessen Model

Vincent Driessen published a branched development model for Git in a blog article in January 2010. His original model was later extended with the addition of support branches allowing long-term support of older releases. The stream generalization allows to group branches in a directory-like fashion.

When mapping this Git-based model to existing Mercurial repositories with a history, is seems appropriate to use the default branch (with the development history) as develop, and create a new branch called master for generating the releases.

Driessen Branching Model

There are two main trunks with an infinite lifetime (i.e., created at initialization and never closed):

There are four additional streams (groups of branches), which carry a varying number of branches (i.e., branches are created and closed as part of the workflow):

Sub-streams of develop and feature can be created in a directory-like hierarchy, to collect branches that belong together. E.g., the developments during a certain code sprint or codeathon could go into a develop sub-stream, or different parts of a complex feature could be developed in a feature sub-stream.

The following table lists the different types of branches in the Driessen model, with their most important properties.

Branch Types in the Driessen Model
Name Purpose Properties Created from Merged into
develop trunk main line "next release" development minor features (~2 commits or less) are done directly on develop, major features or developments for a future version are done on a feature branch - -
master trunk main line of releases every commit is a release - -
support branches support releases (for older releases) long term support (e.g., bugfixes) for older releases, every commit is a release master trunk (at the appropriate release) -
develop branches grouping developments developments for a specific release, or during a specific sprint/codeathon develop trunk (or develop branch) (its parent branch and) develop trunk, closing creates a release branch
feature branches feature development each major feature uses a dedicated feature branch for development develop trunk (or develop branch) (its parent branch and) develop trunk
release branches release preparation contains all changes that are necessary for a specific release, between first pre-release and the final release development trunk (or develop branch) master trunk and (its parent branch and) develop trunk
hotfix branches fixing bugs fixes for urgent bugs in released versions master trunk, or support branch master trunk and develop trunk, or support branch

Within the context of hg flow, the term stream is used for either a set of related branches, either at the top level or at the branch level, or for an individual branch. Some hg flow operations can be applied as well to an individual branch as to a stream, i.e. to all individual branches that belong to that stream.

Features of this branched development model that might or might not be present in other models:

Simplified Branching Model for EPICS V4 Development

The full Driessen model seems too complicated for use in V4 development. Only a small subset of the described features and streams is needed to support the current workflow. If streams and features are considered useful at a later time, they can be added as needed.

EPICS development does not follow the idea of one strict release branch that all releases can be created from. Usually, any EPICS release has long-time support and receives bugfixes as well as bugfix releases, each of which may or may not undergo a releasing procedure with multiple tagged pre-release versions.

To support this approach, the functionalities of release, master, hotfix, and support branches are combined in the release branches, which are not closed/merged, but left open to receive bugfixes and provide bugfix releases.

Branch Types in the Simplified Model
Name Purpose Properties Created from Merged into
develop trunk main line "next release" development minor features (~2 commits or less) are done directly on develop, major features or developments for a future version are done on a feature branch - -
feature branches feature development each major feature uses a dedicated feature branch for development develop trunk develop trunk
release branches release preparation, bugfixes, bugfix releases contains all changes that are necessary for a specific release, through the release process, bugfixes, bug fix releases, long term support develop trunk -

This maps nicely onto the current workflow:

Advantages:

Usage Examples

After explaining the installation and setup of the hg flow plugin for Mercurial, the Basic Procedures section covers the initial workflow operations. These include the mandatory develop trunk, developing on feature branches, and releasing on release branches.

The Advanced Features section describes operations on that can be introduced as needed. This section is not complete.

Sample Script

You can execute the sample script, which will create a repository in a subdirectory called sample, and run all described operations using a small set of text files.

Installation and Configuration of hg flow

Use of the hg flow plugin is not mandatory.

Install hg flow (>= 0.9.6) according to the installation instructions:

Basic Procedures

The tables containing the command examples always show the basic hg commands (without using the hg flow plugin) in the left column, while the hg flow commands are shown in the right column.

Be Safe

Branched development is no picnic. Be aware of structures and procedures.

Back up your repository (local copy using cp -a) before complex operations.

Every hg flow command takes --dry-run as an option to print out what the command would do, without changing the repository. Check if it will do what you intend to do.

Use the built-in help function: hg flow help will get you started.

Initialize hg flow for an Existing Repository

The EPICS V4 repositories will have an existing development history in the default branch when being initialized for use with hg flow. This needs a small change to the default setup, as we will use the 'default' branch for the develop trunk, and create a new branch 'master' for the master trunk.

In the top level of your repository, run the hg flow init command:

> hg init
> hg init
> hg flow init
flow: Global configuration:
flow:   autoshelve: on

Branch name for master stream: [default] master
Branch name for develop stream: [develop] default
Branch name for feature stream: [feature/] 
Branch name for release stream: [release/] 
Branch name for hotfix stream: [hotfix/] 
Branch name for support stream: [support/] 
marked working directory as branch master
(branches are permanent and global, did you want a bookmark?)
0 files updated, 0 files merged, 0 files removed, 0 files unresolved

Navigate Between Branches

You can switch to any branch or trunk using the hg flow <stream> command:

> hg update default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
> hg flow develop
flow: Update workspace to <develop> trunk.
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
flow: <develop> trunk: default
flow: Open <develop> branches:
flow:   default*
> hg update feature/addFile3
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
> hg flow feature addFile3
flow: Update workspace to <feature> 'addFile3'.
1 files updated, 0 files merged, 0 files removed, 0 files unresolved

Simple Development on Develop Trunk

Switch to the develop trunk:

> hg update default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
> hg flow develop
flow: Update workspace to <develop> trunk.
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
flow: <develop> trunk: default
flow: Open <develop> branches:
flow:   default*

Edit and commit as usual.

Development on a Feature Branch

Create a feature branch off develop trunk:

> hg update default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
> hg branch feature/addFile3
marked working directory as branch feature/addFile3
> hg commit --message "Create branch 'feature/addFile3'."
> hg flow feature start addFile3
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
marked working directory as branch feature/addFile3

Feature Branch

Edit and commit as usual.

When feature development is complete, finish the branch (first you might have to navigate to the branch you want to finish):

> hg update feature/addFile3
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
> hg flow feature addFile3
flow: You are already in <feature> 'addFile3'.
> hg commit --message "Close <feature> 'addFile3'." --close-branch
> hg update default
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
> hg merge feature/addFile3
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
> hg commit --message "Merge <feature> 'addFile3' to <develop> ('default')."
> hg flow feature finish
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
1 files updated, 0 files merged, 0 files removed, 0 files unresolved

The feature branch is closed, and merged back into the develop trunk.

Release Process

Create the release branch off develop trunk:

> hg update default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
> hg branch release/1.0
marked working directory as branch release/1.0
> hg commit --message "Create branch 'release/1.0'."
> hg flow release start 1.0
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
marked working directory as branch release/1.0

Change, commit and tag preparing for the release on the newly created release branch. This includes all changes necessary for pre-release and release-candidate stages, up to preparing the final release. At any point, changes and/or bugfixes on the release branch can be merged back into develop:

> hg update default
3 files updated, 0 files merged, 3 files removed, 0 files unresolved
> hg merge release/1.0
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
> hg commit --message "Promote <release> '1.0' (b009fb5075d1) to 'default'."
> hg update release/1.0
2 files updated, 0 files merged, 1 files removed, 0 files unresolved
> hg flow release promote default
3 files updated, 0 files merged, 3 files removed, 0 files unresolved
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
2 files updated, 0 files merged, 1 files removed, 0 files unresolved 

The release branch is never closed, and works as a bugfix and support branch for this release. I.e., in this example it will carry the releases 1.0.0, 1.0.1, 1.0.2, ...

Advanced Features

Autoshelve

Developing using a branched model often leads to situations where you want to switch to a different branch while having uncommitted changes in your working directory that you want top keep with the original branch.

The autoshelve feature (see Installation section) uses Mercurial's mq extension to automatically shelve your uncommitted changes when you leave a branch, and unshelve them when you return. It tracks your uncommitted changes for each open branch.

Start Branch from Dirty Workspace

The --dirty option allows to carry uncommited changes in the workspace onto a newly created branch.

Assume you have started to work on a feature on develop trunk, and at some point you realize you rather want to develop it in a feature branch. Using this option you can create a feature branch and move your changes onto the new branch in one step:

This section is not complete.


Ralph Lange, Helmholtz-Zentrum Berlin für Materialien und Energie / BESSY II
2013-11-04