In a previous article, The Road To Build Enlightenment, we discussed several ways to get more out of your build system. One of the techniques described was automated releases: scripting the process of turning your source code into a package you can ship to your customers. In this article, we look into automating the release process in more depth. We begin by reviewing and expanding on the benefits of automated releases. We then take a look at common steps involved in the automation process, and some of the challenges they may present.
The great thing about automating releases is that it is not rocket science. Although there can be some roadblocks along the way, most tasks are quite easy to automate if you are using the right tools. Taking a few moments to think about and automate the process up front will pay off many times over.
A properly implemented automated process will reliably produce a working package. By avoiding manual steps, you are also avoiding the possibility of human error. Nothing is worse than a small error causing you to ship a broken package!
Automating your release process makes creating a new release package trivial and fast. If, for example, you need to deliver an urgent bug fix to a customer, you can do so easily (and without resorting to short cuts). Packages are made on-demand, without angst.
A package made by an automated process may be reproduced precisely. To allow this, it is necessary to version the packaging scripts with you source code (a good practice in any case). The ability to reproduce packages precisely is critical for customer support. When diagnosing a bug in a specific build of your software (and potentially providing a patched release) it is essential to work from the right base.
The full benefit of continuous integration is realised when you automated releases as part of the process. This allows you to generate a software package as part of your continuous build. This allows full end-to-end testing of the build and packaging process. It also means you always have a working version of your product available to demonstrate concrete progress.
Automated Acceptance Testing
Just like unit tests, you should automated as much of your acceptance testing process as possible. For full end-to-end coverage, your acceptance tests should be applied to the actual packaged product. Automated releases facilitate this by providing a way to integrate the packaging step into your acceptance testing process. Automation of releases and acceptance testing go hand-in-hand: you need automated releases to automate your tests, and you need tests to ensure the quality of your automatically-created packages.
Stages in Release Automation
We assume that you already have a scripted build in process, using a tool such as Ant or make. This build allows your software to be compiled and tested with a single command. This build should output any libraries, binaries and other generated artifacts that are a part of your final release package.
The Release Script
The packaging process for a software product involves a few common operations. These include interacting with other tools to obtain required information, gathering the required files into the package layout, performing simple text processing on some of the files and packaging the files up into an archive and/or installer. The range of tasks required usually goes beyond what can be handled conveniently using a build tool such as Ant or make. In our experience, the best tool for this sort of job is a scripting language. More specifically, a shell scripting language (such as bash) is often the easiest route. Depending on your exact experience and requirements, you may find a more general purpose scripting language such as Python to be more convenient. The release script is easily integrated into your Ant or make build.
Non-trivial software projects make extensive use of existing libraries. Some or all of the dependencies may be bundled in your package to ease deployment for the end customer. Thus the release script must be able to obtain the required dependencies. Two approaches are commonly used to solve this problem:
- Storing dependencies in a known central location, where the changes to dependencies are handled manually. This is a simple and often effective way to manage dependencies. A popular place to store the dependencies is in the source control server, as this allows easy access and is simple way to track versions of dependencies versus versions of your software product. In this case the dependencies will be available in a checkout of the project source, where the release script can easily access them.
- Using a dependency management tool. Examples of such tools in the Java space include Apache Maven and Ivy. These tools allow you to deal with dependencies declaratively. You tell the tool which versions of the dependencies are required and the tool manages details such as indirect dependencies, version conflicts and dependency retrieval. Usually the dependency management tool will be integrated into your normal build process. The release script can call the tool either directly or indirectly (via the build tool) to obtain the dependencies required for a package.
Of course, dependency management is a larger problem in its own right, and is worthy of more in-depth analysis in a future article.
Assigning a Version
Designing a version numbering scheme seems like a simple task, and indeed it should be. In our experience most complications occur when mixing the marketing requirements of the version number with the technical requirements. Technical requirements include:
- No two builds should ever have the same version.
- It should be simple to determine the more recent of two given version numbers.
- At the most basic level, a version number for a new build should be cheap to generate: no marketing meaning or other baggage should be attached.
- For automation purposes, it should be simple to increment the version number to derive the next version.
The most common solution is either a three or four digit numbering scheme. At least three digits are usually required as the first two digits (the major and minor version number) often have meaning attached. The third (or fourth) digit is used as a counter that just increases with every build to ensure the version is unique. This system is simple to automate and effective. It is crucial that the temptation to reuse a version number (just to fix that one little thing...) is avoided. Using an otherwise-meaningless build number removes this temptation as you can just increment the number, even for a trivial fix.
Release notes are an important tool for communicating changes in your software to your users. Useful release notes must be:
- Written in language the user understands. This is the main reason that you should not use the changelog from your SCM server for release notes. The audiences are different: commit messages are for developers, release notes are for end users. Trying to mix the two degrades the quality of information for one or both audiences.
- Accurate: don't try to remember off the top of your head what has changed. Record everything as you go or, better yet, leverage an existing record.
In our experience, by far the most effective way to generate release notes reliably is to use your bug tracking software. Your bug database should already include a record of all changes made, be they bug fixes, feature requests or whatever. Even better, bug summaries and descriptions should be written in terms the user understands, making them ideal for release notes. Good bug tracking software will have the ability to generate release notes built in. At a minimum your bug tracker must be able to process a query that returns the relevant fixes for a build. Tie this query in to your automated release script and viola: you have auto-generated release notes.
A piece of the release puzzle that can be neglected is the associated documentation. Typically the documentation will need to be updated to reflect the changes in a new version. Thus it makes sense to make a release of the documentation at the same time as the software itself. This documentation can then be included in the software package or marked with the full version number to match it with the software build. For documentation built from a text source, marking the version can be achieved by simple substitution: keep a placeholder string (e.g. __version__) in the documentation source and substitute it at build time.
Unfortunately, many documentation tools use a binary format that can be more difficult to process automatically. If you are stuck with such a format, consider leveraging versioning capabilities of the authoring software itself. If the authoring software has its own macro language (such as VBA in Microsoft Office), you may be able to use it to perform automatic edits of the documentation.
The simplest way to distribute software in this day and age is via a single file download. The file is typically an archive or executable installer. Creating a distributable archive is typically as simple as invoking a command line archiving tool. Creating an installer often depends on a specialised third party product. There are key requirements to keep in mind: any tool used to create installers must be scriptable, and must be easy to integrate into your automated release scripts. A tool that requires human interaction to package software should be avoided as a drain on productivity.
The final step in the automated release process must be testing of the newly-created package. Testing the source code just before you decide to release, although sensible, is not good enough. There can still be bugs in the packaging process itself that can render the package unusable. Such bugs are likely to be found instantly by an end user, leaving a distinctly bad impression. The best way to avoid these problems is to test from the package itself: emulating the experience of the user.
Testing a package unavoidably includes unpacking, installing and setting up the software from your script. Ideally, this process will follow the same steps as a first time user of your software. This may include driving a graphical user interface using a testing tool. If this is infeasible, consider adding specialised support to your software to allow it to be set up in a scripted manner. Automated setup can itself be a boon during development, allowing you to quickly set up a fresh instance of your software for ad-hoc testing.
That's it! As stated up front, it is not rocket science. We hope that after seeing how simple each step is, you will be motivated to automate your release process. Be assured that once you are churning out perfect releases with a single command you will not regret it!