Build Reproducibility

When designing a build process, you should design a way that the build process can be re-run for a particular build.  The easiest way to do this is to divide each step into one of three categories:

1. Master
2. Build
3. Test

Build category

Any step that is required to build the software, documentation and installers are those that fall under this category.  These steps are heart of the build process.  The build steps should be unaware of source control, versioning, build output distribution, etc and should assume that everything it needs is on the box it is running on.  These build steps should ideally also contain only relative paths making them location independent.

If you had all of the build steps contained within its own build file or able to be executed by themselves, then someone who had all the build software installed on their computer should be able to fetch the source code himself, execute the build and have the same output that he would if he ran the same thing on the build computer.  Also, in the event of a catastrophe where you lose a build, you simple restore the source code for that particular build number and run this file again, then when it is done running, you should have the same output as you did when that build was originally run.
 

Test category

Any step that tests the build belongs in this category.  These steps should, like the those steps in the build category, be unaware of source control, versioning, build output distribution, etc and should assume that everything it needs is on the box it is running on.  These build steps should also ideally contain only relative paths making them location independent.

Master category

The steps contain in this category are those that fill in the gaps between the build and test category.  These steps include maintaining and incrementing the version number, pre-build modification of the source code (for example updating resource files with the most recent translations), tagging or labeling the source code, fetching the source code (preferably from the tag or label), executing the build and test steps and distributing the output of the build (such as making the installers publicly available or at least available for QA).  The master build file should also be the one who actually updates any build status application such as a build status web page.  It is often beneficial to have notices on the individual steps of the building of the applications, so you may have to develope your build category steps to take in a flag as to whether it should output its status to an external source but should default to not update that external source.  This way, only when the Master build steps execute the Build steps will they actually notify others of its status.

Example:
Here's a quick example of how this works.  Say I have a .NET project that I build 1 solution, generate 1 help file and make 1 installer.  This project has a database that the build process automatically generates the install scripts for.  This application also works for multiple languages so I have a database in which I keep all the translations for my software which the build process reads and updates all of the resource files in the project.  Below is how I would define the build process.

Master Build File

• Increment the build number
• Read the latest translation data from the database and update the resource files.
• Commit changes to build number and resource files to the database.  Note: at this point, since I have truely consumed another build number, I can consider the build to actually have started.
• Tag/label the source code with the new build number.
• Fetch the source code from the tag/label.
• Execute the Build file
  
• Save the installers and binaries from the build to a public share for others to pick up
• Execute the Tests file
• Distribute the results of the tests.

Build File

• Generate Database install scripts
  
• If executed by the Master build file, update status
• Build the .NET solution
• If executed by the Master build file, update status
• Build documentation
• If executed by the Master build file, update status
• Build installer
• If executed by the Master build file, update status

Tests File

• Test Database install scripts
  
• If executed by the Master build file, update status
• Build and run unit tests on .NET solution
• If executed by the Master build file, update status

Reproducibility
Now imagine that catastrophe strikes and our build machine dies.  Luckily we have instructions for setting up a new build environment to be the same as the old one.  The instructions contain all of the software and their versions along with the keys needed for them.  So within a day we have a new build environment up and running.  Now I need to recreate version 1.0 of our software so I create a revision for it.  I fetch the source from the tag that we made for v1.0.  If I've defined the build process properly, I can simply run the Build File and have output that is nearly the same exact output that we had for v1.0.  Hopefully the only difference will be dates of files.