One thing that is often overlooked in smaller development organizations is the build process for your software. If you’re not a software developer, the build process is the sequence of steps that takes your source code and creates a final package that is delivered to your customers (or to your servers). 

Large organizations have been automating their build process for many years, for the simple reason that the build process has to work for numerous software developers. It has to work on everyone’s machine, every time, and produce reliable results.  Therefore, there is not a large step between the amount of documentation required to communicate this process to everyone and simply automating it. However, in smaller organizations, there is a temptation to bypass build process automation as an unnecessary piece of overhead. The logic goes like this — if Sam is the only one who works on product X, then he’s the only one who needs to build it. As long as the build system works easily for Sam, that’s all that matters. However, there are many advantages that build process automation brings even to single-developer organizations. Below I will show you the components of such a process and why they are important to any development organization of any size.

While there are several ways to construct an automated build process, every build process starts with version-controlled source code. Version-controlled software with frequent commits should be the foundation stone of every development organization. The automated build process should only build directly from the version-controlled source code repository. This forces developers to use the system and makes sure that all code for all builds have been properly checked into the repository. I can tell you about a number of occasions where I have had to track down code that some developer forgot to commit that was in the release of a product. I’ve had to search the backup hard drives of developers who have left an organization to find the code that was actually delivered. Additionally, when code has to be committed before the build occurs, it means that you can reliably find which versions of code went into which releases. Was the bug introduced between version 3.5.1 and 3.5.2? If these were built from the version control system, getting a list of all changes between these two versions is trivial.

The next thing needed for an automated build process is the build process itself.  This is a script or collection of scripts that actually perform the build. Having an automated process forces the developer to actually write down explicitly all of the things required to do to build the process. I can’t tell you the number of times that a developer told me that “the build process is straightforward” when in reality it required knowing to perform one or more non-standard steps.  Having an automated build process means that the developer has to write all of these steps into a script, which is then easy to inspect.

Standardized Environment

Another important aspect of build processes is to have a standardized environment. Two developers can have the same code, run the same build steps, but it compiles for one developer and not the other. Is it the wrong version of the development tool? The wrong version of Windows? Is there something that one developer has installed but the other one doesn’t? Tools like Docker allow you to create and even have a recipe for, your exact build environment. Using Docker to run your build environment allows you to specify (and version control) the exact components you need for your build environment, but also a bit-for-bit image of the build system. This leaves no ambiguity for how your code was built. For instance, let’s say there is a security bug in version 2.1.1, but you are working on releasing version 5.6.2. Oftentimes, development environments will have changed enough that you don’t even remember what tools you needed to have installed for version 2.1.1. However, if you were using Docker as part of your automated build system, the entirety of the toolchain would be documented for each version.

Finally, there is the versioning process itself. One annoying task of development is making sure that the correct version number gets attached to the code. This task can be automated with an automated build system. I usually set up my build systems so that tagging a repository version with a specifically formatted tag (such as release-1-2-3) will do multiple important steps. First, it will cause the automated build process to set the version information of the release to 1.2.3. This is usually done by a shell script which modifies some constants in the code to set the version information. Additionally, the automated build process tools usually have a build number that can be used as well. Second, I often have the automated build process store the resulting code in a special location that is itself versioned. For instance, if it is a web application, I might build a Docker image and store it in a Docker repository that is tagged with the version information as well.

So what tools do you need to get started doing automated builds? The fact is, the tools to do this are readily available.  Most version control software has this sort of tooling embedded in the software. Github has “Github Actions” and Bitbucket has “Bitbucket Pipelines”. There are also other tools such as CircleCI which you can connect to your repository and will perform similar functions. If you want to manage it all yourself, the open-source tool Jenkins is readily available. Personally, I’ve had the most experience with Bitbucket Pipelines, and have been extremely pleased with it.

One cool trick that the automated build process enables is the ability to make small changes without even having access to a computer with the right tools available. Since the automated build process has everything needed to perform builds, you can actually make simple changes directly in the repository via the web, and have the automated build system build the final product. While not the most exciting result of an automated build process, it sometimes comes in handy if small changes are needed when developers don’t have access to their computers.

Continuous Integration/Deployment

Having automated builds also allows other tasks that can be highly beneficial to a development team, known as CI/CD. CI/CD stands for “Continuous Integration / Continuous Deployment”, which are two tasks that can be added to an automated build process. Continuous Integration refers to the ability to automatically perform tests with your automated builds and report on test results. The goal here is a check to the development process to make sure that, at least on some branches, developers aren’t checking in code that is causing tests to fail and thus cause problems for other developers on the project. It basically continually tests the results of the collaboration of your developers and lets everyone know when there is a problem.

Continuous Deployment allows you to do full deployments from your build system, whether it is to a website or to an app store.  Personally, I don’t like the build system itself performing deployments, but I tend to have the build system get a project deployment-ready, so I only have to click a single button or perform a single command to make everything live. For instance, for web projects released to Amazon Web Services, I like to have my continuous deployment process build a docker image of the resulting web application and then send it to Amazon’s Container Repository tagged with the release version. Then, I only have to change the name of the image for the container task to start the deployment process. Note that I also keep my Amazon configuration in version control so that I have a record of which versions were released and when.

In all, automated build processes allow for the standardization and systematization of your development pipeline. This forces development teams to make all steps of their build process explicit, repeatable, and auditable for every version of the software. Using automated build systems forces developers to release products the “right” way without cutting corners, while adding advantages to both the developers and the organization. Whether your development organization is a single individual or a large team, automated build processes provide numerous benefits to your group.