The Make Dilemma

In the Unix/Linux community, Make is by far the most common tool used to manage the building of software. It has many great features, but it also has many drawbacks. Over the decades that I have used Make, I have often desired to switch to a better solution, yet I still use it.

Benefits

Make was design to manage complex dependencies. In a language like C, many compilation and linking commands may need to be run in a specific sequence in order to build software. The declarative design of Make provides a way for developers to specify dependency constraints so that software can ensure that the commands are executed in an appropriate order.

In the Linux community, Make plays a very important role in building software across the many architectures and systems where Linux is used. GNU Make and GNU Autotools (Autoconf, Automake, and Libtool) handle extremely complex build constraints for large software projects that we rely on every day.

Though Make is best at dealing with complex dependencies, many people use it in cases where it does not need to manage dependencies at all. The development environments for many programming languages provide build tools that do that job. Developers still use Make, however, because of another major benefit: Make is so ubiquitous that it is the de facto standard interface for managing projects.

Linux users are generally very familiar with running the following sequence of commands to build and install software:

# ./configure
# make
# make install

Running make without an argument runs the default goal, and a Makefile is usually written so that it builds the software. Running make install installs the software. There are many other commands that have widespread usage, such as running make test to run tests.

I personally use Make in order to provide a common interface for managing projects that are implemented using a wide variety of languages and tools. I use Base to configure the development environment for each project, and Make provides a common interface that works within the configured environment. There is no need to remember/execute special commands that are particular to specific projects, and I also provide a make help command to easily discover what commands are available in addition to the common ones.

One of the reasons why Make is used so often is that it is so widely available. For example, it is part of the build-essential meta-package on Debian-derived distributions. It therefore tends to already be installed in environments where software needs to be built. The dependency on Make for building is therefore often seen as “free,” while using different software requires adding that software as a new dependency.

Drawbacks

The vast majority of Makefiles that I write do not make use of complex dependency management. Some people argue that this is a misuse of Make, but I think that it is unfortunate that the de facto standard for building software does not have good support such a common use case. Others have discussed drawbacks (example, another example) when using Make as designed, so I will focus on the drawbacks that I find most frustrating when using Make as a common interface for dispatching commands.

To me, the biggest drawback of Make is the language. Both variables and functions are implemented using a very basic type of macro that does recursive text-substitution. Since macro substitution is done before parsing, parsing itself is dependent on the value of variables, which can be specified on the command line when running a command. The syntax is quite tricky, especially when you have to be careful about when macros are expanded. The built-in functions are quite limiting, and writing inline shell scripts (with lots of semicolons and backslashes) results in a mess of code. Error handling is particularly poor, which leads to difficult debugging. The many pitfalls makes development and maintenance time-consuming as well as frustrating.

Note that I am not an expert on Make, and I cannot be certain that everything that I “know” about it is correct. Even after using it for many years in many different ways, I still find it tricky to use. I find it frustrating that even things as simple as conditionals and iterations/loops can be so difficult.

One drawback that I avoid is the use of the tab character for indenting the commands that implement a rule. I use the terminal for development, and I use my mouse to copy and paste text, but this method of copying text does not copy tabs because the tabs are rendered as spaces in the terminal. It is frustrating when the syntax of a language is incompatible with such basic computer usage. Thankfully, GNU Make provides a .RECIPEPREFIX setting that allows you to avoid the use of tabs.

The Dilemma

Should I continue to use Make in my projects? On one hand, the familiar interface is user-friendly. On the other hand, the above drawbacks make it extremely painful to use as a developer. I seem to revisit this question quite frequently over the years. I have decided to continue using Make so far, but I wonder if I will eventually give up on it.

Alternatives

There are many alternatives to Make, but most are filtered by the following basic constraints:

• The tool should provide a common interface across projects using a wide range of programming languages and tools. The tool therefore needs to be general-purpose, not specific to a programming language or ecosystem.
• The tool must be portable. At the minimum, it should work on a wide range of Linux distributions, but a good candidate should also work on BSD, macOS, and Windows.
• The tool must support developing software for many different systems. For example, Nix is not an acceptable solution outside of a Nix ecosystem.

Since Haskell is my preferred programming language, I quite like using the Shake build system. Unfortunately, it requires a Haskell development environment to build and use the build tool itself. This is a huge dependency, especially for non-Haskell projects, and preparing the environment and building the build tool for the first time can be very time consuming. I therefore would not use Shake as a general purpose build system.

In the past, I have run into some things that I was unable to solve (somewhat elegantly, in a short amount of time) using Make. In such situations, I wrote separate scripts to implement what I needed and delegated to the scripts from the Makefile. Shell scripts have many limitations, many pitfalls, and poor error handling (see the Rewrite When More Than N Lines blog entry), so I usually write such scripts in Python. Like Make, Python is already installed in most environments, and the “batteries-included” approach to the Python standard library provides plenty tools to use without having to require external libraries (which can be particularly difficult for people who are not Python developers).

Writing the “complex” logic in a Python script is the most promising alternative that I am considering at this time. Python code is much easier to develop and maintain than Make code, it is much more powerful, and it can provide excellent error handling and debugging tools for when things go wrong. A Makefile can delegate to the script in order to provide the familiar interface, and the script can be used directly to access advanced options that are difficult to provide with Make. By only using the standard library, the added dependency is not very expensive at all. The most significant downside to this approach is that common code would need to be copied to each project, which is not a good way to manage software.