Makefile foreach Commands

In preparation for writing some articles about Make templates, I am making some improvements to the Makefiles used in my projects. I use diff (or a graphical diff program such as meld) to easily see which parts of a project Makefile are specific to that project. It is preferable to put as much project-specific information as possible in variables at the top of the Makefile. Some projects have multiple executables, so some rules require iterating over the list of executables. The foreach function can be used to do this, and it illustrates some frustrations described in The Make Dilemma blog entry.

The PhatSort project has two executables: phatsort and seqcp. The Makefile defines a variable named EXECUTABLES that lists these. Note that Make variables are actually macros, and this macro acts as a list because Make splits on whitespace.

EXECUTABLES := phatsort seqcp

The install-man rule installs man pages for each executable. It makes a good example because it is very simple. Without using EXECUTABLES, it uses a separate command to install each man page. I would rather use a generic version of the rule so that it does not have to be customized for each project.

install-man: # install man page(s)
> @mkdir -p "$(man1dir)"
> @install -m 0644 -T <(gzip -c doc/phatsort.1) "$(man1dir)/phatsort.1.gz"
> @install -m 0644 -T <(gzip -c doc/seqcp.1) "$(man1dir)/seqcp.1.gz"
.PHONY: install-man

The foreach function expands text for each value of the passed list, concatenating all of the text together. In the following implementation, the semicolon at the end of the command is required because all of the commands are concatenated together and run as a single command by Make.

install-man: # install man page(s)
> @mkdir -p "$(man1dir)"
> @$(foreach EXE,$(EXECUTABLES), \
    install -m 0644 -T <(gzip -c doc/$(EXE).1) "$(man1dir)/$(EXE).1.gz" ; \
  )
.PHONY: install-man

This function works, but it ignores errors in all but the last command. This is not desirable because errors allow the developer to find issues quickly. We want the macro to execute each install command as separate commands. This is done by putting them on separate lines.

Since Make operates using text-substitution, the semicolon just needs to be replaced by a newline. Make does not provide a built-in way to insert a newline (such as \n). The easiest way to create one is to define a macro for it.

define newline


endef

The newline macro is defined using two empty lines because Make ignores the final newline. The install-man rule can be rewritten using this macro. Note that the @ character, used to instruct Make to not display the command before executing it, is moved inside of the macro. This simple example runs one command per executable, but multiple commands can be run by putting newlines at the end of each command. The @ character can be used on each command separately, selecting which to (not) display.

install-man: # install man page(s)
> @mkdir -p "$(man1dir)"
> $(foreach EXE,$(EXECUTABLES), \
    @install -m 0644 -T <(gzip -c doc/$(EXE).1) "$(man1dir)/$(EXE).1.gz" $(newline) \
  )
.PHONY: install-man

The foreach function is expanded to text before the rule is parsed, so each line is interpreted as a separate command. I guess it has some leeway about the prefix character (the leading > character in this example, tab by default).

Testing this, I find that it does not work as desired for a different reason! I used process substitution in order to compress the man page in the same command that writes the file to the destination and sets the permissions, but errors within the process substitution command do not cause the calling command to fail!

The following version first copies the man page to the destination directory, setting the permissions, and then compresses the destination file in place using a separate command. This version correctly fails when the source man page is not found.

install-man: # install man page(s)
> @mkdir -p "$(man1dir)"
> $(foreach EXE,$(EXECUTABLES), \
    @install -m 0644 "doc/$(EXE).1" "$(man1dir)" $(newline) \
    @gzip "$(man1dir)/$(EXE).1" $(newline) \
  )
.PHONY: install-man

I selected this rule because it is simple, and yet it had a bug in it! I would really like to avoid shell scripts and Make…