After chatting with some users of the forum, I realized that the most unusual part of the tool is probably not the lexer itself, but the build model behind it. In this follow-up post I show:
- how the generated dependency graph is intended to be used,
- how
.mod and .o ownership are separated,
- how
-J/-I integrates with the generated rules,
- and how this compares conceptually to
gcc -MMD in C/C++.
This is a dependency-file-driven Make model, not a “scan everything up front” model. The core idea can be traced back to Peter Miller’s classic article Recursive Make Considered Harmful. One of the key ideas there is the use of dependency files: generated files containing target-dependency relationships (but no recipes) which are later included into the main Makefile. These have also been called “anchor files”.
The main advantage of dependency files is that it allows the user to abstract away interfile dependencies such that pattern rules can be provided.
Two facts about make
- If
make finds -include dep.d together with a rule to build dep.d, it will first build dep.d and then include it in the Makefile.
- The dependencies of rules for the same target are merged by
make before applying the recipe.
In the C case, this leads to common patterns such as:
SRCS := $(wildcard $(SRC)/*.c)
OBJS := $(patsubst $(SRC)/%.c,$(OBJ)/%.o,$(SRCS))
$(OBJ)/%.o: $(SRC)/%.c | $(OBJ)
$(CC) $(CFLAGS) -c $< -o $@
$(OBJ)/%.d: $(SRC)/%.c | $(OBJ)
$(DEPSCRIPT) $< $(OBJ) > $@ || $(RM) $@
DEPS := $(patsubst $(OBJ)/%.o,$(OBJ)/%.d,$(OBJS))
-include $(DEPS)
As you see, there is no need to manually keep track of #include files, as dependency rules already do this for you. All the logic is delegated to a script which generates dependency files. In the C case, I usually go with
#!/bin/bash
# $1: source code.
# $2: object directory.
#Appends "$2/" at the beginning and adds a .d target.
gcc -MM -MP -I. $1 | sed "1s|^|$2/|" | sed '1s|^\([^:]*\)\.o:|\1.d \1.o:|'
Fortran’s case
The main reason why I developed FortranDep is to provide the core functionality to emulate the gcc -MM -MP -I. $1 part. Fortran differs from C in that .(s)mod files are compiler artifacts and require cautious handling. Most of Fortran-Make related headaches come precisely from this fact.
My goal was to design a build system that generates .(s)mod, .o, and .d files which are stored in a $(OBJ) directory, so I could emulate C’s pattern. Mainly, this is useful because provides standardization for hybrid C-Fortran codebases. With FortranDep, I adapt my dependency-generating script:
#!/bin/bash
# $1: source code.
# $2: object directory.
#Appends "$2/" at the beginning of words with suffix .o, .d, .mod and .smod.
./tools/FortranDep -pd $1 |
sed -E "s@(^|[[:space:]])([^[:space:]]+\.(o|d|mod|smod))@\1$2/\2@g"
So I can use the pattern:
SRCS := $(wildcard $(SRC)/*.F90)
OBJS := $(patsubst $(SRC)/%.F90,$(OBJ)/%.o,$(SRCS))
TRASH := trash
$(OBJ)/%.o: $(SRC)/%.F90 | $(OBJ)
$(FC) $(CFLAGS) -J$(TRASH) -I$(OBJ) -c $< -o $@
$(RM) $(TRASH)/*
$(OBJ)/%.mod: $(SRC)/%.F90 | $(OBJ)
$(RM) $@
$(FC) $(CFLAGS) -fsyntax-only -J$(OBJ) -c $<
$(OBJ)/%.smod: $(SRC)/%.F90 | $(OBJ)
$(RM) $@
$(FC) $(CFLAGS) -fsyntax-only -J$(OBJ) -c $<
$(OBJ)/%.d: $(SRC)/%.F90 | $(OBJ)
$(DEPSCRIPT) $< $(OBJ) > $@ || $(RM) $@
DEPS := $(patsubst $(OBJ)/%.o,$(OBJ)/%.d,$(OBJS))
-include $(DEPS)
The key observation is that .mod / .smod generation can be decoupled from object generation by using -fsyntax-only, -J, and -I appropriately (using gfortran).
I needed to use a couple of “directory tricks”: the .(s)mod files generated by .o file generation are discarded (-J$(TRASH)), and we instruct the compiler to look for the .(s)mod files it needs in the $(OBJ) directory (-I$(OBJ)). Similarly, .(s)mod files are now first-class objects and get a proper recipe. They are generated with the -fsyntax-only option and go to the $(OBJ) directory (-J$(OBJ)).
The benefits:
- completely parallelizable builds,
- clear and expandable build system,
- no need to manually bookkeep hundreds of dependencies, and
- clear and deterministic object generation logic.
The drawbacks:
- serial builds take longer, as
.(s)mod are generated twice, and
- imposes patterns, such as: one module/submodule per source file with matching names.
Regarding these, I believe that the benefits from parallelization alone completely outclass the drawbacks. This is not necessarily the fastest possible build strategy for small projects, but it scales well and keeps dependency handling explicit and deterministic.
TL;DR: How this is intended to be used
FortranDep is similar to makedepf90 and meant to be used exactly like gcc -MMD in C projects:
- A script generates
.d files (dependencies only, no recipes).
- These
.d files are -included by your Makefile.
- Pattern rules handle
.o, .mod, and .smod generation.
If you are already comfortable with .d files in C/C++, you already know how to use this tool.