CMake for building a program spread across directories

Help
1

I’ve got a project that has the following directory structure

my_cool_project/
    CMakeLists.txt
    source1/
        CMakeLists.txt
        foo.f90
        bar.f90
    source2/
        CMakeLists.txt
        baz.f90
        qux.f90
    source3/
        CMakeLists.txt
        bif.f90
        bof.f90
        main.f90

The main program is written in main.f90, which calls subroutines defined in the other source files. Some of these source files define modules, while others are naked subroutines. There are dependencies between the directories, such that each one cannot be built independently. I am struggling to write a CMakeLists.txt that automagically figures out the dependency graph and builds the main program.

My top-level CMakeLists.txt is:

# my_cool_project/CMakeLists.txt
cmake_minimum_required(VERSION 3.20)

project(my_cool_project Fortran)

set(CMAKE_Fortran_MODULE_DIRECTORY ${CMAKE_BINARY_DIR/mod})

add_subdirectory(source1)
add_subdirectory(source2)
add_subdirectory(source3)

While the lower ones are

# my_cool_project/source1/CMakeLists.txt
set(SRC ${SRC} foo.f90 bar.f90)

# my_cool_project/source2/CMakeLists.txt
set(SRC ${SRC} baz.f90 qux.f90)

# my_cool_project/source2/CMakeLists.txt
set(SRC ${SRC} bif.f90 bof.f90)
add_executable(${PROJECT_NAME} main.f90 ${SRC})

When I try to build this project, the source files are not built in the right order, so module dependencies are not fulfilled, and compilation fails. It is as if CMake doesn’t recognize that there are dependences across directories. I know the dependency graph is solvable because I have been able to build this project with a plain Makefile together with the tool makedepf90. Have I done something stupid in my setup here?

I know the simplest workaround would be to flatten the directory structure – dump everything under a single source directory, and then have just two CMakeLists.txt files for the whole project. But for external reasons, it is important to me to preserve the source directory structure here. Given that constraint, how to I coerce CMake into building this project?

I’ll happily provide more detail if needed. Many thanks in advance to any Fortran CMake wizards who can help.

1 Like
2

The way I do it is:

# my_cool_project/CMakeLists.txt
cmake_minimum_required(VERSION 3.20)

project(my_cool_project Fortran)

set(CMAKE_Fortran_MODULE_DIRECTORY ${CMAKE_BINARY_DIR/mod})

foreach(dir IN ITEMS source1 source2 source3)
  add_subdirectory(${CMAKE_SOURCE_DIR}/${dir} ${dir}/)
  target_link_libraries(${PROJECT_NAME} ${dir}_lib)
  add_dependencies(${PROJECT_NAME} ${dir}_lib)
endforeach()

and

# my_cool_project/source1/CMakeLists.txt
add_library(source1_lib foo.f90 bar.f90)
target_link_libraries(source1_lib things_source1_links)
add_dependencies(source1_lib things_source1_uses)

So the files in each directory are built into a separate library, which can then be linked against other libraries and have dependencies defined as needed.

I find it works best to use add_dependencies to define the “use module” hierarcy, and target_link_libraries to define the (potentially circular) linker dependencies.

1 Like
3

First, let me comment on the issue you are currently running into. CMake has scopes, each CMakeLists.txt has its own scope and inherits all variables from its parent scope, which is the CMakeLists.txt which used add_subdirectory(...) on it. (This is not completely true, I know, there are exceptions but lets ignore those for now.)

The SRC variable in source1/CMakeLists.txt and source2/CMakeLists.txt are different variables and cannot be seen in source3/CMakeLists.txt. You can run

message(${SRC})

after each add_subdirectory call to convince yourself.

To export a variable back to a parent scope, you have to set it explicitly to the parent scope with

set(SRC ${SRC} PARENT_SCOPE)

But you will notice that this won’t carry the directory information along, this has to be added with CMAKE_CURRENT_SOURCE_DIRECTORY when setting the SRC variable, otherwise CMake won’t find the source files.

Maybe I have time for a more complete answer later, for some reading on CMake I suggest the stdlib workflow document, because you I’m spotting some gotchas here already (CMAKE_BINARY_DIR).

For now I can just recommend you to look into other CMake based projects, I can suggest to look into stdlib-cmake-example or toml-f. The latter is using extensive nesting of source directories and might provide you with the hints you are looking for.

2 Likes
4

Maybe a bit too much “quick and dirty” but does the following work for you?

...
FILE(GLOB_RECURSE ${PROJECT_NAME}_SOURCES "src/*.f90")
...
ADD_EXECUTABLE(${PROJECT_NAME} ${${PROJECT_NAME}_SOURCES})

All in one CMakeLists.txt in the root directory?

2 Likes
5

I think the cleanest solution is to use target_sources. Your top-level CMakeLists.txt file will define a target using add_executable (or add_library) and afterwards call add_subdirectory for each of your subdirectories. The CMakeLists.txt file in each of the subdirectories will then call target_sources with the name of the target and the sources in that subdirectory. No messing about with variables at all.

To be explicit using your example, your top-level CMakeLists.txt file would be:

project(my_cool_project Fortran)
add_executable (${PROJECT_NAME})
add_subdirectory(source1)
add_subdirectory(source2)
add_subdirectory(source3)

Your CMakeLists.txt file in source1 would look like:

target_sources(${PROJECT_NAME} PRIVATE foo.f90 bar.f90)

And similarly for the other subdirectories.

2 Likes
6

Thanks very much for the suggestions! Indeed, @epagone, that solution is very quick and dirty, but it’s doing the job for the time being. And thanks to @nncarlson for the suggestion to use target_sources.

@awvwgk Thanks for diagnosing the issue and for the links. My CMake knowledge is very crude, so I appreciate you having the patience to point out the basics.

7

Let’s try a more complete answer (I tend to quote a lot in CMake even if it is not required, but it helps to distingish keywords from strings IMO)

# ./CMakeLists.txt
cmake_minimum_required(VERSION 3.20)

project(
  "my_cool_project"
  LANGUAGES "Fortran"
)

# Collect all source files
set(src)
add_subdirectory("source1")
add_subdirectory("source2")
add_subdirectory("source3")

# Create target
add_executable(
  "${PROJECT_NAME}-exe"
  "${src}"
)
# Using set_target_properties for each target is usually preferred
set_target_properties(
  "${PROJECT_NAME}-exe"
  PROPERTIES
  OUTPUT_NAME "${PROJECT_NAME}"
  Fortran_MODULE_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/mod"
)

My personal preference is to define targets, at least libraries, in the top-level, it makes it easier to inspect the top-level CMakeLists.txt and know what the build actually does, while most of the subdirectories only collect sources and pass them back up to the top-level.

For each subdirectory I’m usually defining the current directory as local variable because I don’t want to type CMAKE_CURRENT_SOURCE_DIR every time. Finally, you have to pass the appended source files up again to the PARENT_SCOPE.

# ./source1/CMakeLists.txt
set(dir "${CMAKE_CURRENT_SOURCE_DIR}")

list(
  APPEND src
  "${dir}/foo.f90"
  "${dir}/bar.f90"
)

set(src "${src}" PARENT_SCOPE)

# ./source2/CMakeLists.txt
set(dir "${CMAKE_CURRENT_SOURCE_DIR}")

list(
  APPEND src
  "${dir}/baz.f90"
  "${dir}/qux.f90"
)

set(src "${src}" PARENT_SCOPE)

While you have the main.f90 file in this directory, I would find it surprising to have the main program compiled here, but its modules stored in the top-level instead, therefore I’m just collecting main.f90 like every other source and handle the compilation in the top-level.

# ./source3/CMakeLists.txt
set(dir "${CMAKE_CURRENT_SOURCE_DIR}")

list(
  APPEND src
  "${dir}/bif.f90"
  "${dir}/bof.f90"
  "${dir}/main.f90"
)

set(src "${src}" PARENT_SCOPE)

As usual with CMake there is more than one way to write build files that just work. I recommend to avoid GLOB because it is expanded at configure time rather than at build time, adding new source files requires to trigger a reconfiguration to have CMake recognized there are new files in the tree that should be included. This is especially annoying if you just started the project and add new source files as you write new modules and always have to remember that you must touch some CMakeLists.txt to trigger a reconfiguration or your build will not work correctly.

Find a style you like and you can work with, I personally adopted the above style because it produces smaller diffs in the build files and makes the intention of certain values clearer. Also, with clear separation of the tasks performed in different CMakeLists.txt files it is always easy to check if a change in a file just adds/removes source files or changes the build logic and therefore requires extra attention.

Edit: seems like a change in the GLOB result will trigger the reconfiguration automatically since CMake 3.12, this should remove this minor annoyance. For some projects like stdlib or DFTB+ which use fypp, a reconfigure step might result in rerunning the preprocessor, therefore changing all the files and triggering a complete rebuild, which can become expensive in rebuild times for adding one file.

3 Likes