Why should I use CMake?

I agree that makefiles are really useful, even (or perhaps especially) in the case of large projects. My main problems with makefiles are:

  • they need to be set up in such a way that customisation/configuration is easily possible.
  • some makefiles are construed in arcane ways (the makefiles generated by both autotools and CMake are prime examples, but I do not need to customise them).
  • some makefiles use features/extensions that are specific to a particular version of the make utility.
    That said, would it be an idea to write a tutorial on such tools with specific attention to the typical needs of Fortran programs?
1 Like

Obfuscation is possible in every language… Bash scripts can also be terrible if the programmer makes no effort to follow good practices. Make files and bash scripts are sometimes just considered as auxiliary files, with disregard. But it’s programming, as noble as any other language.

1 Like

The thing I don’t like about makefiles is having to change them so frequently. In order for one to work reliably, it needs to know exactly all the dependencies between targets and sources.

For Fortran, an object file depends on the source file it is compiled from AND all the .mod files for any modules USEd in the source file. A .mod file depends on the source file in which that module is defined. An executable depends on all the object files for all the code it uses, and code they use, etc, even if that code wasn’t in a module, and so doesn’t get included via a USE statement.

So compiling a source file; 1) has multiple dependencies that are not simply pattern based, 2) produces (possibly) multiple targets, which again may not necessarily be pattern based. 1) means you have to manually write every single rule. For 2), Make doesn’t support rules with multiple targets, so you have the (in my opinion clumsy) workaround of specifying redundant, empty rules. And since dependencies between files is something that changes frequently, you end up making changes to the Makefile all the time. In my experience, this is something that sometimes gets forgotten, or isn’t done carefully enough, and you end up with an unreliable Makefile.

Number 1) caused me to abandon trying to manually maintain a Makefile, and 2) caused me to abandon trying to write my own scripts to generate one. CMake has the nice trick of inserting itself as a dependency into the Makefiles, so it gets to automatically keep them up to date, but it still isn’t an ideal solution because you still have to manually tell it your source files and final targets. I don’t have any experience using autotools for active development, so somebody else will have to comment on how that is.

Once I got fpm up and running and switched to using it, I never have to even think about the build system, let alone spend any time maintaining it. It has saved me an incredible amount of time and energy. I for one am never going back.

3 Likes

I agree that Makefiles are not perfect and certainly there ought to be a better way, but CMake definitely isn’t it - it’s just too arcane and unreliable.
I’ve given fpm a go this evening and it fails to build my application (two source files and one main program). Something to do with submodules I think but I’m disappointed it didn’t work out of the box. I’ll have to spend some time now trying to figure out why and looking at the issues and seeing if I can fix them. There doesn’t seem to be an ‘fpm clean’ or hooks into svn, only git. So another build system and another world of pain.

I have been following the progress and I am really really excited about fpm. I guess that it is very close to being able to link widely used libraries like MPI?

1 Like

I haven’t quite tried it out yet, but you should be able to just put

[library]
link = ["openmp"] # or whatever the name of your library

in your fpm.toml file and have it work. Feel free to submit issues and/or add to the discussion if you find something that doesn’t work.

2 Likes

Yeah, CMake isn’t particularly user friendly and I try to avoid it whenever I can.

With fpm, were your source files where fpm expects to find them? The convention is for library files to be in src and for programs to be in app. If you have trouble, don’t hesitate to ask questions.

1 Like

It’s possible to link as Brad said, but it’s not possible to build with libraries that have pre-built modules (like system-provided MPI, HDF5, NetCDF, and similar, or if you built them yourself by hand). This is because fpm currently assumes that all module dependencies come either from the package itself or its fpm dependencies.

Relevant issue: #405

1 Like

Sorry to hear of your painful initial experience with fpm @simong, this is certainly not what we want the process to be like. We’re keen to get existing packages to be fpm compatible and identify any shortcomings in fpm so please submit an issue if you think something is not working that should work or feel free to ask for help in the discussions. If the project in question is available online, are you able to share a link?

2 Likes

I have submitted issue #434

It’s possible to link as Brad said, but it’s not possible to build with libraries that have pre-built modules

Thank you, Milan. I realize that building with MPI is not as simple as I thought, as I can see now by checking what the wrapper does:

$ mpif90 --showme:link
-pthread -I/usr/lib/openmpi -L/usr/lib64 -Wl,-rpath -Wl,/usr/lib64 -Wl,-rpath -Wl,/usr/lib/openmpi -Wl,--enable-new-dtags -L/usr/lib/openmpi -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi
$ mpif90 --showme:compile 
-I/usr/include -pthread -I/usr/lib/openmpi

I really appreciate the efforts spent developing fpm and look forward to using it. To all developers, thanks a lot for your good work!

3 Likes

To follow up on fpm and MPI, with fpm/#438 merged, it is now possible to compile fortran MPI projects in fpm if you’re using the Intel mpi compiler wrapper (mpiifort).

1) Add the following to your fpm.toml file:

[build]
external-modules = "mpi"  # (if using)

2) Compile with:

$ fpm build --compiler mpiifort

3) Run with:

$ fpm run [NAME] --compiler mpiifort --runner mpirun

This only works for mpiifort by virtue of the existing compiler detection logic, but MPI support in fpm isn’t official in that fpm currently remains ignorant of the fact you are compiling MPI (see fpm/#354).

1 Like

Awesome!! Thank you!

I just gave it a try, and it builds! Although I get an error when I try fpm run. And I get the same if I try to run the executable built by fpm using mpirun.

  1. I created a new project:
    $ fpm new check_mpi

  2. added under fpm.toml

 [build]
 external-modules = "mpi_f08"

and this is my module

 module check_mpi
   use mpi_f08
   implicit none
   private
   public :: say_hello
 contains
   subroutine say_hello
     integer :: irank
     call MPI_INIT()
     call MPI_COMM_RANK(MPI_COMM_WORLD, irank)
     print*, 'hello from rank', irank, '!'
     call MPI_FINALIZE()
   end subroutine say_hello
 end module check_mpi
  1. compiled with
    $ fpm build --compiler mpiifort # works! :star_struck:

  2. ran with
    $ fpm run check_mpi --compiler mpiifort --runner mpirun
    this was the output:

[0] MPI startup(): I_MPI_CAF_RUNTIME environment variable is not supported.
[0] MPI startup(): Similar variables:
	 I_MPI_THREAD_RUNTIME
[0] MPI startup(): To check the list of supported variables, use the impi_info utility or refer to https://software.intel.com/en-us/mpi-library/documentation/get-started.
Abort(805362447) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init: Other MPI error, error stack:
PMPI_Init(110): Cannot call MPI_INIT or MPI_INIT_THREAD more than once
Abort(805362447) on node 1 (rank 1 in comm 0): Fatal error in PMPI_Init: Other MPI error, error stack:
PMPI_Init(110): Cannot call MPI_INIT or MPI_INIT_THREAD more than once
Abort(805362447) on node 2 (rank 2 in comm 0): Fatal error in PMPI_Init: Other MPI error, error stack:
PMPI_Init(110): Cannot call MPI_INIT or MPI_INIT_THREAD more than once
Abort(805362447) on node 3 (rank 3 in comm 0): Fatal error in PMPI_Init: Other MPI error, error stack:
PMPI_Init(110): Cannot call MPI_INIT or MPI_INIT_THREAD more than once
 Command failed
ERROR STOP

Error termination. Backtrace:
#0  0x55bf4d02bc5f in __fpm_environment_MOD_run
	at ././src/fpm_environment.f90:144
#1  0x55bf4d030ec0 in __fpm_MOD_cmd_run
	at ././src/fpm.f90:420
#2  0x55bf4d01c4a4 in MAIN__
	at app/main.f90:28
#3  0x55bf4d01c4a4 in main
	at app/main.f90:9

EDIT: Using Intel oneAPI’s ifort 2021.1 Beta 20201112

Ah I thought I’d fixed this issue (Remove: coarray single from default Intel flags. by LKedward · Pull Request #437 · fortran-lang/fpm · GitHub), I’ll open a new PR. Thanks for reporting!

For the time being, perhaps try the following instead:

$ fpm run [NAME] --compiler mpiifort --profile release --runner mpirun
1 Like

For the time being, perhaps try the following instead

It works now:

$ fpm run check_mpi --compiler mpiifort --profile release --runner 'mpirun -n 4'
 hello from rank           1 !
 hello from rank           2 !
 hello from rank           3 !
 hello from rank           0 !

Thanks again!

1 Like

and it also works using the MPI wrappers from NVIDIA HPC SDK.

1 Like

I was able to build a larger project which requires MPI and two external libraries (fftw3 and HYPRE)

My quite hacky way of building with gfortran and Open MPI is:

fpm build --compiler 'gfortran' --flag '$(mpif90 --showme:link)'

1 Like

Thanks @simong for reporting the issue at Build failure with submodules · Issue #434 · fortran-lang/fpm · GitHub, it looks like that will get fixed as we implement custom compiler flags. It seems that for your use case, you can improve your code to compile without the -fdefault-real-8 flag. Btw, this is actually one of the flags that I have used in the past and I just added it to the list here for fpm to support natively: Should fpm be compiler independent as much as possible with regards to flags? · Discussion #443 · fortran-lang/fpm · GitHub.