I agree that makefiles are really useful, even (or perhaps especially) in the case of large projects. My main problems with makefiles are:
they need to be set up in such a way that customisation/configuration is easily possible.
some makefiles are construed in arcane ways (the makefiles generated by both autotools and CMake are prime examples, but I do not need to customise them).
some makefiles use features/extensions that are specific to a particular version of the make utility.
That said, would it be an idea to write a tutorial on such tools with specific attention to the typical needs of Fortran programs?
Obfuscation is possible in every language… Bash scripts can also be terrible if the programmer makes no effort to follow good practices. Make files and bash scripts are sometimes just considered as auxiliary files, with disregard. But it’s programming, as noble as any other language.
The thing I don’t like about makefiles is having to change them so frequently. In order for one to work reliably, it needs to know exactly all the dependencies between targets and sources.
For Fortran, an object file depends on the source file it is compiled from AND all the .mod files for any modules USEd in the source file. A .mod file depends on the source file in which that module is defined. An executable depends on all the object files for all the code it uses, and code they use, etc, even if that code wasn’t in a module, and so doesn’t get included via a USE statement.
So compiling a source file; 1) has multiple dependencies that are not simply pattern based, 2) produces (possibly) multiple targets, which again may not necessarily be pattern based. 1) means you have to manually write every single rule. For 2), Make doesn’t support rules with multiple targets, so you have the (in my opinion clumsy) workaround of specifying redundant, empty rules. And since dependencies between files is something that changes frequently, you end up making changes to the Makefile all the time. In my experience, this is something that sometimes gets forgotten, or isn’t done carefully enough, and you end up with an unreliable Makefile.
Number 1) caused me to abandon trying to manually maintain a Makefile, and 2) caused me to abandon trying to write my own scripts to generate one. CMake has the nice trick of inserting itself as a dependency into the Makefiles, so it gets to automatically keep them up to date, but it still isn’t an ideal solution because you still have to manually tell it your source files and final targets. I don’t have any experience using autotools for active development, so somebody else will have to comment on how that is.
Once I got fpm up and running and switched to using it, I never have to even think about the build system, let alone spend any time maintaining it. It has saved me an incredible amount of time and energy. I for one am never going back.
I agree that Makefiles are not perfect and certainly there ought to be a better way, but CMake definitely isn’t it - it’s just too arcane and unreliable.
I’ve given fpm a go this evening and it fails to build my application (two source files and one main program). Something to do with submodules I think but I’m disappointed it didn’t work out of the box. I’ll have to spend some time now trying to figure out why and looking at the issues and seeing if I can fix them. There doesn’t seem to be an ‘fpm clean’ or hooks into svn, only git. So another build system and another world of pain.
I have been following the progress and I am really really excited about fpm. I guess that it is very close to being able to link widely used libraries like MPI?
Yeah, CMake isn’t particularly user friendly and I try to avoid it whenever I can.
With fpm, were your source files where fpm expects to find them? The convention is for library files to be in src and for programs to be in app. If you have trouble, don’t hesitate to ask questions.
It’s possible to link as Brad said, but it’s not possible to build with libraries that have pre-built modules (like system-provided MPI, HDF5, NetCDF, and similar, or if you built them yourself by hand). This is because fpm currently assumes that all module dependencies come either from the package itself or its fpm dependencies.
Sorry to hear of your painful initial experience with fpm @simong, this is certainly not what we want the process to be like. We’re keen to get existing packages to be fpm compatible and identify any shortcomings in fpm so please submit an issue if you think something is not working that should work or feel free to ask for help in the discussions. If the project in question is available online, are you able to share a link?
To follow up on fpm and MPI, with fpm/#438 merged, it is now possible to compile fortran MPI projects in fpm if you’re using the Intel mpi compiler wrapper (mpiifort).
1) Add the following to your fpm.toml file:
[build]
external-modules = "mpi" # (if using)
2) Compile with:
$ fpm build --compiler mpiifort
3) Run with:
$ fpm run [NAME] --compiler mpiifort --runner mpirun
This only works for mpiifort by virtue of the existing compiler detection logic, but MPI support in fpm isn’t official in that fpm currently remains ignorant of the fact you are compiling MPI (see fpm/#354).
I just gave it a try, and it builds! Although I get an error when I try fpm run. And I get the same if I try to run the executable built by fpm using mpirun.
I created a new project: $ fpm new check_mpi
added under fpm.toml
[build]
external-modules = "mpi_f08"
and this is my module
module check_mpi
use mpi_f08
implicit none
private
public :: say_hello
contains
subroutine say_hello
integer :: irank
call MPI_INIT()
call MPI_COMM_RANK(MPI_COMM_WORLD, irank)
print*, 'hello from rank', irank, '!'
call MPI_FINALIZE()
end subroutine say_hello
end module check_mpi
compiled with $ fpm build --compiler mpiifort # works!
ran with $ fpm run check_mpi --compiler mpiifort --runner mpirun
this was the output:
[0] MPI startup(): I_MPI_CAF_RUNTIME environment variable is not supported.
[0] MPI startup(): Similar variables:
I_MPI_THREAD_RUNTIME
[0] MPI startup(): To check the list of supported variables, use the impi_info utility or refer to https://software.intel.com/en-us/mpi-library/documentation/get-started.
Abort(805362447) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init: Other MPI error, error stack:
PMPI_Init(110): Cannot call MPI_INIT or MPI_INIT_THREAD more than once
Abort(805362447) on node 1 (rank 1 in comm 0): Fatal error in PMPI_Init: Other MPI error, error stack:
PMPI_Init(110): Cannot call MPI_INIT or MPI_INIT_THREAD more than once
Abort(805362447) on node 2 (rank 2 in comm 0): Fatal error in PMPI_Init: Other MPI error, error stack:
PMPI_Init(110): Cannot call MPI_INIT or MPI_INIT_THREAD more than once
Abort(805362447) on node 3 (rank 3 in comm 0): Fatal error in PMPI_Init: Other MPI error, error stack:
PMPI_Init(110): Cannot call MPI_INIT or MPI_INIT_THREAD more than once
Command failed
ERROR STOP
Error termination. Backtrace:
#0 0x55bf4d02bc5f in __fpm_environment_MOD_run
at ././src/fpm_environment.f90:144
#1 0x55bf4d030ec0 in __fpm_MOD_cmd_run
at ././src/fpm.f90:420
#2 0x55bf4d01c4a4 in MAIN__
at app/main.f90:28
#3 0x55bf4d01c4a4 in main
at app/main.f90:9
EDIT: Using Intel oneAPI’s ifort 2021.1 Beta 20201112
