Dear all,
does anybody have advices/templates, how to set up a project with fpm, where the MPI dependency is optional? I’d like to add fpm build config files to a project, which can be either compiled serially or optionally with MPI. I did not find any ways with fpm so far, only the workaround to set up two repositories, one containing the non-MPI, the other the MPI-dependent part of the library. This works, but feels somewhat unsatisfying…
I agree with you @aradi that the absence of customization is perhaps the greatest source of frustration when using fpm. Attempts have been made at customizing profiles, but haven’t made it into the codebase yet.
So right now, I think the only way to deal with that is have a “base” project that does not use MPI, and a “parent” project that has both base and MPI as dependencies, where all MPI wrappers are contained. Exactly like you did, and not really easy to accomplish.
I personally like the way Cargo deals with customization: add features that can specify custom targets via command line (or in a dependency to another project), where features can be many things, including os-dependent compiler/link flags, optional dependencies, etc.
@aradi, do you call the MPI functions directly or do you (as most large projects I’m familiar with do) write wrappers around the MPI calls and call them instead. This allows you to isolate the MPI calls into one source file and then conditionally compile using the preprocessor based on setting some compiler macro (ie ifort -DUSE_MPI). I have also seen “dummy” MPI packages that provide subroutines and functions with just the MPI interfaces and usually don’t do anything and are only used to satisfy entry points during linking. Not the fpm solution you are looking for but this is a common way of getting around your problem at the source code level.
Edit.
See John Burkardt’s mpi_stubs code for an example of a dummy MPI package.
Note: it looks like Professor Burkardt has moved to South Carolina and some of his old Florida State links aren’t working. The following should work.
https://people.math.sc.edu/Burkardt/f_src/mpi_stubs/mpi_stubs.html
@IanMartinAjzenszmidt, this reads like a ChatGPT generated response. I’m afraid it is factually incorrect, as the fpm profiles aren’t ready yet (@FedericoPerini would know this as a fpm developer.)
I’d find it respectful to disclose the fact, if you’re answer was generated by a chatbot (even if only partially). If that was not the case, ignore my response.
@FedericoPerini Thanks a lot for the advice, this is exactly, what will do then. I just wanted to make sure, that I am not missing some more obvious and elegant solution.
Actually, the situation is even more complicated, as the library I work on (Fortuno unit testing framework) has also an optional co-array dependent part. So I’ll probably have 3 repositories, the serial implementation, the MPI-extension and the coarray extension. Is there actually any way to indicate coarray-dependency in fpm?
@rwmsu Thanks a lot. I was thinking about the approach you suggest. My biggest problem with the approach you suggest (which we use ourselves in several projects), that I don’t know how to specify in fpm that the dependency is optional. In CMake or Meson, I’d allow for a -DWITH_MPI user option and have some logic in the build configuration to create the add the MPI-dependency only when really needed. But how do I achieve that with fpm?
Thanks @aradi, if you have ideas to propose about how to introduce some flexibility in the fpm manifest, please do so here or on the github issues page!
There is currently no explicit way to use coarrays either. I myself do not use coarrays, so I may be mistaken. I think the only current way to use coarrays in fpm is via a custom compiler command:
fpm --compiler=caf
fpm --compiler="ifort -coarray"
With current FPM you probably want to use multiple response files and save “extension specific” flags in each .rsp file. The option --option caf works for OpenCoarrays but one needs an extra --runner "cafrun -n 8" to run the executable. Here is a template of the response file
$ cat fpm/test_caf_openmpi.rsp
# gfortran 13.2.1
# openmpi 4.1.5
# opencoarrays 2.10.2
option test
option --target "test"
option --compiler caf
option --profile debug
option --flag "-Wunused -std=f2018"
option --runner "cafrun -n 8 --oversubscribe"
Recently, the OpenCoarrays built from mpich 4.1 + gfortran 13.2 has some known issues (issue781, issue783, and I can reproduce that on my laptop) so you might want to switch to OpenMPI + gfortran when installing OpenCoarrays.
$ cat fpm/test_ifx.rsp # ifx (IFX) 2024.0.2 20231213 option test option --target "test" option --compiler ifx option --profile debug option --flag "-coarray=shared -coarray-num-images=8"
Burkardt is currently a lecturer at the University of Pittsburgh. When I emailed him this year at the address shown there regarding missing files, he replied from a gmail address based on his first two initials and last name.
@han190 Thanks a lot, I was not aware of the response file possibility in fpm yet. However,how the selection via response file choice can be propagated between projects? My project with optional coarray components is a library. How does an other project, which uses my library as a dependency, specifies, that it wishes to use the my library with or without coarray support? (It is important to allow the consumers to opt-out for mpi and coarray support, as they might not have the right environment for those at all.)
I might be wrong but I don’t think there is such an option in FPM yet. The workaround that I can think of is to create four github repositories, for example,
fortuno-infrastructure,fortuno-MPI,fortuno-coarray,fortuno-serial.The first repo contains shared contents and the rest three contain library specific contents and their own fpm.toml. Then FPM users could add corresponding dependency as needed. For example, if the coarray version is used, the following could be added to the user’s fpm.toml
[dependencies]
fortuno-coarray.git = "https://github.com/fortuno-repos/fortuno-coarray"
This is not elegant at all and I guess CMake or meson are still the best choices if you need sophisticated compilation options. For example, Meson does have an explicit coarray dependency.
@han190 Thanks, this is exactly, how I planned to do it at the end. Yes, CMake and Meson would be indeed better choices in this respect (I usually use CMake for my projects), but the unit testing framework is supposed to integrate with all 3 build systems seamlessly, so the less flexible one dictates the constraints. 