Installing Fortran module files

Help
1

Is anyone aware of a clear recommendation for installing Fortran module files?

I usually avoid this issue by not installing mod files at all and just reuse Fortran dependencies within meson or fpm. This works fine when all dependencies are available in the respective build system, but CMake or make based Fortran projects would have a hard time to interface with the library. Here, one strategy is to install the project and find it by some means in the dependent project (pkg-config), for Fortran this requires of course to install the module files as well.

The mod files shouldn’t be installed in the top level include directory because of ABI compatibility issues in Fortran and because not all Fortran compilers handle the system include directories in the same way (this is important because the project could be potentially installed in the system prefix).

A notable reference here might be Python, since it has similar issues with ABI compatibility, therefore compiled extension modules include a lot of information like the Python implementation (CPython, PyPy, …), the version of the Python ABI (37, 38, 39, …), the architecture (x86_64, aarch64, …) and the platform (Linux, …). For an extension module compiled against CPython 3.9 and Linux x86_64 the shared object is suffixed with cpython-39-x86_64-linux-gnu.

Since Fortran has similar ABI issues this seems like a viable strategy for installing mod files as well.

I’m currently experimenting with the directory layout in meson (dftd4/dftd4#95), using the package name as well as the compiler identifier and version as subdirectory in the include directory (include/dftd4/gcc-10.2.0 here):

$PREFIX
├── bin
│   └── dftd4
├── include
│   ├── dftd4
│   │   └── gcc-10.2.0
│   │       ├── dftd4.mod
│   │       ├── ...
│   │       └── dftd4_version.mod
│   └── dftd4.h
├── lib
│   ├── libdftd4.a
│   ├── libdftd4.so -> libdftd4.so.3
│   ├── libdftd4.so.3 -> libdftd4.so.3.1.0
│   ├── libdftd4.so.3.1.0
│   ├── pkgconfig
│   │   └── dftd4.pc
│   └── python3.9
│       └── site-packages
│           └── dftd4
│               ├── ...
│               └── _libdftd4.cpython-39-x86_64-linux-gnu.so 
└── share
    └── ...

The module directory is communicated via pkg-config to downstream projects:

prefix=/some/path/to/install
libdir=${prefix}/lib
includedir=${prefix}/include

Name: dftd4
Description: Generally Applicable Atomic-Charge Dependent London Dispersion Correction
Version: 3.1.0
Requires.private: lapack, blas
Libs: -L${libdir} -ldftd4
Libs.private: -fopenmp -fopenmp
Cflags: -I${includedir} -I${includedir}/dftd4/gcc-10.2.0
1 Like
2

I recall reading several discussions on this topic in the comp.lang.fortran archives. I’ve tried to filter out some of the ones that might be relevant:

These are just from the period 2016-2021 and there are likely many older threads too.

If we could come up with a good set of guidelines (and publish them at fortran-lang.org) I believe it would go a long way to improve the accessibility/ease of use of Fortran libraries.

1 Like
3

When creating pkg-config files for libpardiso I ran into the issue, that when linking into a C executable, the -lgfortran flag was necessary, but is added automatically when gfortran is used to drive the compilation. I was wondering if your library might also have this problem?

From a very Fortran-centric perspective, it seems like all the problems related to packaging and distributing Fortran (modules) using existing tools (make, CMake, pkg-config, ninja…) stem from the fact that the tools were developed by system programmers with C libraries as their primary target. This also holds for the conventions on folders. In all build systems which have some form of Fortran support it has been retrofitted rather than included in the initial design.

2 Likes
4

True enough, but it is not that they are completely unusable for Fortran at all. You can run into similar problems when mixing different C++ projects which come with different build systems, not to mention precompiled headers and stuff like that.

The question is more, can we find a common exchange format and directory structure that enough build system “speak” to get the job done, relying on tools used by system programmers makes a good starting point in my opinion. pkg-config is not perfect, but there are basically no cross-platform, multi-build system alternatives available.

For the average user linking against the shared object this will be no problem. Things get tricky with real time libraries like mkl_rt which somehow hide the Fortran runtime from ld or if you plan to link the static library in a shared module, e.g. for building Python extension modules which are independent from the shared object,

5

Not sure if this addresses what you want, but I noticed that programs that I rewrote no longer worked after a computer crash (which had nothing to do with gfortran).

When I reinstalled gfortran they worked. This is where Synaptic put gfortran [Ubuntu-Mate 20.04]:

/usr/bin/gfortran

My programs ran fine afterwards…

6

Fortran wiki also has a page about precisely this issue: Library distribution in Fortran Wiki

7

Thanks for those links. I found a good summary on the topic here, concisely summarizing my past experience with this issue.

Looking to the prior discussion the preference is a subdirectory in the lib directory including all the required information (package name, compiler name, compiler version, maybe even platform and architecture), or a separate subdirectory in the installation prefix (like finclude). But not really a consensus on how we can portably install (and find!) module files.

Sounds like a good idea to me. Maybe we can start this by establishing such guidelines in stdlib’s CMake build files?

8

Just noticed that Fedora have their own Fortran Packaging Guidelines.

2 Likes
9

Interestingly, the FreeBSD packagers decided that it was harmful to include the compiler information in the module directory path and removed it by patching the build files for several of my projects. The argument was that you can easily switch GFortran versions on FreeBSD and including the compiler version would break the package file list (discussed in toml-f#51).

10

For CMake there’s CMake Package Manager. By using it you can essentially handle dependencies exactly the same way one would do with fpm. When writing the build system one need to keep in mind that it will be used in two different scenarios though: Both for local development and when used as a dependency.

Makefiles is another story though, but perhaps it’s time to acknowledge that hand written makefiles is an outdated idea?

The downside is that one have to maintain multiple build systems for a library, but I would feel more comfortable with this solution than distributing mod-files which I’m not sure would be compatible with the user’s compiler. I think it would be easier to set up a CI test procedure that tests the library through the different build systems.

If I were to distribute .mod-files I’d for sure include the compiler version number in the folder structure. The downside is that if a compiler release actually has backwards compatibility for .mod-files it might not be straight forward to use those old .mod-files.

1 Like
11

Thanks, I already looked briefly into it. Maybe worth to create a separate thread to discuss the CMake package manager? Might be of more general interest.

I’m fine with ignoring Makefiles, both generated and hand-written ones :wink:. We have better tools nowadays than make.

1 Like
12

I think that I would follow what Python does, which is pretty much what we do in fpm now. And then slowly perhaps follow the same approach for CMake, especially after fpm can generate the CMake build system?

13

It would be great if compilers could help with this. Adding a compiler dependent marker to the module file name should avoid other compilers ever trying to read incompatible module file formats by accident.

1 Like
14

Here is how LFortran handles it:

It checks a compiler string and also a version, and if it doesn’t agree, then it gives an error message.

I think most compilers do something similar.

Are you thinking that in addition to this, they should also encode the name in the filename of the modulefile, something like a.f90 -> lfortran_mod_a.mod?

1 Like
15

Pretty much like Python encodes ABI versions in byte-compiled code or extension modules directly in the filename, just add a suffix or prefix which is impossible to produce with a Fortran module name, like lfortran-a.mod or a-lfortran.mod for example.

1 Like
16

Awesome, I created this issue for this: Add "lfortran" and version into all mod file names (#653) · Issues · lfortran / lfortran · GitLab

2 Likes