First release of the Fortran standard library

We are more than happy to announce the first release of the Fortran standard library (stdlib). We just published version 0.1.0:

Many thanks go to all the contributors at stdlib participating in the discussions, reviewing patches and sharing code. Almost two years of work are collected in this release since the project started in Dec 2019. (We plan to release in shorter intervals going forward.)

Feedback is very welcome, feel free leave a comment here or open a new issue at the stdlib repository. We are also always looking for new contributors and reviewers.

You can find the full changelog below or in the linked release.


The following modules are available in version 0.1.0:

  • new module stdlib_ascii #32
  • new module stdlib_bitsets #239
    • new derived types bitset_64 and bitset_large
    • new abstract base class bitset_type
  • new module stdlib_error #53
  • new module stdlib_io
    • new procedures loadtxt and savetxt #23 #37
    • new procedure open #71 #77
  • new module stdlib_kinds #63
  • new module stdlib_linalg
    • new procedures diag, eye and trace #170
    • new procedure outer_product #432
  • new module stdlib_logger
    • new derived type: logger_type #228 #261
  • new module stdlib_math
    • new procedure clip #355
    • new procedures linspace and logspace #420
    • new procedure arange #480
    • new procedure gcd #539
  • new module stdlib_optval #73 #96 #139
  • new module stdlib_quadrature
    • new procedures trapz, trapz_weights, simps and simps_weights #146
    • new procedures gauss_legendre, gauss_legendre_lobatto #313
  • new module stdlib_random #271
  • new module stdlib_sorting
    • new procedures sort, ord_sort and sort_index #408
  • new module stdlib_specialfunctions
    • new procedures legendre and dlegendre #313
  • new module stdlib_stats
  • new module stdlib_string_type
    • new derived types string_type #320
    • new procedure move #467
  • new module stdlib_stringlist_type
    • new derived types stringlist_type and stringlist_index_type #470
  • new module stdlib_strings
    • new procedure to_string #444
    • new procedures strip and chomp #343
    • new procedures starts_with and ends_with #384
    • new procedure slice #414
    • new procedure find #433
    • new procedure replace_all #436
    • new procedures padl and padr #441
    • new procedure count #453
  • new module stdlib_system
    • new procedure sleep #54

The stdlib supports compilation with CMake, make and fpm and was tested with the following compilers:

Name Version Platform Architecture
GCC Fortran 9, 10, 11 Ubuntu 20.04 x86_64
GCC Fortran 9, 10, 11 MacOS Catalina 10.15 x86_64
GCC Fortran (MSYS) 11 Windows Server 2019 x86_64
GCC Fortran (MinGW) 11 Windows Server 2019 x86_64, i686
Intel oneAPI classic 2021.1 Ubuntu 20.04 x86_64
Intel oneAPI classic 2021.1 MacOS Catalina 10.15 x86_64
26 Likes

Congrats and thank you all, fantastic achievement!

2 Likes

Congratulations.

3 Likes

Congratulations.

I suspect you already know this, but I tried building stdlib with Intel oneAPI Classic 2021.3 on Windows (an untested combination as I’m well aware) using cmake, just out of interest…

I get a compilation error cascade starting with
2>Compiling with Intel® Fortran Compiler Classic 2021.3.0 [Intel(R) 64]…
2>stdlib_stats_moment.f90
2>C:\Source\stdlib-0.1.0\build\src\stdlib_stats_moment.f90(1): error #8765: Error in opening the compiled ancestor module file. [STDLIB_STATS]

Looks like the submodule/module dependency checking needs fixing.

I note there’s an open issue #368 (Add testing workflow for Intel oneAPI on Windows) on GitHub; I assume this will be covered by that?

2 Likes

On Windows things are always strange and confusing to me, but maybe I can still help. Would you mind sharing which version of CMake you used and whether the build backend was VS or Ninja? If you didn’t specify the generator (-G option) than CMake will default to VS project files, not sure how well those work for tracking dependencies in Fortran projects with submodules.

We are currently testing stdlib mainly in MSYS2 environments using Ninja as build backend (-GNinja).

1 Like

Yes - congratulations and thanks for all your hard work.
I have grabbed it and have started building and noticed that there are shed loads of warning messages when I’d expect libraries to build cleanly - no doubt this is easy to fix.
More seriously I think there is a structural problem. I picked one file, median, and the source code is over 77k lines long and the .o is over 159MB This is far too big for general use. It looks as if fpm generates all combinations that can be legal. If that’s true I suggest that every one gets written to it’s own source file and compiled separately, that way when building from an archive only the routines that are actually used will be pulled in.
Avoiding this sort of bloat is one of the reasons C++ is introducing modules so it would be good not to start behind the curve on this one.
Since templates aren’t part of the language yet we are a bit hampered but would it be possible to use the old cfront method that C++ used to use? On the HP at least it would compile and link the program then deduce the template instantiations it needed from the undefined symbols, write the necessary code then build and link a second time - slow but always interesting to watch!

6 Likes

With gfortran I use the -Wall option to turn on warnings but also use -Wno-maybe-uninitialized -Wno-surprising to turn some off because I found them to generally produce spurious warnings. Many of the the gfortran warnings for stdlib are -Wmaybe-uninitialized . Gfortran warnings do not necessarily indicate a problem with stdlib, but if certain warnings are going to be ignored, maybe they should turned off in the builds.

2 Likes

I think we currently have the warning level for GFortran and Intel Fortran maxed in the CMake build files, which is probably not a good idea since we are using a lot of implicit conversions from integer to real in stdlib among others. The amount of warnings in stdlib needs some serious trimming down, starting by reducing the signal-to-noise level in the compiler options to create the warnings.

2 Likes

Right. I am not worried about the warnings (which can be fixed one way or another — however indeed things should build cleanly, so we do have to fix it), but this is indeed a real problem.

It seems generating all array ranks and types and kinds like that indeed introduces too much bloat which is unsustainable in practice.

It’s an interesting idea to deduce what instantiation have to happen from undefined symbols. But it seems we need to be able to reliably obtain those on all platforms, and then hook it in a feedback loop. And every Fortran compiler has different name mangling, so we would also have to take that into account. We could probably teach fpm to do that. However in modern Fortran things would not even compile if the given (overloaded) subroutine is not available in the module. So I don’t even know if this is technically doable. We could infer from the compiler error messages what needs to be built.

A more technically sound idea is to implement generics in LFortran, and then LFortran can “instantiate” only what is needed for a given project and write it down as Fortran source file, then one can use any Fortran compiler to compile. And we can integrate the LFortran’s library with fpm, so that fpm does this automatically under the hood.

Still, I am not happy with the solution. I would like to just distribute Fortran sources of the stdlib library that just work, no hacks or code generation required. But I don’t know how.

5 Likes

cmake version 3.21.3
The build backend was VS (default).

I basically followed the instructions in README.md and used pip to get fypp, cmake & ninja.

Build was from an Intel oneAPI Tools command prompt with PATH prepended with the folder containing the programs that pip installed.

Thanks for the suggestions. I’m happy to try using Ninja for the build backend in MSYS2; I’ll let you know how that goes when I get a chance.

1 Like

Would it already makes sense to provide system packages?

There are https://launchpad.net/ for Ubuntu, https://openbuildservice.org for almost every Linux flavor, https://aur.archlinux.org for Arch Linux and related distributions, https://brew.sh/ for MacOS, https://spack.io for HPC, and system-independent https://conda-forge.org.

2 Likes

Yes, it indeed makes sense to start this. I’m familiar with packaging for conda, spack, brew, Arch and MSYS2 and will probably submit stdlib for those at some point. But I won’t touch a deb any time soon (why does this stuff need to be so complicated?).

Submitting a conda-forge recipe seems like the next step, since I anyway have to package a lot of Fortran projects there in the next time. Maybe even with a short tutorial in case people are interested in how its done.

I already posted a guide for packaging with MSYS2 here:

1 Like

then I would volunteer to be part of a Conda and Spack team and also take care of Arch Linux.

1 Like

Conda-forge recipe is submitted, if anyone wants to join as maintainer, please comment in the linked PR and I’ll add you:

If so, what would be the size of the whole library? Many GB?

I don’t understand why stdlib has to be so complex. In windows it’s so difficult to setup.
Why can’t it be some simple modules which people can download and use ? Why such huge arrangement for generics - fypp. Use only Fortran and with good documentation things should be easy. Since in Fortran we don’t have so many types such as int, float, long, double… I don’t see the such dire need for generics.
This is how NAG and IMSL libraries are written. Just because c++ has generics, we need not run after them.

We do in fact:

program kinds
  use iso_fortran_env, only: &
    integer_kinds, &
    real_kinds

  print *, integer_kinds
  print *, real_kinds

end program

The output of the program:

$ gfortran test_kinds.f90
$ ./a.out
           1           2           4           8          16
           4           8          10          16

So in total 5 different integer kinds, and 4 real kinds with this compiler.

For a fully-fledged library the routines in stdlib need to specialize for each kind, some also for type (e.g. sort), and some also for rank (e.g. mean).

Hence, the need for a preprocessor and the large number of procedures and huge object files.

If the procedures from stdlib were ever to become standardized by the WG5 Fortran Standard committee, then the vendors/compiler implementers could delegate the code generation step to the compiler with the major advantage that only procedures which are actually called are generated.

I don’t expect this to happen any time soon. The more realistic options is what @certik suggested in post #9.

If anyone is in charge of negotiating a large deal for a HPC facility, they can also force such things into the contract with the compiler vendors. I guess we still need some time to reach this point.

4 Likes

Indeed, 5+4=9 options just for real/integer. And then multiply this by the ranks that you want to support, so at least 3, more likely 6 and up to 15, for algorithms that cannot be expressed as elemental. Say we only want to support up to 6, to be conservative, so that’s 54 total subroutines to generate. A lot of times only real makes sense, so that would be 24. Still that is a lot.

Let’s worry about the “any rank” later. First let’s design a good extension for the kinds. What would be the most natural way to do it? Let’s worry about only the real type first. Should we do something like real(*) (or full: real(kind=*))? It seems we want the variable for the kind. So perhaps this:

subroutine f(x, y)
integer, parameter :: wp = kind(x)
real(wp), intent(in) :: x
real(wp), intent(out) :: y
y = sin(x)**2 + erf(x)
end subroutine

Then it’s up to the compiler how it will actually get it done but it would do something equivalent to generating all 4 kinds (for example it can just generate it ahead of time, or only on demand for what is actually used, etc.):

interface
  module procedure f4, f8
end interface
...
subroutine f4(x, y)
integer, parameter :: wp = 4
real(wp), intent(in) :: x
real(wp), intent(out) :: y
y = sin(x)**2 + erf(x)
end subroutine

subroutine f8(x, y)
integer, parameter :: wp = 8
real(wp), intent(in) :: x
real(wp), intent(out) :: y
y = sin(x)**2 + erf(x)
end subroutine

What would be the best design of “any kind” generics?

2 Likes

One further thing to consider. If you want Fortran’s generics to be useful in higher level code, you need to be able to deal with structured matrices. Julia makes it possible to have types like BandedMatrix{Nhi, Nlo} where depending on N you can get a bidiagonal, tridiagonal, or any other type of banded structure. This is really useful for things like writing efficient solvers for partial differential equations.

2 Likes

@oscardssmith, excellent point.

Also for just sparse matrices like CSR.

It would be really cool if user defined matrices could somehow reuse the natural Fortran array syntax. I believe C++ will now be able to do exactly that:

Improving Fortran standardization process (lessons from C++23 getting multidimensional arrays)

To use an agreed upon API for arrays, with user defined implementations.

2 Likes