Who is using stdlib?

Following Ondřej’s @certik inquiry about the status of our Fortran community, I was interested to see whether any project already makes use of stdlib. From a quick search I could only a handful of open source projects on GitHub, however most of them were repositories for staging changes for inclusion in stdlib.

Is anyone aware of a larger projects which is using stdlib? Also, what is holding your projects back from using stdlib (size of the dependency, missing features, fypp preprocessor)?

4 Likes

Within the Fortran community, I see many people like duplicating and using only parts of a library instead of creating an explicit dependency on it, which could subsequently create dependency hell. This is especially true if the library is rather new and experimental (as is the case with stdlib). So, I would not be surprised if this question does not get many responses.

6 Likes

I am one of the devs of Fluidity a CFD adaptive solver. The main reasons why I haven’t attempted to modernise our codebase and incorporate stdlib is because

  1. the large size of our codebase, which also includes legacy Fortran
  2. and our external dependencies. We rely on so many packages PETSc, BLAS/LAPACK, Zoltan, VTK, ParMetis, SCOTCH (the list really is endless) and we deploy in quite a few different HPCs and OSs so we need to be able to guarantee that our code will deploy without too much hassle.

Other than that I don’t think there is anything else stopping us. I am pretty sure that we have developed similar methods to quite a few stdlib methods as part of our own FEM and linear algebra modules.

7 Likes

As @shahmoradi suspected, I have “vendored” a few key routines from stdlib, but am not relying on it directly for anything. My hesitancy is mainly to do with the fypp requirement and the size of the library. I think it’s a worthwhile project, but I suspect it will really begin to shine once the generics capabilities are added to the language.

3 Likes

@everythingfunctional I just tested stdlib using fpm on my Apple M1 and everything built perfectly. I don’t have fypp installed and it is not needed unless you want to develop stdlib. In Debug (the default?) mode it took 27s to compile, but that’s because fpm currently compiles in serial (I think) instead of parallel.

4 Likes

Sorry to have to answer no instead of not replying at all, but I thought it worth commenting this: I wouldn’t expect a larger project using stdlib in the next few years, for a few reasons.

  • Most large Fortran projects are tied to slow (multi-year) funding cycles (science at universities and governments). They are developed slowly
  • There aren’t many new Fortran projects. Most Fortran development is in extending existing large and not-so-new projects, rather than starting new projects. Stdlib is very new, just over two years old. That’s a blink of an eye on the time scale of large Fortran projects. The benefit of replacing the working and familiar home-cooked stdlib-like functionality in these projects with stdlib is probably more trouble than worth it.
  • Many large Fortran projects are not open source. If they do end up using stdlib, we won’t know about it unless somebody reports here or on GitHub.

That being said, I expect stdlib to be used more broadly in toy projects, especially via fpm.

I have not used stdlib myself except in never-published small toy projects, and especially since stdlib became fpm-able. Stdlib becoming usable via fpm was the turning point for me to use it in small experiments.

Granted, I have not programmed Fortran professionally very much since stdlib started. Nowadays my programming workload is about 10:2:1 for Python:JavaScript:Fortran, as much as I wish it were different. Since the start of stdlib, I’ve only had one non-trivial (~3K LOC) Fortran project with six large dependencies. I just scanned through the code and it looks like I could only use stdlib for kind constants, but nothing else. So, for me, I guess I’m still waiting for a Fortran project in which I’ll need stdlib.

4 Likes

Though I agree with you, I’d still call this “using stdlib”. :slight_smile: I think that directly inserting a source file instead of using a dependency the “proper” way may suggest that it’s just easier to do it that way, and I’ve done it many times before.

2 Likes

stdlib currently has a good complement of Fortran built-in functions on general algorithms, I’ve fallen in love with fortran-lang’s fpm and stdlib, I use fpm in vs code, with the updates of the Modern Fortran plugin and Fortran language server (Thanks @gnikit a lot), The experience is getting better.

At present, stdlib and fpm still have great potential. At least stdlib has not yet connected to the perfect lapack interface. At least we don’t have inv, det, and other array calculation, which are the strengths of Fortran; fpm will become more and more easy to use as it continues to develop, of which I’m expecting improved ease of use for linked libraries in my local disk not only from the Internet.

It does take some time to compile stdlib, but if it is compiled and placed on our local disk, it is reasonable to use fpm to call it:

  1. Use CMake or fpm to compile stdlib;
  2. Store stdlib on the local disk and set LIBRARY_PATH=<path of stdlib>
  3. Configure fpm.toml:
[build]
external-modules = ["stdlib_strings"]
link = ["fortran_stdlib"]

[library]
include-dir = "../../inc/fortran_stdlib"

At present, stdlib is more convenient for ordinary users to write some small projects (maybe behave like numpy?) and learn common algorithm writing methods with certain consensus. fpm and stdlib make Fortran users feel that there are still more possibilities. As one of the basic languages, scientific computing language (Fortran) is really suitable for open source co-construction and benefit from open source.

Other links

4 Likes

Absolutely, I only meant to broaden the definition of stdlib user beyond explicit dependency. I am one such user inspired by the stdlib’s optional argument handling module. I ended up (re)implementing the stdlib module for my use case for three reasons: 1) I did not like the current interface, 2) “no dependency on any external library” rule for the package under development, and the need to extend the stdlib interface to other scenarios.
It would be great if stdlib modules are (re)designed to be used as independently as possible from each other. It will extend the userbase of the library over time. As evidenced by the above comments, most developers are hesitant to add a new dependency on a large library whose interface is still evolving. However, I bet many people would want to use parts of the library relevant to their work if the module-inter-dependencies are minimal.
It takes quite a while to gain users’ confidence in a library. It’s too early, in my opinion, to make any judgments based on the number of users who have an explicit dependency on stdlib. It’s a great worthy project that should be continued. Is there a vision statement for stdlib somewhere? If not, it would be good to have one.

6 Likes

I agree, we need to get the linalg module to call lapack and provide a nice high level interface, then it will become very useful to many people.

9 Likes

Recognizing the value, it looks like someone has already done this splitting of the stdlib for their usage (link was by @zoziha in the above).

3 Likes

@shahmoradi let’s discuss the splitting in Idea: Splitting stdlib into smaller separate libraries. If you could write your opinion there, that would be awesome!

3 Likes

I feel like the fypp preprocessor makes the codes look not so Fortran. I once looked for a normal distribution function in the stdlib. When I saw the preprocessor stuff, I instantly gave up.

2 Likes

My experience is very similar to what @milancurcic describes. Most of my Fortran programming at the moment is on old(ish) projects which already have their own home-baked versions of stdlib utilities. I haven’t had a chance to either update these to use stdlib (definitely on the radar for some of the bigger ones) or start new ones using stdlib.

I also find myself writing more Python than Fortran, especially when performance isn’t critical, which has historically been because of easy access to utilities like those provided with stdlib, package management and ease of use in Jupyter Notebooks. All of these are things being addressed by Fortran Lang and so I see myself having fewer and fewer reasons to turn to Python in the future. I suspect a lot of folk and projects are in the same situation.

So, give it a few years and I reckon stdlib will be ubiquitous!

4 Likes

I am using stdlib in one of my work projects. It is a library, which defines functions to store and load variables to and from netCDF files in MPI applications.
The only role of stdlib is optval and the logger. Maybe I will use the strings from stdlib in the future.

If stdlib wouldn’t be available via fpm, I wouldn’t use it for this project.

5 Likes

I expect I’ll slowly start to use more of stdlib. Right now I just am using the save_io_npy routines for a smaller project. However, on the cluster I’m using given the package management, I had to not use stdlib but the original NPY-For-Fortran repository on GitHub, as stdlib seemed to have more modern requirements than what my cluster could [easily] provide. (I probably could have made stdlib work, but it was easier to grab the routines from NPY-for-Fortran).

4 Likes

I use stdlib in one of my largest (not open-source) project (> 25K LOC), mainly stdlib_logger and stdlib_sorting.
Through fpm, I use it daily with other fpm-packages for handling files, computing descriptive statistics, … Note that, without fpm, I would probably not use Fortran for these tasks, even if it would have been slower for some of them. Based on the comments of this thread, it really seems that supporting stdlib in fpm was an important step for stdlib.

I don’t see the compilation time as a problem (except if I develop it :wink: ), because it can be compiled once, and doesn’t need to be compiled each time (like BLAS/LAPACK, or other libraries I use as a user). And I believe that the issue with fypp is not a true one. Indeed, generated code could be provided, as for fpm.

4 Likes

We should start adding high level modern Fortran linalg to stdlib and by default use the Fortran reference lapack: Maintain fpm installable lapack under fortran-lang?

Both stdlib and lapack provide high performing default reference implementation. And then vendors are free to substitute by even higher performing versions (MKL, OpenBlas; and the same for stdlib down the road).

Once stdlib has linalg via Lapack, I think I would use it too in new projects, almost by default, as linear algebra is almost always needed in any numerical code.

7 Likes

Hi all, reviving the post as redirected from Who is using fortran-lang projects (fpm, stdlib, minpack, fftpack, vscode-support)?.

I’m a new stdlib user, for my first “from scratch” new fortran project (which I haven’t yet announced here but might do in a few weeks: it is currently under thorough “real-use-case” testing, running my PhD stuff in a HPC cluster and I definitely see some final additions and fixes before it becomes a real first working version).

I wanted this project to be developed fast and clean so I aimed at outsourcing everything that is not too specific to the core algorithm. Well, probably the exact opposite of what Fortran development has looked in the past…

Hence availability with fpm was a crucial requirement for including a dependency, in total alignment with this:

Especially since I use only sort and to_string and we (the github organization to which the project pertains) have a codebase providing both sorting algorithms (though less smart, hehe) and “number2string” functionality. So yes, thanks so much for the fpm branch, it really brings a lot of value, even more for fast development and precise version management.

Overall I would not call HoneyTools a toy project, in the long run at least, but surely it has started this way. Hence I find Milan’s statement to be quite relevant:

(and I don’t think, in any way, this is a bad thing to happen…)


Coming to what probably is the more interesing part I would like to describe now what instead has not worked or, at least, has put me in trouble. Hopefully those pain points would gradually relieve…

First of all, as I said, the code is meant for aiding HPC computations and I felt quite a bit the “deployment struggle”:

Specifically the stdlib_sorting module has some string related stuff that is using the %re / %im syntax and that does not compile with the processor we use for production on the HPC cluster: GCC 8.3.0 which, I know, is quite old indeed but, despite having a newer one available, it poses some trouble to correctly link with MKL and openMPI, so most people in the department is stuck with the older one.
Eventually I managed to get around the issue but it was an hassle and I think this sort of dynamics might well add to the general hesitance. The stdlib is modern in its very sould and unfortunately HPC environments are quite often unfriendly to modernity (just regular CentOS struggle…).

The other huge pain point is for sure the lack of modularity. The stdlib is divided in modules, sure, but they are somewhat interdependent and that forces a “all or nothing” feeling which often might resolve to “well, than nothing”. After my successful and overall pleasant experience I tried to push my “software head” (the researcher in charge of codebases) to adopt stdlib for more stuff, maybe even slowly replacing our “homebrew” stuff, but the main objection —a hard one— was exactly based on this module interdependence: he does not like the error handling implementation (for whatever reason) and as far as I understand you cannot really use a module without implicitly importing some error_stop call inside the procedures (or at least you are not guaranteed to be able to… here an example). I think getting around the issue here is quite subtle: from one viewpoint I can totally see how the authors of some well thought error handling suite would like to adopt it in other parts of their wide project, surely composability is a great value and the whole point of a standard library, but I can also see how this forces to really “like” everything in stdlib (as it currently is) for choosing to adopt even a small part. In this embryonal stage I think the most likely outcome is people just discarding the library as a whole or at least postponing adoption until everything has clearly settled and there would no more room to argue.

So yes, this has been stressed a lot already, but I’d like to do it once more :slight_smile:

I’ll end saying that despite these points I might add yet another stdlib dependency on HoneyTools, namely a hash map (preliminary details are here). Though I haven’t yet inspected the docs with enough care and I’m actually still evaluating some possible alternatives… the use case would be not so straightforward and flexibility would be key.

So yeah, surely

but the few that are seeing their first sun are quite interested in stdlib! :smiley:

7 Likes

Some really interesting comments and feedback.
I wonder if some resolution could start with getting your “software head” to join this discourse?

2 Likes