Containers using F202Y's generic programming

Help
81

GFortran, for example, does all the things you said are not done (https://godbolt.org/z/48axM3Yxh): it handled transpose of the argument by simply telling sgemm that it is transposed, no temporary (I think it handled the result without a temporary as well).

1 Like
82

There are multiple, starting from the multidimensional arrays themselves, which if one would like to express in C++, would need to write a rather sophisticated object.

I think anyone “wishing for a standard library” really brings the mindset of other languages to Fortran which is a mistake. Fortran is not complete. You cannot write many intrinsic functions in Fortran itself. If you do not believe, find a way to implement a custom function that does this:

! pseudocode
real(4) :: x = myfunction(arguments, kind=4)
real(8) :: x = myfunction(arguments, kind=8)

Fortran does not really need generics (more fancy that the most basic “kind” genericity which I believe is half of the current proposal). The need for:

  • generic kind for procedures and
  • generic containers

is something that would get 99% of the job done. Implementing traits or other fancy thing feels like waste of time for a simple reason:

  • from my experience, at the moment (2025) no freely available fortran compiler can correctly run very heavily object-oriented (Fortran 2008) code (flang is the closest to the best). Even between minor versions of the compilers, the stability may be patchy.
  • in practice: Fortran features 95-03+ are really mostly useless!
  • therefore, any Fortran generics proposal other than a very simple one will ever work reliably enough to be used in production
  • and when people finally implement their linked list in 20 different ways and everyone cools down, it will turn out that this part of the language is so overdeveloped for just this one purpose that it will become a ballast for the future improvements
  • not to mention, everyone will again have their own implementation, which in most cases will be terrible - not everyone wants to include the bloat of stdlib in their code or is even aware of its existence
  • so why waste resources and everyone’s time, what we need is basic built-in basic data structures which the leading scientific computing language, Python (sorry if that hurts) has shown to be sufficient: map/dict, list, set. There is no need for overcomplicating things or endless discussion about if. If the committee is unsure of the data structures needed, I can deliver an A4 with this list printed or punched on cards.
  • LFortran is doing it the right way.

I know turning Fortran into Rust is cool and exciting stuff but that will never ever be stable enough to be used in any production code.

2 Likes
83

Given the history and nature of Fortran applications, I would have thought symmetric , diagonal, upper or lower triangular, etc matrices would be more appropriate extensions than any of the features mentioned.

2 Likes
84

Just out of curiosity, I am wondering what kind of computations do you mainly use Fortran for in your works / studies (possibly, astrophysics)? I feel there are several different “camps” in the Fortran community according to the type of their calculations, which seems to lead to significantly different opinions about the current status and desirable features. (FYI, I have been working on particle-based simulations in the last few years, though I worked on PDEs much much ago when I was a student… time flies XD)

1 Like
85

I guess the “usual” reply from the Standard committee would be like “it is possible to define your custom derived types for representing such matrices by wrapping raw data and providing appropriate operators” etc (essentially for any request…) FYI, Julia has various matrix types, which might be some sort of wrapper types (maybe for use in multiple dispatch), but I am not sure about details.

1 Like
86

@gronki’s post is so self-contradictory, that I have to share some thoughts here:

  1. Fortran (as a programming language) neither exists in a vacuum nor is it particularly special. It must evolve to remain useful – through the proven process of adopting features that have demonstrated their worth in other languages. Fortran 90, that @gronki seems to hold to high esteem, is to a large degree a rip-off of Modula-2. @gronki’s animosity against ideas from other languages therefore makes absolutely no sense.

  2. To regard the functionality that Fortran 90 introduced as somehow more fundamental, when compared to some of the newer Fortran standards, is also utterly wrong. The same was thought by incorrigible FORTRAN 77 vendors and users alike when Fortran 90 was about to be introduced, which was one of the factors that nearly killed the language. If Fortran development were to follow all of @gronki’s ideas, the language would ultimately go the way of the dodo. Which I presume would be good news for some hardware vendors, as they would then finally have their long-sought-for excuse to drop Fortran compiler development entirely, and force us all to program in C or C++.

  3. @gronki conflates the lack of willingness of hardware vendors to implement new Fortran language features with the difficulty of doing so. None of the OO features introduced by Fortran 2003 and 2008 are actually fundamentally difficult to implement. LLVM’s Flang is the best example for that. Said OO features were basically implemented by a single knowledgeable developer (@clementval over at the LLVM discourse, kudos to him) once the rest of the new compiler’s infrastructure (scanning, parsing, intermediate representation (FIR), etc.) was available.

  4. The fact that said OO features do not work satisfactorily in the free gfortran compiler is due to the fact that the few gfortran developers seem to have lost control of their old code base. To insinuate, like @gronki does, that multi-billion dollar companies like Nvidia or Intel are also incapable of delivering a compiler with modern language features is absolutely laughable. If Nvidia or Intel really wanted to, we’d have working traits and generics in the language tomorrow!

  5. Thanks to the existence of LLVM, the (Fortran) compiler ecosystem is fundamentally changing. I expect that in ten years time there will only be two major Fortran compilers in wider use: LFortran and LLVM’s Flang. Both are new, LLVM based, code bases. Most hardware vendors have already ditched their own C and C++ compilers in favor of LLVM’s Clang. The same laws of economy will force them to do so also with their presently different Fortran compiler front-ends, and to instead adopt Flang (once it has matured). Old compiler code bases were one of the most important reasons why it was difficult to get new features into the language. These old code bases are now being phased out.

  6. What also made the incorporation of new features difficult is that, as it was noted in this thread, the committee has bloated the language with unnecessary features, while leaving out truly crucial capabilities like compile-time polymorphism. Some of us (e.g. the OP) need to solve rather complex problems, and desperately need this latter capability. @gronki obviously doesn’t. But he also seems to be totally fine with denying it to the rest of us. Emulating Python is no solution, since the type systems of dynamically typed languages (like Python) are polymorphic by default, whereas those of statically typed languages (like Fortran) are not, and require some subtyping mechanism (e.g. traits) to achieve comparable functionality. To not use such general polymorphism capabilities for implementing also (at least some of) the containers would go against the laws of economy. More generally, putting new features into the core language makes the time for implementation more protracted rather than more brief. Which would be the contrary of what @gronki seems to truly want. Hence his post is conflicting.

1 Like
87

@kkifonidis I think you are too harsh on @gronki. Let’s assume the best intentions.

I agree with @gronki, I think it’s better if the most used data structures are in the language itself and Python shows it’s absolutely doable and also shows which data structures you actually need in practice. Python and C++ missed multidimensional arrays, we need those too. Once you have that, then the need for generics is greatly reduced. I think that’s @gronki’s main point, as I understand it. I agree with this.

It’s the old question of language features vs libraries. In my opinion it’s better to have a library than nothing at all, and it’s good to have the ability to write libraries. And it’s even better to have a well-designed language feature for the task at hand (scientific computing).

I agree with @kkifonidis, that since Fortran already has OO, might as well do it “right”, and the generics proposal that he and I developed is one way to do it. In terms of new syntax or semantics, the changes are actually quite minimal, it seems to fit well with the existing language. Yes, you can use it to develop custom containers and many users will. But just like in C++, if we want the most common containers to be common across code bases, they have to be part of the language, otherwise we’ll end up with 20 incompatible implementations like C++ did for arrays. The @kkifonidis proposal is just a few changes here and there on top of an existing Fortran OO features, so I am excited to prototype it, hopefully soon (after we can compile existing OO). I don’t see the generics as ways to have containers, but rather as a way to structure large codes. It’s not the only way however, I personally also like a simple Fortran 90 style, but I do like some of the more modern additions like do concurrent or hopefully soon lists, dicts, sets. But for large codes, it’s good to at least have an option to use the interface-based generics. @kkifonidis I wish you could make your code or some “proxy” version of it public, so that people can learn how you use OO.

3 Likes
88

I do not feel strongly about how those containers should be implemented. But if the thesis is that their presence somehow obviates the more general need for generics (and that Fortran therefore shouldn’t have generics), then I disagree.

I also didn’t mean to be harsh. I just tried to share my thoughts on the matter.

2 Likes
89

Dear @kkifonidis ,

I have what one could name a decent knowledge of several programming languages. Python, in fact, is my least favorite one, although the one that I use to win bread, and that one that undoubtedly won the race in scientific programming. Rust, on the other hand, even though I do not know it very well, seems the most appealing to me. I mostly use C++ these days because I simply have more experience in that language. Therefore I do not know where you take my dislike of other languages. I am just being realistic about what Fortran can be and where is its place in the market. I agree Rust is cool and it’s done right, but I simply do not see Fortran going this way with all the external factors.

I have spent last 10 years programming Modern Fortran, and apart from rather typical computational tasks, I did tackle some issues which required more complexity (such as writing a scripting language parser in Fortran), although likely too trivial for you to consider worthy. That experience also showed me where limits of the language are. So while earlier I was a walking advertisement for Modern Fortran, recently I simply had to admit that the new features are simply too unstable to be used in production. I really enjoyed using them and they failed miserably for me when it comes to portability across compilers. I was not writing unit tests to test code; i was testing compilers. Effectively I had to burn months of work, cut the losses and rewrite that part of the logic to C++, because it was like spinning wheels in one place.

You can attribute the problems to the gfortran team and the developers at Intel and other companies as much as you like, but the truth is that the language does not exist in a vacuum. Even if one compiler supports the entire Fortran language, without a glitch, it would be still a sane decision to write a code that would target market median. And from my experiences, I could not even fully trust allocatables.That does not give me much hope that generics will work flawlessly across compilers anytime soon. And I do not blame compiler developers, they are one of the smartest people, but maybe it is not as easy as it appears to be.

Since you like to speak about economics, I will bring up a few things I learned about managing projects: know who your target market is and find the most cost-effective way to appeal to that market. What are the greatest needs and how to accomodate them with the least effort? Generics are not a need, they are one of the solutions. Are they the solution that would in a finite time fix 90% of the issues that people have with Fortran? The answer depends who you ask. In the past I would agree with you that generics are desperately missing from the language, but I reconsidered. Maybe something much simpler will do the job.

I really have no issue with old OOP nor new OOP in Fortran. I do not deny anything to anyone because I have no power, and as you mentioned I clearly have never done anything significant. But the opinion I voiced is realistic from the user’s perspective. The language is a dodo at the present moment. There is amazing effort at the community side, and tools like fpm, lfortran, flang, stdlib, ford, …. have completely transformed the experience, but we are nevertheless a niche group. The reality and expectations need to meet somewhere.

4 Likes
90

I feel this very hard. Whenever I write code in Fortran I make sure to test it with every compiler before I think about committing it to my main branch. If it doesn’t work, I implement the option that works for every compiler.

For me, developing quantum chemistry, oceanography, and shallow water solvers I don’t really need generics. I don’t mind writing a template function and then letting copilot (or fypp) redo it for me for the datatypes I need.

I treat Fortran right now as my “because I can and it does what I need” programming language. I used C++ for a couple years straight and got pretty tired of it - this has been a nice break from C++ and has helped me realize what C++ has to offer and what Fortran offers in return.

My main idea right now is write code that I need/want using Fortran and reporting bugs to every compiler I find any in - this has been pretty fun and entertaining. I have written some cool things and learned a lot. But for now this is a “for fun” kinda thing. In my day job I am mostly using Fortran and Python/C but the Fortran is limited to what the project supports and uses, which is pretty basic Fortran.

Maybe if I change jobs or targets I might either become more vocal about it or not…no idea. THanks for this great discussion, it is very cool to see.

2 Likes
91

I (somehow sadly) completely agree with that…

Point 1: the experience with the parameterized DT, which are poorly supported even as of today (it seems that NAGfor is the only compiler that does them completly right) indeed shows what could happen with the full genericity.

Point 2&3: full genericity has sense only if true standard library (i.e. defined by the standard and shipped with the compilers) finally shows up, and this apparently won’t happen soon.

Point 3: I wanted very much to include the current stdlib in our industrial software, but faced many issues. I could find some workarounds for some of them (applying some patches where the compiler was struggling), but not all of them (because they were deeply related to our build system, which is somehow unconventional). Conclusion: even if you are aware of stdlib and want to use it, it’s not always possible.

Point 1 (again): the proposal about auto-generic procedures look quite simple and will fill 90% of my needs as an “end-user” (although I probably won’t be able to use it in my career, as we have to produce code that is compatible with sometimes 10 years old compilers :frowning: … At the moment I can only use f2008 features, as the code still has to be compilable with Intel Fortran 18)

1 Like
92

@PierU, I wonder, beyond the patches and the stability discussion we already had (and that has not landed in deaf ears, is just in digestion process) which build inconveniences have you encountered? I have a similar inconvenience because our current production compiler is intel 19 (and gnu 11 for validation), so I also had to do some patch work, and for the build system which also has a lot of homemade quirks, what I did was to use just the fypp pre-preprocessing manually once (or every time I update stdlib) and launch python config/fypp_deployment.py just to get the raw files in a single folder which I have fixed and from there on, I just let our build system to take on the rest of the job. (Note: I do this extra step because I point to the github master branch, but if you recover from the stdlib-fpm branch, this step has already been done so you would just need to take raw Fortran and C files from src)

There is an on-going effort to modularize stdlib (at least with cmake) such that one can cherry-pick parts of it.

Regarding the discussion, I’m among those how has a few home made containers classes, but I would also prefer to replace them with something that is more robust/flexibel beyond the limitations of my own biased use, which is community maintained, not depending on one single compiler being able to provide it since I know that it won’t happen in my active life time… So for me stdlib is the right place.

Just to add to the controversy, the fact that there are many Fortran compilers might add to the robustness of the language as a concept, but it actually hinders practitioners because of the ecosystem fragmentation. There where a Rust or Julia practitioner might not even know the difference between the language and the optimizer/processor/compiler (however one wants to call it) in Fortran we are forced to carry that mental and infrastructure load. Even if there is one hero compiler team that manages to get all these issues solved, it will unfortunately just add to the entropy. We might hope that it might force other compilers to follow, will them? We might hope that it will make the Standard’s comitee to actually take our community demands into account, will it?

2 Likes
93

The Fortran Array is not a data structure.

  1. Everything is an Array, some have rank zero.
  2. The Array has an implied order of elements and there is a set of rules for referring to collections of subobjects, leveraging that implied order.
  3. That’s it. There’s nothing mandated about time complexity for operations and, if you have two connected computers, a conforming Fortran processor could place A(1) in Melbourne and A(2) in Novosibirsk and break no rules.
  4. The genius of the language is that it specifies just enough to make efficiency possible, for most of the workloads it was designed to address. The Array abstraction has influenced the hardware organization of today, but most computer cycles today are running workloads Fortran was not designed for and that won’t change.
  5. In my opinion, we should not be trying to retro-fit to Fortran features designed for “alien” workloads (web browsers, operating systems and control of machinery, comm networks, & ) but develop abstractions that would help traditional workloads run better on distributed hardware (the Co-Array, brilliant in its economy, is a start but will it influence hardware innovations as much as the Array did?).
3 Likes
94

The no-show issue for us is the non-stability of the ABI. We have libraries on the one hand (and stdlib could be one of them) and processing modules (“module” here does not refer to Fortran modules) on the other hand. Both are versioned and compiled to static object files, and the point is that any version of a processing module can be linked to any version of a library. The link takes place only when a production job has to be run, linking together the needed processing modules (which can differ from job to job) and the needed libraries.

This means that the ABI must be strictly the same between the different versions of a library. This can be ensured by using only the C ABI for the libraries. Now, stdlib exposes the API of Fortran modules, and the stability of the ABI then requires:

  • using the very same compiler version and compilation options to compile everything (this is the case in our system)
  • no change at all in the public API: this is not guaranteed in stdlib, where optional arguments can possibly be added to the public procedures, public module variables can be added to the modules, etc…

I am aware that this is not a usual setup…

1 Like
95

We solved this problem in ESMF about 15 years ago by requiring keywords be used for all optional arguments. We used a compile-time technique to enforce it. This then had the following effects:

  • Required arguments could be either positional or use keywords
  • Optional arguments were required to use keywords
  • We could add new optional arguments as needed without breaking old codes
  • If needed we could rename optional arguments (e.g., for consistency around the library), or even delete them (obsolete/bad feature) and any code would immediately know during compilation.

On the last bullet, once we had the ‘keywordEnforcer’ dummy arguments inserted in all the entry points (at the dividing point between the required and optional arguments), we then were able to go through all the argument names and make them consistent throughout the library. The compiler would tell us at compile time where any issues were.

It is a simple technique that I think more Fortran libraries, such as stdlib, should use. Of course this is at the API level, not the ABI level. Nonetheless, it worked well in practice.

96

I think this is good practice for ALL arguments (required or optional) when you are trying to use a 3rd party library and you don’t have immediate access to the full library documentation. About 13 years ago, I was writing some code that used some LAPACK 90 interfaces. Much to my surprise, I found that some of the LAPACK 90 functions in the Intel MKL implementation did not have the same signature (ie argument list order) as the ORNL/UTenn/Netlib versions. The argument names were the same but their order/position in the routine argument list were different. Fortunately, keyword arguments saved me from having to add MKL specific code if I wanted to use MKL. Don’t know if Intel ever changed this and I’ve always wondered who at Intel thought this was a good idea.

1 Like
97

I like your solution (which you mentioned before), and I have already used it in a project.

Nonetheless:

My problem is precisely related to the ABI.

1 Like
98

I am sorry, but this comment is something that has to be addressed. Nobody is writing a web browser in Fortran, it is a completely strawman type argument. But if we consider anything other than multiplying matrices is an “alien” application of Fortran, then there is no need for Fortran at all, since many other langages can do it better and faster. Any language is useless today without good ways to process data, which is not neccesarily in a matrix form, but may come as columns (pandas) or even more irregular shapes. You need to do this to read, write and transform the data, because your computation is not allocated in pure vacuum. This is not some “alien” use case of the language which is supposed to be go-to choice for science. Fast key-value storage is not any luxury, it is essential basics, and it is incredible that we still discuss whether this is a neccessity in 2026.

6 Likes
99

I’ve said the following so many times on this forum that I’m beginning to sound like a broken record, but modern solution algorithms in several fields that Fortran once dominated have moved away from structured data that maps directly to an array or matrix format. Having features like unordered maps (hash maps), dictionaries, some lists and trees etc. that can handled data not aligned in memory in an efficient way is essential for developing fast codes on modern HPC architectures. I urge everyone who thinks these things are only used for “alien workloads” to go look at modern CFD and Finite Element codes written in C++ that have to deal with unstructured computational mesh topologies.

Edit.

I would also suggest reading Luca Formaggia’s article on “Data Structures for Unstructured Mesh Generation” in the Handbook of Grid Generation. This book is from 1999 so the need for advanced non-array data structures has been known for a very long time now.

9 Likes
100

A necessity from the point of view of the programmer does not make a necessity from the point of view of the language designer.

The Linear Algebra folks recognised that their problem was social, not technical, so they got together and stabilised the (FORTRAN 77) interfaces they needed. The result is that wherever you find a Fortran compiler, you will find BLAS/LAPACK/ScaLAPACK.

You cannot solve a social problem with technical means.

1 Like