Containers using F202Y's generic programming

Dear all,

does somebody has any insights, whether (and if yes how) the implementation of a generic container is possible using the generic (template) features planned in F202Y? Following the related FortranCon25 talks, I don’t see it it yet.

Let’s say, I’d like to implement a generic list template, which can then be instantiated by a consumer as a list of integers, reals or any (user defined) type which comparison is defined for. It should provide (preferably type bound) routines for adding elements of the given type to the list, retrieving them and for checking, whether a given value is contained in the list.

I would like to see, whether and how we can go beyond the approaches we have now, which are usually macro based or use some “include” trick for instantiation. Also, it should not pose any polymorphic constraint for the value types (e.g. they should not be required to be derived from some base class). I am basically looking for the Fortran equivalent of C++’s std::vector<...> using the new feature to be introduced.

Bálint

Ps. My main motivation is to understand, what is possible with the new features beyond the typical “number crunching” use cases. Having robust generic containers without dirty tricks in Fortran is something, I am missing a lot.

4 Likes

I won’t speak for the committee’s planned “templates”.

But what you are looking for will definitely be possible with our traits proposal that @certik is about to prototype in the LFortran compiler:

See Section 6.4 of this proposal for a simple Vectorexample, and also this thread for further illustration.

@certik and team are presently making excellent progress on implementing Fortran’s legacy polymorphism, in the framework of getting LFortran to beta status. Once they’re done with this, we can expect them to start implementation work on the new traits and generics facilities.

It will get done. We just need to be a bit patient.

7 Likes

@kkifonidis Perfect thanks, this is exactly along the lines of my wishes. :slight_smile:

I would be very curious to know, whether the template features planned for F202Y by the committee can deliver something similar. I think, generic containers are really must have features in modern languages.

(Unfortunately, having your traits proposal implemented in LFortran, but not included in the standard, would keep me from using it, as the projects I am working on must compile with several different compilers… So ideally, I wish for a solution which is capable of something similar what you demonstrate, but is backed up by the next Fortran standard.)

Examples of templates can be found here, most of which have not been updated to the latest syntax. Among them is this example of a generic “vector” template. One could quibble about the best design of a particular container, but generic containers were definitely one of our motivating use cases, and as some of the examples demonstrate are certainly doable with templates.

3 Likes

Perfect, thanks, this is exactly what I was looking for!

Either templates or traits will make containers simpler to build and use, but none will make them as convenient and performant as if they were 1st class objects in the language.

1 Like

Unfortunately, this will never happen in my lifetime or anyone else’s lifetime as long as there are Committee members who think things like container classes belong in libraries and not as an integral part of the language.

LFortran already supports containers internally, and we recently exposed them to Fortran, for example here is a test for a dictionary: lfortran/integration_tests/dict_test_01.f90 at bd6ecfac52a1f372b5c3eb4c7b1c17a8cfde4e22 · lfortran/lfortran · GitHub. For now we used the absolutely minimal syntax, just so that we test the feature itself. We can design some nice syntax now. If you have ideas that you would like, let us know.

Regarding other compilers, we can generate Fortran code from our internal representation, so we’ll call some library for dictionaries, so that one can still use any Fortran compiler and is not locked into LFortran.

1 Like

While I applaud your effort and am thankful someone in the Fortran compiler development community has the courage to lead rather than follow, I’m sceptical that containers will ever be included in any future Fortran standard. Having said that, I personally would love to see (in order of my needs) a C++ vector (horrible name for what is just a dynamic array) class, a hash table/list class, a doubly connected list, stacks, priority queues, and Fortran versions of Matlab’s set utilities.
I have at one time or the other implemented (or tried to implement) all of these at the cost and expense of a great deal of my time that I would not have had to waste if they were already available in Fortran in some form (either as first class objects or through an STL like facility).

2 Likes

Fortran2023: typeof & classof

While having it as part of the core language would be great, chances are indeed low. However, as a second best option, having it as part of the “Fortran Standard Library” would be already a big help, It seems to me, that the planned generic features in 202Y will enable proper and robust implementation of such containers, so we might have something usable and generally available within a few years time.

1 Like

Yes and no. While having a Fortran Standard Library is a step forward, I doubt that any of the commercial compilers will ever ship it as an integral part of their compiler distributions. Users will still be left with the requirement that they build the library themselves and then waste time with their build systems trying to make sure the library is correctly detected, the path to the library and module files are correctly set up and detected, and the library is loaded and linked correctly. What I want at a minimum is something like the C++ STL where all the above is hidden from the user. That would require all the compilers both commercial and open source support the same code base for FSL and ship the FSL with their compilers which I doubt will ever happen at least without the Standards Committee getting involved. If you can’t get all the compiler developers to get together and define a standard, transportable module format (something I think is a higher priority than templates etc), good luck getting them to agree to incorporate FSL or something similar into the language infrastructure.

This entire thing is a vicious circle in the Fortran language. Fortran compilers don’t receive as much attention as C/C++ compilers because the community is smaller → some people might stay away from Fortran because they’re afraid of patchy compiler support (lack of portability) → the community remains the same or decreases → Fortran compilers don’t receive more attention because the community is smaller. This repeated ad nauseam

Even though there are a lot of big applications out there a lot of them have a preferred compiler, for a long time everyone was using the Intel Compilers, some used PGI, very few used GCC as their “standard”. A lot of Fortran applications are still either source available or closed source, with decades of development - since these applications are targeted towards a commercial goal or within a commercial ecosystem there’s little incentive to support anything beyond what works. A lot of applications flat out don’t support certain compilers, even our own stdlib does not build with the nvidia fortran compiler! But how do we force the compilers to implement things if not by breaking things?

In C and C++ projects I’ve collaborated with, there is an emphasis in making sure the code builds well with the most popular compilers: gcc, icc, clang - scientific apps now throw in to nvc and amd’s compilers. Whereas, from my experience, few Fortran applications try to get coverage over multiple compilers. This comes to the next vicious circle:

I want to write my code using X feature → Feature is not available in nvfortran → does not use nvfortran → nvfortran does not implement the feature or is not aware the feature is desired → I want to write my code using X feature which is not available in nvfortran

In my own personal opinion, the most important job for current Fortran users is to fortrangelize and show that you don’t always need to write your code in C or C++ (or now rust!). I feel we’re doing a very nice job of this, I certainly was fortrangelized a couple of years ago.

I am happy that LLMs and AI thingies have good Fortran support - I have dutifully explored generating Fortran code with LLMs and I have seen a DRASTIC improvement in the last two years. I feel that this will help reduce the barrier height for more people to get into the language and this is good. More community + more voices = more noise

This turned into a kind of a rant. Sorry.

6 Likes

Rant away :grinning_face_with_smiling_eyes:. Before I retired a couple years ago I would be ranting with you. In my case it would be about the slow pace of implementing new standard capabilities in all the most commonly used compilers, the reluctance by the Standards Committee to implement changes the user community ACTUALLY wants, and a general disregard by the Committee of usability and code maintenance issues like a standardized module format. When I was working, I saw all of these as a large roadblock to my ability to do the job I was being paid to do. Since I retired they are not as important anymore. The issue of module formats is a big deal to me because I was tasked to install and maintain some large Computational Mechanics codes on a DoD HPC system. The system offered users a choice of several different Fortran compilers so I had to build a compiler specific version for each one due to the incompatibility of module formats. I suspect there are others like me who believe that if you are developing code you plan to release to others to use, good software development practices require you to compile and test your code with as many different compilers (and versions of a given compiler) that you can get access to. I have four installed now on my personal workstations and would have six if I could justify the cost of a NAG license and there was a precompiled version of LLVM flang that I could install just from a tar or zip file or (even better) as a Debian archive (deb file). While Fortran is lightyears better now than what it was 20 years ago, its still just too frustrating to deal with in a lot of areas particularly for new users who are used to having things readily available in either C++ or Python that “just work” without them having to do much more than a pip or conda install in the case of Python.

3 Likes

This is great, and best you can do.However, if it is was part of the langage:

   call _lfortran_set_item(dict_i, -15, 6)
   if (_lfortran_get_item(dict_i, -15) /= 6) error stop

Would be:

   dict_i(-15) = 6
   if (dict_i(-15) /= 6) error stop

Not only much simpler to write/read, but also easier to optimize by compilers (compared to function calls). Also, for <vector>-like containers it would allow refering to sections vect(i:j), which is not possible if <vector> is implemented in a library.

I would have preferred native containers rather than templates in F202Y (knowing that the F202Y auto-generics is all what I really need in terms of genericity)

That’s a very good point about the compactness of the notation for the builtin type. And having the most typical container implemented as built-ins would definitely help in a lot of projects.

But there wil be always new types of containers, which are not in the language yet, but you wish to use (with a similar compact notation). So, probably we should more push for requiring language features, which enable the implementation of such user defined containers. For example, the language could offer methods to define a () operator for an object or getitem/setitem methods as in Python. Although I am not sure, whether this could be done in an unambiguous way in the language.

1 Like

FYI Overloading () · Issue #119 · j3-fortran/fortran_proposals · GitHub

1 Like

From the Swift Generics documentation (to amplify what you just stated):

Many languages bake common data structures (arrays, dictionaries, tables) into the language itself. This is unfortunate both because it significantly increases the size of the core language and because users then tend to use this limited set of data structures for every problem, even when another (not-baked-in) data structure would be better.

1 Like

As mentioned in the discussion, and performances apart, overloading () as an accessor raises a number of potential issues. For instance, using it on LHS of an assignment would require returning a pointer to a component of the object, in turn requiring the object to have the target or pointer attribute. Also, it does not provide any solution for slices.

On the other hand, multidimensional arrays as first class objects have always been a major plus for Fortran compared to C-like languages.

The question is whether an implementation in a library can guarantee the same level of performances as an implementation in the core language. What is possible in languages like C++ with source code in header files can be more difficult in Fortran.

Multidimensional, contiguous, dynamically allocatable arrays are the cornerstone of much of scientific and numerical computing and thus a language like Fortran must support them natively.

For everything else the above quote from the Swift documentation is valid.