Generics in Fortran 202Y: Petition to WG5?

[certik]

A good intro to generics is this document that I helped create a few years ago: generics/theory/comparison/comparison.md at main · j3-fortran/generics · GitHub.

As the document shows, C++ has “weak concepts”, while the proposed generics are “strong concepts” like most other languages (Go, Rust, Haskell, etc.).

Here you can browse more examples: generics/examples at main · j3-fortran/generics · GitHub. @kimala an example with more than one template parameter: generics/examples/matmul/LinearAlgebra.F90 at b8679d4ffbb76e1eaf55d6e1774cc67af48cb0c1 · j3-fortran/generics · GitHub.

[AniruddhaDas]

I just say the latest paper on generics and it not only reverted the changes made to the deferred argument delimiter from {} to (), but also introduced a new inline instantiation syntax ^(). What happened, can we know the reason for this change?

[certik]

@AniruddhaDas wow, you are right: https://j3-fortran.org/doc/year/24/24-126r4.txt

Instead of

SUBROUTINE S2{T, F}(Y)
...
CALL S2{integer, operator(-)}(i)

they now use:

SUBROUTINE S2(T, F)(Y)
...
CALL S2^(integer, operator(-))(i)

@everythingfunctional do you know the reason for this change? I like the previous one better.

This was a change from https://j3-fortran.org/doc/year/24/24-126r3.txt. So within one day a new paper (r4) was submitted, and passed at the committee, changing syntax that was set for almost a year, and even implemented as a prototype in a compiler.

Don’t the INCITS rules of the committee say that all substantial papers must be submitted 14 days prior to the committee start, so that members have enough time to read them (and ideally seek community feedback on big changes)?

I keep track of the generics and the overall direction of the committee and the generics subgroup in particular, and this is a big syntax change that should have been discussed with the community.

[AniruddhaDas]

@certik

So within one day a new paper (r4) was submitted, and passed at the committee, changing syntax that was set for almost a year, and even implemented as a prototype in a compiler

This way one can actually see why no compiler will be willing to implement/prototype features. I am not sorry to say this but this leaves a very bad impression, how important the reason for this sudden change might be.

In this meeting plenary should have notified for this change, even if it’s one compiler that is putting the effort to prototype it, and solidify it in the next meeting.

[tyranids]

lol, just why… New syntax seems clearly worse and extremely “Fortran unique” for no good reason, just like derived_type%field

[Jweber]

I also think (personal opinion not at all familiar with language specifiation) that the curly braces are easier to “parse” by humans, as they distinguish templates from normal functions (and arrays …).

The hat opertor is used as bitwise or arthmetic operator in other languages.

[certik]

Every language (Go, Rust, C++, Swift, Julia, …) uses different parentheses for arguments and template parameters. It’s easier to read to realize it is a generic function. And Fortran in the old proposal did too.

An argument might have been to save {} for some other feature and to only use existing characters. But ^ also introduces a new character, typically used for bit operations or exponentiation in other languages.

All in all, this seems strictly worse, but let’s hear the counter arguments.

The main problem however is that the committee didn’t discuss this with the community to seek feedback, nor required to test this in a compiler first, to see if there are any parsing and other issues. I should not need to be at every committee meeting to ensure no last minute changes get quickly proposed and passed behind closed doors. The committee has to have my trust that it will proceed in the direction it has been going in months, or discuss any major change well ahead of time.

Maybe it’s just one big misunderstanding. Let’s hear from the committee.

CC @everythingfunctional, @tom_clune, @rouson, @sblionel, @gak.

[sblionel]

Parsing issues were discussed - there are representatives from multiple compiler developers at the meeting. All of the committee documents are public - if we waited to get “community” response to each and every decision, nothing would get accomplished.

We are some two years out from finalizing technical content; if the choice here, which applies only to “inline instantiation”, proves unworkable, we will find another solution. The use of ^ was voted the “least objectionable”. Several of us, myself included, objected to {} as being visually difficult to distinguish from (). We did discuss use of ^ in other languages and didn’t feel that was relevant to Fortran.

You’re still a member of the committee though you no longer attend meetings. Posting here is not a way to get your voice heard.

[tyranids]

@sblionel

Several of us, myself included, objected to {} as being visually difficult to distinguish from ().

Isn’t this an argument in favor of curly braces? I would count myself among those “objecting to {} being visually difficult to distinguish from ().” If anything, more instances of multiple layers of (((parenthesis))) is what will prove difficult to read.

[jwmwalrus]

@certik

An argument might have been to save {} for some other feature and to only use existing characters. But ^ also introduces a new character, typically used for bit operations or exponentiation in other languages.

I also liked the curly brackets version better, but I think the double parentheses, (...)(...), is more “Fortranic”, i.e.: it’s already used for substrings of array elements and for the PDT constructor.

If generics ever get extended to account for derived types, it will (imho) look less out of place to have this

type :: mytype(k, l, T)
    integer, kind :: k = 4
    integer, len :: l
    type, deferred :: T
end type

Than this

type :: mytype{T}(k, l)
    integer, kind :: k = 4
    integer, len :: l
    type, deferred :: T
end type

And… wouldn’t integer, kind and integer, constant serve the same purpose?

[sblionel]

There wouldn’t be multiple levels of these characters in this context. Please read the paper for more details. You could also look at j3-fortran.org/doc/year/24/24-125r4.txt for the choices we considered.

[sblionel]

@jwmwalrus

And… wouldn’t integer, kind and integer, constant serve the same purpose?

I objected to the new keyword constant as having too much overlap with parameter. General agreement was to use deferred, parameter here, which opens up the possibility of additional kinds of things being deferred.

[certik]

Thanks Steve for the feedback.

My understanding was that the {} syntax was pre-approved at the last meeting.

You’re still a member of the committee though you no longer attend meetings. Posting here is not a way to get your voice heard.

I do not have time to attend every meeting, but I have participated through the generics subgroup and discussed the important decisions for the generics as well as prototyping it in a compiler. You do not want me to do this work anymore?

[sblionel]

I know you have been prototyping the generics/templates stuff - that’s valuable. What I object to is what I consider “throwing stones”, especially when you’ve been part of the process in the subgroup. You know how we get work done.

It’s not unusual for early decisions to get revised as we get more experience, especially this early in the process. The initial choice of {} was to avoid an ambiguity that has since been resolved in other ways.

[certik]

@sblionel

I know you have been prototyping the generics/templates stuff - that’s valuable. What I object to is what I consider “throwing stones”, especially when you’ve been part of the process in the subgroup. You know how we get work done.

Thank you. I object to the accusation that I am “throwing stones”.

Yes, I know well how the committee gets work done, and I think it should do things more in the open.

[sblionel]

And you know that INCITS and ISO don’t allow us to be any more open than we are.

[FortranFan]

@sblionel

. INCITS and ISO don’t allow us to be any more open than we are.

failure of imagination is what comes to the fore here.

INCITS and ISO are effectively there for standardization, not development of anything, be it analytical method or testing protocol or a programming language.

The development can easily occur outside the realm of these bodies and once that is completed and validated to a degree of practical utility can it be carried for international standardization.

The real issue is the Fortran compiler vendors are effectively forcing a square peg into a round hole by deferring many crucial aspects and steps of language development to the ridiculously bureaucratic and opaque workflow of INCITS and ISO and being inadequately cooperative with the TRUE advancement of the language.

[rwmsu]

I think this discussion raises the question (at least for me), do we really need a standards committee beholding to bureaucrats in the age of Open Source. Way back when when computer hardware was only Big Iron and each vendor had a compiler that would only work on their hardware, standards were important. I think they are less so today. I’m for letting the Open Source compilers lead instead of follow. Let them implement things in ways that make sense to the user community and let the best ideas and implementations become the de facto standard. In other words let LFortran, Flang, and gfortran cooperate and lead language development in a manner similar to what Python and Julia do. If the commercial developers don’t want to follow along then thats their choice. Users will gravitate to using the compilers that give them the features THEY want and not what a few members of the committee want. Most HPC systems I’ve worked on in the last 20 or so years has offered users a variety of compilers, not just a vendor specific one so the old reason for standards is just not as valid as it once was. Also, as much as people try to interact with the committee either by direct participation or via the J3 proposals web site, I don’t see many (if any other than the one we all agree on -generics) of the ideas posted by the user community at large on the j3-fortran/proposals site actually being implemented. Instead, we are forced to live with the preferences and predjudices of the committee members. For all the talk by committee members of being open to listening to users and what they say they need in terms of missing features, I don’t see that in the committee’s work product.

[everythingfunctional]

@certik

@everythingfunctional do you know the reason for this change?

There was an argument made that we should not allow templates and procedure names to be the same in a given scope, so advance instantiation of a standalone template procedure must rename it. With that change there was no longer the argument that the syntax is ambiguous when parentheses are used for template arguments, which was motivating the usage of {}. It’s worth noting that until last meeting (February) we had been using parentheses, not curly braces.

Without an argument that we needed to use the only pair of brackets we have left for this feature, the committee decided we should go back to using parentheses. However, given the possibility of nesting inline instantiation, the committee felt an extra character to signal it happening was warranted.

@certik

Don’t the INCITS rules of the committee say that all substantial papers must be submitted 14 days prior to the committee start

Yes, and these papers were. The only changes we made to them during the meeting were relatively minor compared to the feature as a whole. I.e. specific symbols and some spelling of keywords. I’ll admit this is a potentially contentious/controversial one, but it’s not a substantial one.

@certik

this is a big syntax change that should have been discussed with the community.

I don’t think it is a big change, because it’s just the symbols used, not any reorganization or extra constraints. Controversial perhaps, but not big.

I’m sympathetic to the argument that we should have stuck with {}. It is still my preferred choice, but neither I nor any other one person gets to make that choice. It’s a committee and those in attendance get to vote on it. That said, it’s still not set in stone (actual edits to the standard haven’t been made yet), and if there is a real ground-swell of support from the broader community (write to your representative/compiler vendor sort of thing), this is still pretty easy to change. Also, the standards body doesn’t actually have any real power over implementors, so if they all decided to allow curly braces as an alternative the committee couldn’t stop them and then would have to deal with the reality on the ground. The community does still have the final say in some sense.

It’s design by committee. It will never be perfect. We are doing our best to at least find the things that make people the least unhappy while at the same time being able to make progress. We’re all volunteers, and none of us can work on this full time, so for those of you complaining about this or any other decision the committee makes, I, and I think all the members of the committee, would appreciate if you could grant us at least some amount of understanding on that front. Criticism and feedback are fine and welcome, but expecting perfection and full conformance with your specific vision is unreasonable, as is accommodation for your feedback before making decisions if you’re not willing/able to attend the meetings.

Hopefully the above answers some questions. If you do still have some constructive and polite feedback, I’m happy to hear it and take it back to the committee.

[PierU]

@rwmsu

I think this discussion raises the question (at least for me), do we really need a standards committee beholding to bureaucrats in the age of Open Source.

Raising the question is perfectly legitimate, but doing it in such a polemical way is not so…

@rwmsu

I’m for letting the Open Source compilers lead instead of follow. Let them implement things in ways that make sense to the user community and let the best ideas and implementations become the de facto standard. In other words let LFortran, Flang, and gfortran cooperate and lead language development in a manner similar to what Python and Julia do.

Python and Julia have a reference implementation, and everyone else follows, which makes a huge difference. If lfortran, flang, gfortran, all develop their own ideas, how will they coordinate at the end to avoid a divergent evolution? Maybe they will have to setup a committee for that… Oh, wait…

Note that absolutely nothing, and especially not the committee, prevents any compiler editor/writer to implement whatever new feature/extension they like… This mostly does not happen, and blaming the committee for that is nonsense.

@rwmsu

If the commercial developers don’t want to follow along then thats their choice. Users will gravitate to using the compilers that give them the features THEY want and not what a few members of the committee want. Most HPC systems I’ve worked on in the last 20 or so years has offered users a variety of compilers, not just a vendor specific one so the old reason for standards is just not as valid as it once was.

The availability of several compiler is not enough, one has also to consider the performances, the ecosystem around the compiler, the support… And quite often the vendor compiler has an edge. Not to mention cases where there’s no choice : if you want to write Fortran kernels for Nvidia cards, you have to use the Nvidia compiler (AFAIK).

Finally, note that C++ is a highly successful language, and yet it is ruled by a committee and an ISO standard (bureaucrats).