I would like the founders of this website to contact their US colleagues listed as authors of http://www.netlib.org/lapack/lawnspdf/lawn297.pdf and ask them to explain on what technical basis they plan to abandon Fortran for the successor of Sca/LAPACK libraries (the BALLISTIC project) in favour of C, which at first glance lacks the modern features of Fortran.
I know nothing about this effort, so please excuse me for “speaking out of turn.” But can anyone expect any response beyond what is stated in section 2.9.10 Improving the existing software base, “Most scientific software written today is not based in Fortran, and a recent study of programming languages ranks Fortran 38th, with the top-3 programming languages listed as Python, Java, and C [129]”
The use of the ranking they mention, the one by IEEE Spectrum, is eerily similar to what we’ve noticed in the commercial and other domains for a while now where the low ranking of Fortran reflects a situation where further investment ($$/time) into the language is considered unwise, as I stated here earlier.
[129] Stephen Cass. The top programming languages 2019. IEEE Spectrum, 2019.
Full Page Reload.
1 Like
Related discussion here from the thread “Samples in Fortran are not acceptable.”
1 Like
The usual actual reason is that they can’t find anyone who knows Fortran (and is willing to work for work-study student rates). The official reason is usually that C is perceived to be a symbolic assembly language and should be able to produce optimal code (though the potential for pointer aliasing often prevents this from happening). Official reason #2 is usually that any other language can call a library written in C, so the library in C is more widely usable than one written in a different language. This ignores that fact that a Fortran subprogram can be written to “look like” it is written in C to the world of callers. But this bit of Fortran is new since 1966 when the Fortran education of many CS types ended.
3 Likes
I already did in 2015, I talked with one of the authors of the article about this effort. The reason is simple, Fortran is not popular and not doing well. Because if it was doing well, people would not be moving away from it to C++ (my understanding is that they are using C++, at least for some parts, not C). In fact, I also talked with another author of this article in 2007 and when I asked him why they don’t use C for Lapack, he said that they identified that Fortran is still the best choice after evaluating all the options (this suggests that they have changed their mind since then…).
Here is another high profile project moving away from Fortran to C++: NWChemEx: Computational Chemistry Code for the Exascale Era. Here is another: convert CTU hydro subroutines to C++ · Issue #525 · AMReX-Astro/Castro · GitHub.
We can’t fix the problem with Fortran if we are unable or unwilling to look at the reality straight in the eye and realize that people are moving away from Fortran to C++ one large production code after another.
If we can’t even identify what the problem is, we can’t fix it. I truly believe that.
There are some who disagree that “Fortran is not doing well”. They think it is bad “public relations”. I am not a public relations expert, so I don’t know. But even if we do not communicate this publicly, we internally have to understand that the fact that the three famous projects above moving away is just not good. There is no sugar coating of it. I love Fortran. So it hurts that we have lost the above 3 battles. But we have lost them unfortunately. Below is how we can win future battles.
I talk to people who move away from Fortran to C++ all the time. They have valid technical reasons. When we deny that there is any problem with Fortran, our users who want to use Fortran but are forced to move away are infuriated that we do not even admit that there is a problem. If you have doubts, please talk to our users who like Fortran but are forced to move towards C++. Ask them and please listen to their response.
Here is what I suggest instead as our approach how to communicate the situation: “Yes, Fortran is not doing well. Yes, people are moving away from it for good technical and social reasons. But we have identified what the problems are, we have a plan to fix it, and we are executing the plan. We need better compilers, more organized community, have a good story for GPUs, better Fortran libraries, package manager, we need more projects moving to Fortran, etc. We believe there is nothing fundamentally wrong with the Fortran language itself, we just need to rejuvenate it and the tooling around it. We have made a huge progress in just the past year. The atmosphere around Fortran is starting to change. But we are not done yet, we keep pushing as much as we can. If you can, please postpone your decision to abandon Fortran by about 3 years. We believe we can fix many (if not all) of these problems by then.”
With this approach, I believe most people actually become fans and are rooting for us. Instead of thinking “those Fortran folks don’t understand our problems”, they will think “those Fortran folks really identified our problems and are fixing it, very well!”. And many may even join our efforts.
14 Likes
I have done the opposite: taken a C++ sparse factorization package and coded it in Fortran, with every function call-compatible. You can choose an arbitrary subset to be in C++ or Fortran. No meaningful performance difference (a rival package, coded in Fortran and with finer-grained parallelism, shows better performance). But in Fortran I can ask (the NAG compiler) for all my array bounds to be checked, all my calls to be checked, all my variables to be checked for valid data, all integer overflows to be reported, find any dangling POINTERs, do a memory trace, etc. Guess which version I would rather debug.
1 Like
No argument there. So, what are the candidates and how are they ranked ? I will start, in no particular order
- Fortran code needs to run on GPU without performance penalty
- Universities don’t teach modern Fortran.
- Flagship Fortran compilers are not producing the very fastest code where it matters.
- The tooling is insufficient.
- The language is not up to scratch for millions of processes.
2 Likes
Exactly, thank you, this is all there is to it!
I don’t disagree, but there are lists of courses, tutorials, and books at the Fortran Wiki and at fortran-lang.org.
as I mentioned above, also:
- More organized community: Forum, Fortran webpage, community compiler(s), etc. (I think we might have almost fixed this one with fortran-lang.org)
- Rich ecosystem of Fortran packages and libraries, package manager (I think
fpmwill fix this) - We need more projects that started in C++ and moved to Fortran
- Make Fortran interactive, work in Jupyter notebooks, in addition to the direct compilation traditional usage, also make it work interactively like Julia or Python (I believe
LFortranwill fix this one) - Active Fortran Standards Committee that works with the wider community to improve the language itself when needed (we have made progress there also, from new members joining the committee to the incubator repository)
What else? Let’s brainstorm it.
Regarding “ranking”, I don’t know, I think all of this is important. 
Regarding 2., the universities, that is of course very important, foundational and influential, but we can’t fix that directly. The good news is that it will fix itself once we fix all the other issues. All we need to do is make Fortran exciting again for new users. I talk with university professors in CS departments. Many of them are actually not against Fortran as you might think. If we make 3. and 8. exciting enough, they will join us. There are tons of interesting CS problems to solve and work on. Either way, I would not worry about 2. and concentrate on the rest.
P.S. Also see my older post here: Embed a Jupyter Notebook in fortran-lang.org/learn - #6 by certik where I quote Fernando Pérez and Guido van Rossum, both very influential people in the Python community, who publicly praise Fortran. So we have much bigger support “out there” than you might think. We just have to fix the issues above.
3 Likes
When I first heard about rust and decided to dive into it, I almost immediately found very detailed books for free in the internet. But most important I knew where to start: They have one book (nicknamed “the book”) which is highly recommended, available online and offline for free.
If I search for “learn fortran” and navigate to fortran-lang.org I get a crash course and mini-books (which is nice), but no single book which will explain everything in detail (alike “the book”). Instead I get a list of over 50 links to all varieties of books, courses and topics which seam to be a little bit overwhelming (and untidy).
In my opinion quality is more important than quantity.
Therefore, it may be better to established “the book” for Fortran, which explains everything about modern Fortran in detail for free and with the option for the whole community to contribute. We don’t need many books as long as one book covers everything important in a comprehensive way.
3 Likes
I’d add: cheap, modern texts on Fortran for applied science. It shouldn’t be about “Fortran”, but “Computational X with Modern Fortran” where X could be: physics, chemistry, biology, etc.
Dover Publications – an entity with a strong catalogue of scientific materials of both historic and current interest, might be worth approaching.
Also – materials for high school through undergraduate instructors would be critical in reviving the ecosystem.
2 Likes
There are books on “Computational X with Matlab” for all X. Matlab and modern Fortran are similar languages, and there are partial translators matlab2fortran and matlab2fmex. There is also Mc2FOR: A tool for automatically translating MATLAB to FORTRAN 95 (2014). Back in 1996 there was a paper A MATLAB to Fortran 90 Translator and its Effectiveness. I don’t use Matlab and have not tried these tools, but the existence of a translator that did 99% of the work would open up many algorithms to Fortran.
Fortran is not going to be the first programming language, or even the first scientific programming language for most people, but it should be easy to transition to it.
2 Likes
Excellent, excellent point. Thank you @certik for explaining it so clearly.
Re: “What else?”: feature gaps in Fortran language.
The ground reality is for year 2021, the base language Fortran is feature deficient.
From simple convenience (but powerful features) such as proper enumeration type support and unsigned integers and bitset types to proper exception handling and comprehensive generic metaprogramming facilities, most budget managers will not sign off on a decision to use Fortran for any part of a significant commercial software endeavor involving many code contributors, cross-functional teams, multilanguage and multiple programming paradigms with goals toward speed-to-market and so forth.
With each of above features, one is looking at 20+ years before robust language support becomes even conceivable e.g., some generics feature (possibly diluted even relative to 1990s Ada or C++98) may go into 202Y and reasonable compiler support may come around 2035 to 2040. That’s way too late. No manager in the commercial sector will be willing to wait a quarter for their decision, especially considering so many other better-featured language options on offer with so many highly skilled and motivated people working to elevate those languages.
The computational technology lead architects and budget managers and peers laugh at me that Fortran 202X will not allow something simple like assigning values to Fortran enumerators when a programmer so desires, something every modern language including Julia provides simply as a matter of fact: “@enum Fruit apple=1 orange=2 kiwi=3”
https://docs.julialang.org/en/v1/base/base/#Base.Enums.Enum
Not acknowledging and addressing the feature deficiency is also a big problem in Fortran: simple, basic things needed toward modern applications get in way too late, often with little comprehensiveness for the practitioners. Every little thing becomes such heavy-lifting in Fortran.
So I remain extremely dismayed and saddened by the lack of recognition and acceptance and addressal by the dozen or two active Fortran practitioners posting online (e.g., here) as those with voting power (e.g., Fortran language committees) on the gaps in the base language itself.
Decisions like the one in the original post have been dime a dozen over the last 2 decades and the few remaining codebases of any significance will soon migrate away from Fortran.
3 Likes
I presume there is a way to call fortran code to speed up Matlab calculations. Is this documented anywhere?
Yes, Fortran Source MEX Files. The matlab2fmex tool I mentioned earlier translates Matlab code to Fortran so that the Fortran can be compiled to a MEX file. MathWorks would prefer that your code stays in Matlab:
MEX files are not appropriate for all applications. MATLAB is a high-productivity environment whose specialty is eliminating time-consuming, low-level programming in compiled languages like Fortran. In general, do your programming in MATLAB. Do not use MEX files unless your application requires it.
I remember that when a Matlab salesman visited my company in the late 1990s, he presented Matlab array operations and said that in Fortran you would have to write loops. That’s what the company had told him. There was an awkward silence when I interjected that Fortran 90 had array operations.
1 Like
Would it be helpful if NAG made more noise about how we produce Libraries for C, C++, Fortran, Python, Java, Matlab and .NET out of 126k LOC of C++ and 2284k LOC of Fortran? Sometimes it seems that people are not aware of the possibilities.
In the short term, I agree, but there should be a long term goal of introducing programming as a way of problem solving. Fortran is good for the numerical aspect, but other languages are currently preferable for the symbolic. This corresponds to the distinction between declarative knowledge and procedural knowledge made in logic programming (of which I am a fan of).
Matthias Felleisen (invenor of Racket and the How to Design Programs curriculum) has given much thought on:
- The need for multiple languages in a project
- The need to integrate computing science into the broader approach to mathematics education.
Worth reading:
1 Like
I love Fortran and want it to be the best tool in its target market.
At the same time, if people are moving away from Fortran for well informed technical reasons, I think they should.
If people are moving away from Fortran for well informed social reasons, such as difficult to hire, I think they should.
If people are moving away from Fortran because they’re misinformed, then I will try to convince to change their mind, and we all should.
At the same time, we all need to work hard to improve numerous aspects of Fortran and its compilers, libraries, tooling, community, marketing, and yes, public relations, so that good technical reasons to move away shrink, and perhaps even completely disappear, and that bad social reasons go away. I don’t want tools and libraries to be built in Fortran just because of Fortran. I want them to be built with the best tool for the job.
If BALLISTIC brings an improved linear algebra system over the current state-of-the-art, in a different language, I’m all for it. It’s not like we’ll lose the Fortran implementations, and people will choose. And the BALLISTIC paper promises standard-conforming Fortran bindings. As an end-user of linear algebra, I want the most stable, performant, portable, and easy-to-use library or framework. I don’t care what’s running under the hood. If it’s Fortran, fantastic. If it’s something else, I doesn’t offend me.
I do worry, however, about these shifts affecting the availability of Fortran programmers, which could (and probably has, a while ago) entered a positive feedback cycle: Fewer large Fortran projects → fewer Fortran programmers → more difficult to hire → even fewer larger Fortran projects.
I agree that the lack of university Fortran courses (in the US, at least) is a problem. However, as @certik pointed out, we can’t fix this in the short term. These trends have a time scale of about one generation, ~30 years. Even if we start reversing this now, it will be decades to reach a favorable level of college-level Fortran courses. Within that time frame, universities may become mostly obsolete, or there may be a new programming language that would make most other languages completely redundant. We live in exponential times.
Speaking specifically about the problem of Fortran at the universities, here’s what we can do (we who work at universities):
- Organize periodic Fortran seminars
- Organize local discussion or reading groups
- Teach a day-long Fortran workshop (free of charge, of course. Tip: bring snacks and drinks for higher attendance). It’s great experience if you consider doing more teaching.
- Reach out to the university-wide mail list and say you’re a local Fortran expert (yes, if you don’t feel like one, you are), and invite anybody who programs Fortran to ask you for help at any time. There may be Fortran programmers around that you’re not aware of.
- Write blogs and articles about Fortran
- Talk about and promote Fortran
Like with most systemic problems, this one we need to address locally, from the bottom up, each of us.
5 Likes
Worth remembering that it is only the reference implementation of Sca/LAPACK that is in (very naughty) Fortran. You would have to ask each vendor if they substituted other source codes. At least now I know why they have never bothered to fix the non-conformances in those reference Fortran codes.
One more for the pile of concerns: How easy is it to contribute to code-generation improvement for a Fortran compiler? I expect the LLVM folks will make a dent on that, I hope flang is not hitting big problems.
2 Likes