I have written a Fortran package that interoperates with MATLAB both serially and in parallel (via MPI). As @Beliavsky mentioned, all MATLAB interoperations with C/Fortran are done via MeX files. Unfortunately, the Fortran MEX API is too archaic relying almost exclusively on FORTRAN77. Fortunately, Fortran/C interoperability resolves the issue and you can interoperate with MATLAB as if the Fortran code is C. If you need more help with the inner workings of the above package and the how-to of MATLAB-Fortran interoperation, I would be happy to direct you to more specific details. Keep in mind that MATLAB Mex API changes without any notice. The Fortran/Mex binary interface that you generate may work with one MATLAB version, but not with the next. Currently, I am sure at least MATLAB 2018a and 2018b have incompatible APIs (meaning you will have to compile your code separately for these two).
A lot has been said about Fortran compilers capabilities on this forum recently. On the positive side, I can attest that the package that we wrote in Fortran is orders of magnitude faster than the equivalent that we wrote in pure MATLAB.
I agree with @pmk above. Iāve been a member of the committee for about 2 years, so I am now partly responsible for this failure myself. If anybody has any ideas what I and others can do differently at the committee, please let us know. One way to contribute is to have a solid plan how to implement generics in Fortran. Anybody can join our effort here: GitHub - j3-fortran/generics. @pmk, if you have spare cycles, you can prototype some of the proposals in Flang, or even create your own āgenericsā that you think we should do. That would be a huge help. I plan to prototype them in LFortran also after we deliver MVP.
The same with macros, the best thing is to simply prototype this in a compiler and let people play with it.
One reason is that commerial Fortran vendors do not understand that by not making their compilers available for the community they are killing the language. Probably because they donāt understand open source, or they think itās irrelevant.
One example: I spent some time on trying to make scipy, a very popular scientific Python package, which uses several Fortran libraries, not only Lapack, be useful with the NAGās nagfor compiler.
I spent some time expaining to NAG people why they should provide few nagfor licences to scipy for testing purposes. (I tried to find Fortran reviewers for these little changes in Fortran code there - please, have a look if you have time).
No, nothing has happened on the NAG side. Probably NAG thinks itās irrelevant to their shareholders. Good luck to them with keeping profitable with their shrinking range of customers then.
The situation with NAG is I think a bit different from others (like Intel that is a major chipmaker, or NVIDIA). But perhaps they could make it free to the open-source developers (which would be extremely valuable to the community). Even Intel made its compilers free when Apple announced they would be moving away from Intel chip. I do not know for sure if the two events are truly connected. But the coincidence makes people like me wonder.
Compilers are difficult to write, and I think their developers deserve to be paid. I have paid for compilers from Microsoft, Digital, Compaq, Lahey, and Unicomp (for F) and think it was money well spent. The NAG site says
As a not-for-profit company, NAG reinvests surpluses into the research and development of its products, services, staff and its collaborations.
@dimpase thank you for the comment and welcome! Maybe @themos can comment on the NAG situation, I donāt know what is best for them or their policies.
The situation of the Fortran compilers is complicated by the history, that 30 years ago it was profitable to sell compilers, but things have changed since then and now it seems one has to either do open source or be attached to a hardware company. Either way, the Fortran compilers should adapt to the new economy. I think they are, but it takes time.
Thank you again for the feedback!
I have not said they NAG or any other commercial vendor should open-source their compiler! What I said they fail to acknowledge that Python packages such as scipy is by far the largest install base of Fortran libraries, and fail to do anything to let their products be used in that largest Fortran market segment. Itās pathetic shortsightedness.
What have they lost by not providing few licences for free? Nothing, to the contrary, it only makes Python people more convinced that Fortran is dying, and less Fortran will be used in Python packages, no more. That was a unique moment when NAGās Fortran was the only Fortran compiler capable of natively working on Appleās M1 Arm architecture. Surely gfortran have caught up there, or it will, soon.
So they didnāt sell their licences to M1 users who wanted to build scipy natively on M1 using nagfor, because scipy was not and still is not ported to nagfor. Why on Earth do I have to teach NAG elementary marketing, are they blind, or still think that Python will magically go away and leave them in command of scientific computing market?
I agree. Please make sure you upvote this issue:
I agree and I would even turn the reasoning around: compilers should be driving the development of the language, by implementing various extensions and the Fortran Standards Committee should standardize things that are already in use. I believe new features should not be by default invented by the committee, because it is extremely hard to do it right without a prior compiler implementation and user experience. Ideally it would be some kind of a compromise: the committee can suggest new features, compilers implement them, we get experience with them, and later on the committee can standardize.
I understand the logic, but an advantage of Fortran being standardized is that you are not tied to a compiler. At times I used F, ELF90, Lahey/Fujitsu F95, and g95. None of those compilers are currently maintained, but since I did not use extensions my codes still work with gfortran and Intel. If you are unable to compile a FORTRAN code from 1970, itās because it used an extension that seemed like a good idea at the time. When do the benefits of compiler extensions exceed the drawbacks of code potentially being stranded if those extensions do not enter the standard, and the compiler stops being maintained?
I plan to prototype this in LFortran, but first we need to get MVP out of the door, hopefully in a month or two. I am thinking a realistic time frame for me to start prototyping some of these ideas is in the fall of this year. If you are interested in prototyping this even now, I am happy to get you started in the code base. It is not complicated. Just let me know.
Yes, for production codes that you want to compile with multiple compilers, you would not use features that are experimental and perhaps only implemented in one compiler, but rather you want to use features that are supported in multiple compilers and standardized. As an example, when we released dftatom almost 10 years ago, I restricted the usage to Fortran 95. That way it is very likely to compile with almost any Fortran compiler.
But I know there are other users who would love to test out and use some experimental features, and those would then use them and report back and if people like it, we can then submit proposals to the standard committee to get standardized.
Thank you. Yes, we obviously plan to support F77 and fixed form, but I want to get modern codes compiling first.
There is not really much to say. We had an internal discussion of the pros and cons and the outcome was that the case for helping the scipy project (a direct competitor to the Nag Library for Python product, I should note) to mutual benefit simply has not been made. Our prices reflect the cost of producing a quality product and are not outlandish, I believe. There seems to be a systemic issue here, though, that community efforts find it so hard to attract the bare minimum of funds for purchasing the software tools of the trade. Others have more insight into the reasons than I do. As I have remarked before, in my private life I find it easier to part with cash for hardware tools I barely use rather than software I use all the time. There seems to be a psychological/social factor at work.
@certik I agree that the Fortran community needs to recognize the existing weaknesses. But focusing on the weaknesses alone and not boasting about the strengths of Fortran would only make the situation worse. I have the impression that Julia has received a lot of publicity by simply showing what they can offer and simultaneously hiding the weaknesses under the rug until they get resolved (Thatās not the approach Iād would personally want to adopt, but the Fortran community needs to recognize both sides and speak out both on the strengths and weaknesses).
Regarding the issue of packages migrating to other languages (like NWCHEM that you mentioned), one thing this community could do is to reach out to those developers who are still developing and maintaining Fortran software and connect (or invite) them to (join) this community for better communications and support (I do this as much as my time allows daily). For example, I have seen several cases of originally Fortran programmers in the Julia community who migrated to Julia out of frustration with working in F77 or F90, as they described, without knowing much about the recent Fortran standards. This lack of awareness about modern Fortran appears to be deep; I see it among colleagues on a daily/weekly basis. I remember @milancurcic recently celebrated reaching the 200th member joining this forum. The number of Fortran programmers and enthusiasts is a far beyond this. The Reddit community for example has ~6K members. Bringing all of these developers to a centralized (semi-formal) forum (like this) would go along way toward identifying the problems, offering community-driven solutions, retaining developers, and keeping them up-to-date with the most recent Fortran news and enhancements. People get a sense of belonging to the language when they see an active community of programmers whom they can turn to for help, when they need it (The Intel Fortran forum played the role for me over the past decade). These are some thoughts I wanted to share before I forget.
Yes, Fortran development at NAG for the last 52 years. The earliest alternative language was Algol68 and early code was developed for both. We donāt use macros. We have pre-commit hooks that use our compiler and python scripts to check for correctness and style enforcement. We automatically monitor code coverage, performance and test results with Jenkins. We write documentation in XML which produces not only the doc but all the wrappers/helpers for interfacing from the supported languages (C, C++, Java, Python, .NET, Matlab). In the last decade or so we added some C++ code to the mix, under a thin layer of Fortran. The separation of the code base into āengineā plus āauto-interfacesā was a project started by Mike Dewar and David Carlisle. In the 80s we developed a C variant of the code base (led by Shah Datardina) but the āenginificationā success meant that this was no longer necessary.
So, Algol68, C and C++ have all been in the picture but were never strong enough to replace Fortran at NAG. This was perhaps due to the maturity of the compilers but Fortran language design played a part too.
For me, the lack of datstructures, metaprogramming, and generic functions in Fortran made me stay away even though I was pretty tempted by some of itās nice features, stability and popularity in the physics community.
Having a good story in Fortran about addressing those sorts of things would be a big boon and very interesting IMO.
I have edited some of your post @difference-scheme to stay polite. @Beliavsky thank you for responding to it also, you posted a very nice reply, but it quotes the parts that I donāt want to have on the Forum, so I removed your post also. I hope itās ok.
Let me remind everybody that we should stay polite here, letās not accuse others or other languages.
We can and should discuss how to market Fortran and how to speak externally and internally about Fortranās weaknesses and strengths. I am personally firmly in the camp to acknowledge and be very fair and objective about Fortranās weaknesses and there are many. But there are many strengths and still unrealized potential and I believe in Fortran. (And others might have other opinions about how to market and letās discuss that here. But we will not be accusing others.)
@mason, I am really sorry about the accusations. I have removed them above. If it is ok with everybody, letās move on.
I think the price of the NAG compiler is rather reasonable, but I also feel that the mental barrier (?) for purchasing any compiler (not limited to NAG) has been becoming higher than before. This is probably because various languages distribute their compilers or interpreters for free and people are often used to that situationā¦ (regardless of the quality of them).
But I was previously impressed by the robust bug detection capability of the NAG compiler, so I wonder how the benefit could be āexperiencedā more effectively by potential users. One approach I imagine is to ālower the purchase barrierā by following some other software models. For example, the compiler may be used for free or at a relatively low price up to F95 features (including syntax check), and $xxx unlocks the next feature set (F03), and $yyy further unlocks F08 (coarrays), and $zzz for all features. If one unlocks (= purchases) all features at once, there is some reduction in the total fee, etc, etcā¦ Then I imagine users may have more opportunity to experience strong debug capability, than paying $zzz at once from the beginning (which may have a high mental barrier IMO).
Other models might be subscription-based (?), like $xxx per month etc (and people often forget to cancel the license even not using it ā my patternā¦)
This is an interesting point. Certainly the julia community isnāt shouting from the rooftops about juliaās flaws, and Iāve seen users give unrealistically rosy pictures of the language before, but I think the community is usually pretty good at acknowledging itās problems too. E.g. if you go on the Julia Discourse, youāll find endless threads about compiling to small static binaries, lots of discussion around getting various packages to version 1.0, tonnes of discussion around JIT compilation latency, etc.
I think though that thereās a difference here that requires some strategic thought. Julia is trying to enter various industries as a disrupter, whereas it seems that one of the biggest concerns with Fortran would be to stop people from abandoning it.
When you have a newcomer language like Julia, you often have very little time to make the case for why someone shouldnāt just ignore it. When youāre trying to convince someone who has invested hundreds or thousands of person-hours into a project not to abandon a language like Fortran, thereās much more room for rational discussion, and this can be reflected in the messaging.
I know that when I started my masters degree and I was talking to my supervisor about programming tools, he said
I use Fortran for everything and Iām too invested in it to change. You should use something else, thereās no reason for you to learn Fortran
Convincing people like him that Fortran isnāt a dead-end and wonāt hurt his studentās careers would not only help stem the tide of people leaving the language, but also help induct new people because supervisors like that can play a huge role in the choices their students make.
Yes, that mentality about Fortran is quite pervasive and it frequently, virtually always, stems from a deep lack of awareness about modern Fortran. I think Fortranās biggest barrier at the moment is to change that dominant mentality. The works of @certik @milancurcic and others (forgive me for not knowing all the names) in this community have been quite effective in just a year. The messaging and advertisement needs to get more aggressive.