@rsci all your issues are fixed in main. Go ahead and try again.
Progress update. For context, I’m trying to compile two atmospheric modeling codebases with LFortran:
-
An operational global numerical weather prediction model (~300k lines of mostly Fortran). With LFortran 0.67.0-61, I’ve managed to compile all 254 compilation units. Linking is still in progress, and it’s not yet clear how the lack of default
REAL(8)promotion will affect the model’s functionality. -
Our next-generation atmospheric modeling framework, written in modern Fortran (~114k lines, 650 source files). Here the progress is even more encouraging: I’ve managed to compile and run the hydrostatic atmospheric model on multiple MPI ranks. These are short, idealized test cases, but the output files are byte-for-byte identical to those produced by GFortran 16.1.1.
Both efforts required several compiler workarounds, and I’ve reduced the issues to MREs and reported them.
For MPI, I’m currently using the fortran_mpi bindings, which I’ve extended with missing interfaces with the help of codex. I also plan to repeat the results using Open MPI and possibly MPICH.
Of course, these are very preliminary results, and much more testing and numerical validation are needed.
Many thanks to @certik and the LFortran developers for the rapid fixes!
3 Likes
@rsci this is great news! I feel we are very close to beta now if your modern Fortran compiles and runs correctly. I am fixing all your new 17 bug reports now, should be done soon. We’ll implement the real(8) promotion soon. Submit a PR to fortran_mpi please with everything you had to add. I’ll figure out how to clean it up and get it in.
Once we are done with correctness and most such codes (like yours) work, we’ll focus on performance to run very fast. Also speed of compilation.
2 Likes
@rsci all your issues are fixed. If you have time, do you mind please trying again and reporting anything else that fails?
2 Likes
Do yourself a massive favour and add an
IMPLICIT REAL(desired_kind) (A-H,O-Z)
to your codes, remembering that 8 is not meaningful across compilers (i.e. is not required to be what you think it is or even exist as a valid kind number).
As far as I understand, IMPLICIT REAL(desired_kind) only affects implicitly typed variables. It doesn’t change the kind of explicitly declared REAL variables or real literals like 1.0, right?
Of course, relying on default real kinds isn’t the best practice, but with this legacy atmospheric model, I just wanted to see if we could get it compiling with minimal code changes.
For our new codebase, we do our best to stick to standard-conforming Fortran and good coding practices.
Yes, you are right. I have a conditioned reflex of recoiling in horror when I see codes still relying on such tricks from the compiler. Even if compilation succeeds, the chances of unintended and unforseen runtime consequences are significant.
@rsci I see the 4 issues from this morning (one of them already fixed): Issues · lfortran/lfortran · GitHub. I’ll fix them soon.
What is the current status of everything — do the two codes now work and tests pass, if you workaround the last 3 issues?
Yes, after updating LFortran, all the previously reported issues are gone.
Of the four new issues, #14699 is probably the most annoying, since it only shows up at runtime. So if you think it could be fixed soon, I’d rather wait for it before next try.
1 Like
Excellent. Two PRs are submitted, the third one in the making. Should be fixed by tonight or tomorrow the latest.
@rsci all your bugs are fixed. The slow compilation bug Excessive LLVM code generation time for repeated literal array-section assignments · Issue #14698 · lfortran/lfortran · GitHub is fixed for the most part, hopefully it unblocks you (if not, let me know). I have about 5 more PRs on the way to speed it up further and I’ll iterate on it until it’s as fast as other compilers. I am hoping the current speed is enough to unblock you. Let me know.
2 Likes
@rsci I got all PRs in, the compilation speed on Excessive LLVM code generation time for repeated literal array-section assignments · Issue #14698 · lfortran/lfortran · GitHub should now be fully fixed. Let me know if you still see a bottleneck. The actual benchmarks I ran are in the description of this PR: perf: assign a named constant to a contiguous section with one block copy by certik · Pull Request #15103 · lfortran/lfortran · GitHub, it seems main is now in good shape.
2 Likes
Some more progress on my side: I’ve tried building both MPICH and Open MPI with LFortran. Open MPI already works with mpif.h and use mpi, while mpi_f08 and MPICH still have some blockers (Open MPI, MPICH).
I’ve also started more extensive testing of our modern Fortran codebase including checks for memory leaks. Found and reported a few new LFortran bugs, and several gfortran bugs along the way. The tests unaffected by current issues produce results identical to gfortran and ifort.
I’ll do another testing pass and share some performance numbers as well.
1 Like
Excellent, thanks for all the bug reports.
I fixed the blocker for OpenMPI.
For runtime performance use --fast with LFortran. This might trigger new bugs, so just report them and we’ll fix it. I am hoping we’ll be within 2x from optimized GFortran. We’ll focus on runtime performance after we reach beta, until then we just need to be in the ballpark.
@rsci I fixed all your issues except the two OpenMPI and MPICH tracking issues. I did fix all the blockers in them I think. I would recommend to report dedicated issues if there are more bugs. It looks like both use invalid Fortran. I would recommend still reporting it in a dedicated issue, and check both Flang and GFortran (or other compilers if you have access to them). If they all support it, we can as well, with some options. Invalid Fortran (for example mismatched arguments) makes it super hard to decide if we should support it, because that’s always a sign that either the upstream code should be fixed, or the Fortran language itself needs some extensions to support the use case, but just adding it to a compiler (and no other effort) is not the most optimal, as that keep kicking the can down the road. I rather either fix upstream or fix Fortran.
3 Likes
I hate the fact that there are at least 2 different compilers called flang. The one I have is AMD flang, and IMHO ifx and gfortran are better. I also use g95, which omits some f2003 features. All those compilers cost me $0.00.
Thanks a lot for all the fixes! I’ll report back when i get a chance.
I completely agree about the compiler-specific details and nonstandard features. It’s a bit sad that even such widely used HPC libraries still rely on these. On the other hand, HPC and parallel programming are a big part of the Fortran world, so I think it’s important to make using MPI with LFortran relatively straightforward. For many people, this could be a deciding factor in whether to give LFortran a try.
Maybe some of these issues are worth reporting upstream to Open MPI and MPICH, though I’m not really sure what can realistically be fixed on their side or what would be most useful to prioritize.
Also, I just came across the recent MPICH 5.1.0a1 release, which introduces standalone Fortran MPI bindings that can work with any C MPI implementation supporting the MPI 5 ABI. Since it’s a newer codebase, maybe it’ll avoid some of the issues we’ve run into.
1 Like
@rsci I agree, we definitely need to get LFortran working with OpenMPI and MPICH. In the past I submitted patches to other projects that use non-conforming Fortran to make them conforming. So that’s the best way. Often times these projects didn’t even realize that they use non-conforming Fortran and that there is a way to make it conforming.
The new MPICH approach seems similar to the fortran_mpi library above, and I think that’s the way to go. If done well, the Fortran compiler optimizes and inlines the Fortran wrappers, so there is no overhead, the final code just calls the C MPI directly and efficiently.
So I think a good plan forward is to get LFortran working with OpenMPI and MPICH one way or the other. And I want to finish fortran_mpi as the recommended way to use MPI from Fortran and get it working with all codes and all compilers.
2 Likes
I think I can now draw some preliminary conclusions from my attempts to run our atmospheric modeling codebase with LFortran.
LFortran can now build and successfully run all 59 test drivers of our modern Fortran atmospheric modeling framework (~114k lines across 650 source files). The tests have been run on a single CPU core and with up to 12 MPI ranks. There are still two or three small compiler workarounds remaining, which I’ll report separately. So far, all numerical results agree with gfortran, with several tests producing byte-for-byte identical outputs.
Given the size of the codebase and its extensive use of modern Fortran OOP features, I think this is a pretty significant result.
I’ve also made a first optimized performance comparison using the hydrostatic atmospheric model. For a short experiment on my local machine (3 MPI ranks, Intel i5-12400), I obtained:
| LFortran | GFortran | |
|---|---|---|
| Clean build (357 application source files) | 124 s | 63 s |
| Model execution | 13.1 s | 7.7 s |
This uses LFortran 0.67.0-965-gd9048e2e6e (--fast --march=native) and gfortran 16.1.1 (-Ofast -march=native). The MPI bindings differ between the two builds. Of course, these are preliminary workstation measurements, not a comprehensive benchmark.
So runtime performance is already within the same order of magnitude as GFortran, about 1.7× slower in this particular experiment.
The compilation results are particularly interesting. The complete build takes about twice as long, but the median per-file compilation-time ratio is only 1.1×, and LFortran is actually faster on 167 of the 357 common source files. Much of the additional compilation cost is concentrated in a relatively small subset of high-level modules and factories.
For example, consider a nearly empty program:
program probe
use h_model_factory_mod, only: create_h_model
end program
It takes 4.1 s to compile with LFortran versus 0.04 s with GFortran. The imported module has 352 transitive dependencies, and LFortran spends most of the time in ASR passes and LLVM optimization, even though the imported procedure is never called.
Overall, LFortran can already run a large, heavily object-oriented scientific Fortran codebase with good numerical agreement and reasonable performance. For perspective, gfortran only started compiling this codebase out of the box with version 16.
4 Likes
@rsci ok, this is much better than I expected for LFortran. I am especially happy that it both gives the correct answer and the performance is only 1.7x slower than GFortran. The compilation speed is also within 2x. So overall this is good enough for beta. Please report the remaining bugs, as well as any bugs as you test it more, I am sure there will be some. Let’s make everything just work.
Regarding runtime performance, I started working on GPU offloading of do concurrent and optimizing for both NVIDIA and Apple GPU, and so far what I see is super encouraging, I think we’ll be able to extract high percentage of the available performance quickly. We’ll do the same for a CPU, I know exactly how to do it and this will be the most fun, getting the codes to run at maximum performance. We’ll do all this after beta.
Regarding compile-time performance, my goal is an order of magnitude faster than GFortran, that seems to be the potential. Right now we are 2x slower on your code, but as you found out, we do lots of inefficient things, like the one you found out. Fixing those, and writing our own backend (equivalent to LLVM) should speed things up about 10x, at least when compiling without optimizations. The internals of LFortran have been designed from the start to be as fast as possible, so the foundation is good.
Btw, did you compile LFortran in Release mode? If not, you should to do it (./build_release.sh), it’s a lot faster for compilation speed.
2 Likes