I am sorry if this is a naive question. Both are based on LLVM, so what are the use cases where a person would use one over the other and vice versa?
1 Like
As the LFortran page says, it “can execute user’s code interactively to allow exploratory work”, unlike a typical compiler. It will be nice when it supports arrays, which are on the roadmap.
1 Like
I see. So like Jupyter notebooks. Thanks!
LLVM is an infrastructure to build compilers:
In the page https://fortran-lang.org/compilers/, one can read that the new Intel ifx compiler and the ARM Fortran compiler also use LLVM.
1 Like
Both are using LLVM to generate the object code, but their front ends - parsing the code, generating an abstract syntax tree (AST) and converting that to LLVM’s intermediate representation (IR) - are being developed by completely separate teams. Thus they are likely to have different sets of bugs and missing features. The more options we have in compilers the more likely something will be able to compile your code, and then you can submit bug reports to the others saying “they can do it, why can’t you?”
Also, as mention above, LFortran has the goal of also having an interactive capability (a REPL) and being the Fortran kernel for Jupyter notebooks (which I think they’ve already achieved).
3 Likes
I’ve recently started working in a team that works on F18 Flang (the new one). One of the resources they pointed me towards was this: FOSDEM 2020 - Flang : The Fortran frontend of LLVM . They didn’t mention LFortran though, that’s why I asked.
Anyway, I appreciate the responses. For some reason it didn’t strike me that adding interactive functionality requires a whole different compiler, but then I remembered iPython and Jupyter.
1 Like
There are also several C compilers that use LLVM, but only one, CLANG, is an integral part of the LLVM project. As pmk pointed out, there is also an LLVM Fortran compiler under development, based on what started as F18. There are other Fortran compilers that use parts of LLVM, but only the F18-baased one is an official part of the LLVM project, and is now called FLANG. (I’ve heard that the previous “pretenders” that fancied themselves as “Flang” are now informally referred to as “Flung”. )
4 Likes
That is a very interesting take.
The classic Flang frontend (GitHub - flang-compiler/flang: Flang is a Fortran language front-end designed for integration with LLVM.) continues to be developed and is the base of at least the Arm, AMD and Huawei Fortran compilers. The plan is to eventually move to using the F18 (or llvm/flang) frontend when it becomes ready. Classic Flang frontend community members are also part of the F18 community and make contributions to F18.
2 Likes
I’m so old that I remember when g95 was a better compiler than gfortran . Which open source effort is in the lead can change over time.
I cannot use Flang because it has trouble with I/O of derived types. Maybe someone in the GSoC can work on this, after checking with their mentor. This program causes an ICE for
clang version 7.0.1
Target: x86_64-pc-linux-gnu
Thread model: posix
program main
! 12/19/2020 07:55 PM -- crash the flang compiler, reported at https://github.com/flang-compiler/flang/issues/961
implicit none
type :: foo
integer :: bar
end type foo
type(foo) :: x(1)
x%bar = 0
print*,x([1])
end program main
1 Like
Apologies for the issue you are facing with Flang.
Is this part of an application that is commonly used?
No, it is from a program for personal use. I am grateful for all open source efforts, and no apology is needed.
2 Likes
That’s an interesting topic. At some time there was a flang project, probably the first one, which still needed a proper Fortran compiler at the end to actually produce assembler code, so it only transferred the Fortran code into a parse tree. Now, as I understand the new flang/f18 already compiles code into a binary program? And that comes automatically with LLVM?
I had the same question, and it took me a fair amount of searching to find this discussion.
I’ve been following the development of flang for a while, and the opening question arises almost immediately when hearing that someone is developing an LLVM-based fortran compiler - might still be a bit until they have native code generation, but that’s flang’s main focus right now. Their current development tracker & integration branch is GitHub - flang-compiler/f18-llvm-project at fir-dev
The goals of lfortran sound awesome, and IMO are largely overlapping with what the flang-in-LLVM is doing (perhaps with the addition of the interactive REPL). It’s possible that the projects are not as compatible as it appears of course.
Can anyone elaborate on why LFortran and the FLang projects can’t collaborate on a common parser? It seems having one would assist in a number of downstream projects while minimizing redundant work.
At the very least, there should be a common test case reference set. I was able to find one for F77 published by NIST, but haven’t found any others for more recent versions of Fortran.
2 Likes
I somehow missed this thread. I am the original author of LFortran. It turns our LFortran and the new Flang started almost the same month a few years ago. For a while we did not know about each other, but even after I met many Flang developers in person, it is unclear how we can collaborate more tightly because the projects have different goals and priorities:
-
LFortran is interactive, so it can parse more than the standard Fortran, it can also parse loose expressions and statements. Flang cannot (currently) do that as far as I know.
-
One of the requirements for LFortran is a very fast (as fast as possible) compilation speed, which guides the internal data structures to be as efficient as possible.
-
LFortran has multiple backends. That has been the design from the beginning to not be tied to LLVM. It can use it of course (it is the default backend currently), but we also have a C++ translation backend, direct x86 machine code generation backend (very fast compilation speed, but no optimizations) and we plan many other backends (Python wrappers, Julia translation, MLIR backend, etc.). LFortran is not tied to, nor part of LLVM for this reason.
-
The previous point guides a design decision to transform AST (Abstract Semantic Tree) to ASR (Abstract Semantic Representation), which is a standalone representation, but it does not lose any Fortran semantics, everything is figured out and it is the starting point of our backends, makes it very easy to maintain a backend. ASR also allows to write transformations ASR->ASR, which we plan to use for implementing optimizations at this level, before lowering to LLVM or other tools (such as MLIR): Optimizations list (#427) · Issues · lfortran / lfortran · GitLab
-
Our main focus is native code generation. You can follow our progress on compiling this proxy app (Fortran 95): MVP: Roadmap to compile the SNAP project (#313) · Issues · lfortran / lfortran · GitLab
-
We also updated our front page with the latest status: https://lfortran.org/
-
One of our goals is to build a large team of developers and contributors and make it easy for LFortran to be included in other libraries and tools — for that reason we want to make it as easy as possible to contribute to the compiler source code. LFortran currently fully compiles in about 20s on my laptop (it assumes LLVM is already built). We are thinking of (also) making it an fpm package once fpm can compile C++ code. That would make it very easy to install and develop for Fortran users.
The parser, while tedious to initially develop, is the least interesting part of a compiler. Both Flang and LFortran have now a Fortran 2018 parser. Maintaining it is not a big problem. The rest of the compilers are completely different as explained above, so the way I see it, the best way to collaborate are things like:
-
Prototyping and designing new Fortran features for the standard committee
-
Collaborating on the LLVM or MLIR backend that both compilers could use
-
Collaborating / brainstorming how to implement some optimizations
-
Collaborating on extending (if needed) the Fortran language to run well on GPUs and other platforms. Collaborating on how to compile well for those.
-
Collaborating on creating and maintaining a Fortran standard conformance test suite
-
Collaborating on benchmarks
There is nothing wrong with having more than one compiler. On the contrary! We could have just taken the GFortran parser and port it on top of LLVM. But neither Flang nor LFortran chose to do that for various technical and license reasons. Yet GFortran is the most mature and best open source Fortran compiler.
Speaking as a Fortran user, it is very exciting to be able to use multiple compilers that are independent of each other, because that ensures the Fortran code that I write is truly multiplatform — it is not dependent on any one platform or a compiler, and thus “lives on its own”. Finally, competition is extremely good for progress. We are now seeing some new innovative ideas. I believe this is not a zero sum game and all these efforts will benefit the wide Fortran community.
13 Likes
Thanks for that detailed reply. It is interesting 2 independent teams decided to write or release a Fortran compiler at roughly the same time.
Although I mirror the sources for a number of compiler projects, I have not studied LFortran source code in much depth. While it is likely a moot point now, I was thinking the goals of both teams could have been accomplished by looking at how Common Lisp compilers were implemented. They are simultaneously compiled (for efficient code generation) but also interactive. Julia uses many ideas developed in Common Lisp compilers.
I’m particularly fond of the POP-11 incremental compiler, that included Prolog, a pre-ANSI version of Common Lisp, as well as ML.
Can you provide a bit of guidance on the situation for validating the compiler/interpreter? Is the goal to have LFortran be an interpreter for Standard Fortran, or is it more Fortran inspired?
1 Like
Yes, LFortran works just like Julia does: if used interactively, it compiles a given cell to machine code via LLVM, loads into memory and executes it. It keeps the old cells’s variables alive in memory and allows you to refernce them.
The goal of LFortran is to be eventually Fortran 2018 conforming. When used interactively, Fortran is extended to allow expressions and statements at the global scope, but that’s it. We would like to compile any existing Fortran code, both free form and fixed form. When used in the ahead of time mode, it behaves just like any other Fortran compiler.
4 Likes
Where are optimizations planned in the LFortran compiler? One possibility could be to share the MLIR based high-level Fortran IR (FIR). MLIR also has the OpenMP and OpenACC dialects which LFortran can use. If there are no plans for high-level IRs then the OpenMP IRBuilder project in LLVM can be used for OpenMP support in LFortran just like Flang does.
It would have been great to share the array descriptor and module format. But I think it is too late. I hope we can at least add compatibility layers to understand each other’s formats.
1 Like
The optimizations are planned at the LFortran’s ASR level (ASR->ASR transformations).
We plan to have an MLIR backend also, once it stabilizes. There are many disadvantages of MLIR, such as speed of compilation, but for optimizations it might work very well.
@kiranchandramohan if you are interested in collaboration, please let me know.
Sorry for this question, because I don’t know any thing about compilers, I am just trying to understand.
If LFortran compiler is written in C++, I what way it will be faster than C++? I understand that compiler is a program that converts source code into binary. If that is written in C++, then it is another C++ program. Then Fortran becomes a higher level C++. How then it becomes another language ? Are there any specific rules of memory allocation in Fortran compiler ? Or what makes it a Fortran language ? Semantics ?
If I write
print*, “hello world”
If this is processed by a C++ program, then I have written a higher level C++ that’s all ! Or am I missing anything here ?