I have just issued v3.0.0, a new major release, for fortls the Fortran Language Server. The new release contains a lot of changes, fixes and new features that have accumulated over the last 2+ years.
I wanted to personally thank everyone, who over this long period, took the time to contribute and solve real problems that affect users. Thank you all!
You can find a manually generated and less verbose CHANGELOG for this Release here.
Future plans
Compared to other open-source language servers for languages like C and C++, our project still has a lot of room for improvement. As it usually goes, there are a lot of reasons behind this. On the human side, I don’t have as much time as before. On the software side, actively maintaining a full Fortran parser and LSP interface is a ton of work, leaving less time to work on the language-server responses.
Some of you might have noticed that I added a section in the README of the project stating our future plans with regards to funding allocation and fortls. The goal is to ease some of the burden by shifting Fortran grammar parsing to LFortran, which would let the language server focus more on its core functionality, the content of its responses. I’m also hoping to implement a similar approach for the Language Server Protocol interfaces via pygls.
It’s still unclear if these changes will happen within fortls by overhauling its core, or if it will kick off a new project under Fortran-lang. It really depends on what’s easier to develop and maintain, but what is clear to me is the need for an open-source parser that can be used in projects like fortls and ford.
Thanks again to everyone who contributed in this Release!
Personally, I would like it to be LFortran, but a compelling argument can be made about flang. Realistically and with a bit of effort flang can make their own language server.
Flang has “access” to clangd language server, their compiler parser is probably structured like the other clang compilers, and very importantly, they have the collective knowledge to implement a language server given their compiler architecture, since they did it for their C and C++ compilers.
The real challenge is taking a parser designed for reading valid grammar, erroring in invalid one, and generating compiled programs and using it in a language server, which most of the time contains invalid and incomplete code. The general solution to this is you mark erroring AST (ASR in LFortran) nodes as incomplete and instruct the parser to continue. Obviously this is easier said than done.
The beauty of fortls is that it has a lazy PEG parser which doesn’t concern itself with producing valid machine code and hence it can tolerate being wrong from time to time, while a functioning compiler cannot.
In an ideal world, we would fund multiple parsers, but since funding is scarce, it is probably smarter to first focus our resources into getting a fully functioning community compiler (LFortran) and then see how we can leverage its parser for third-party tools like language servers, formatters and auto-documentation. Therefore, the choice to not fiscally fund fortls’ parser development is an attempt to not divert resources from other ongoing efforts.
P.S. I will still continue to work on fortls in my free time.
I agree. I haven’t done a lot of work in language parsers, compilers, and language servers, but I suspect that the optimal* parser (and possibly even AST representation) for a compiler and for a language server are different.
* optimal being not just in terms of speed, but also maintainability and fitness for purpose.
That is my understanding as well. The LLVM devs behind clangd have done some great work. For this future LSP work, I believe there is great value in reaching out to available devs and asking questions in order to understand some of their choices.
I am also curious if my assumption about new flang is valid. If so, fortls could conceivably interface with flang in the not so distant future.
… On initial release, Ruff used the Python parser from the RustPython project. As Ruff evolved, we learned that a Python interpreter and linter have different needs, and the ideal AST for those two use cases can look pretty different. Ultimately, we pulled the parser into Ruff and maintained it separately as we evolved our AST structure…
Thanks @MattAl, I was unaware of Ruff’s parser capabilities, it seems like a very interesting project. The blog post pretty much sums up my own experiences and thoughts on error tolerant parsers; they are quite peculiar beasts. However, I still do hope that we can leverage some of the modern Fortran compiler parsers, it would make writing and developing developer tools like fortls so much easier!
Great work, thanks @gnikit and the rest of the fortls contributors!
You might like to look at tree-sitter-fortran. I’ve started looking at using it in ford, and it is much faster than the current regex-based parser (although that turns out to not be the bottleneck in ford by a long way).
The two big advantages of tree-sitter parsers is that they are 1) fault tolerant and 2) incredibly fast. They’re designed for things like syntax highlighting as you type, so they work really well for tolerating syntax errors because you’ve not finished typing.
But on the other hand, tree-sitter isn’t designed for things like compilers, so you still need to do a bunch of work to resolve names etc.
Congratulations @gnikit for the release and for the hard work behind the scenes in your spare time! I believe that a language server like fortls is definitely of great value for fortran developers.
Regarding which fortran parser to use in the fortran language server, I would like to contribute my five cents…
In the medium/long term I would advocate for using LLVM flang for the same reasons why we leverage the flang parser in our Codee tools. The development of flang is supported by US DoE, Arm, AMD… as far as I know. At the moment the flang project is focused on releasing a fully-featured working Fortran compiler, so the Fortran parser will eventually be complete, robust and improved by the industry.
Finally, at Codee we have designed the Codee tools to be interoperable with compilers (today gfortran, ifort and others), with CI/CD pipelines (we presented a demo using Jenkins at CUG2024 las week) and with IDEs through SARIF (leveraging the VSCode SARIF plugin). And we envision Codee integrated with a language server like fortls, so there might be opportunities for collaboration in the future.
Again, congratulations @gnikit for the great work as I was not aware of any language server for Fortran until I came across your fortls project here in Fortran-lang discourse!