Lfortran testing

Fortran users may like to know what happened when I started using Lfortran recently. Its own web page says it’s “in alpha”. Having tried it with about 150 test programs (some copied from comp.lang.fortran, some from Fortran Discourse, some written by me) I found 8 bugs and 2 standard-conforming features not yet implemented. Many thanks to @certik, who dealt quickly with my bug reports, and to the rest of the Lfortran team.

6 Likes

Of those 150 test programs, how many worked correctly?

Over 120 of the 150 worked correctly. I can’t say exactly how many because some exposed the same bugs or unimplemented features as others, or were not standard-conforming and didn’t cause internal compiler errors. I didn’t keep notes of those. ICEs are always worth reporting. A compiler-writer once told me that compilers should compile good programs and tell you what was wrong with bad ones.

2 Likes

Absolutely. Thank you so much @Harper for testing LFortran. I am pleasantly surprised that it could compile so many of your programs. Once we reach “beta”, then the expected behavior will be that it must either compile it and run correctly, or give a nice error message. Never any ICE. Right now we are in “alpha”, meaning the expected behavior is that it will break on your code, and then when it does, definitely report us all bugs and we’ll fix them.

Sorry, but something was very seriously wrong with my testing system. I have now revised it and tried 190 programs. Many of them were written to test for bugs or unimplemented features suspected in various compilers. Some are not standard-conforming Fortran. Lfortran had 56 good compilations and 45 good runs. The other four compilers I tried (AOCC flang, gfortran, ifort, ifx) had from 129 to 145 good compilations and from 119 to 136 good runs. “Good” in this context means merely that a particular compiler reported no error with a particular program, not that the output complied with the Fortran or IEEE standards.
Much of lfortran’s apparently poor performance is because many of the programs found the same few bugs or unimplemented features.

1 Like

Thanks for the update. That’s more like what I would expect. If you report the bugs (I think you already did), we’ll fix them.