Note that Julia gives you some really powerful tools for ensuring reproducibility. The system of having a project.toml and a manifest.toml make it trivial to get exactly the same versions of all your packages in the future. At the end of the day, the best way to get reproducible results is to run the same code, not code that the author promises doesn’t break anything.
2 Likes
I agree, I think there is a cathedral and bazaar/encyclopedia brittanica and Wikipedia type disagreement here. Based on your posts it looks like you like and trust centralized ownership by identified groups and standard commitees; I like and trust decentralized development by amateurs working together to interoperate (the extent to which this is true in Julia has to be seen to be believed). So it’s not particularly surprising that you like fortran and I like julia. I’ll just note that neither of these models is inherently better for long time stability of code. What matters is commitment from developers to ensuring it, and I think the julia community has displayed that. There is also two models here, both with advantages and disadvantages: the Fortran model of ensuring backward compatibility at all costs, and the julia model of making it very clear where compatibility is broken (look up semantic versioning if you’re not familiar) and having tooling to make it easy to freeze a version of the language and libraries. Of course Fortran has a fifty years leg up, I’m not disputing that.
I’ll stop engaging now because I don’t want to be seen as a julia zealot and parasite discussions : I just wanted to address what I saw as misconceptions regarding MPI.jl.
2 Likes
Let’s hope that Julia developers don’t trick themselves by keeping a 0 major version forever and breaking the API along with it forever. Otherwise, the story of the Python ecosystem will repeat itself in Julia. Languages like Julia and Python are great. But they also enable novices with great creative ideas to implement packages without having any professional software engineering skills.
It’s worth noting that in the past year or so, a lot of the base packages have reached 1.0. (Plots, and DataFrames both recently reached 1.0, and SpecialFunctions, CSV, and DataStructures are all probably going to get to 1.0 by the end of the year). Also, the entire Standard Library is all 1.0, which contains a ton of useful functionality.
There are definitely a bunch of important packages that aren’t close to 1.0 (Flux, JuMP etc), but that’s not really that surprising given how young Julia is, and how rapidly ML is evolving.
2 Likes
Yes, and the compilation itself, which must be repeated in each session, can take a long time. Loess and lowess are local linear regression algorithms developed by W. S. Cleveland and coded in Fortran that gfortran compiles in 0.13 s on my computer. Julia takes 36 s to compile loess.jl. Given this speed difference, it can be argued that Fortran (and even more so R and Python/Scipy/scikit-learn, which package Fortran codes such as loess) is better suited for interactive use than Julia, even without a REPL, which is being developed in LFortran.
3 Likes
You can also call the precompiled Fortran library from Julia, and then you won’t have the compile time problem for that library while, at the same time, being able to develop other custom parts in Julia, which can be made as fast as possible. (why that specific code takes so much to compile is something that should be figured out by the developers, that is not exactly “normal”).
But yes, being able to develop standalone precompiled libraries and reducing interactive compile time are two of the priorities of the Julia community, and are important to make Julia an alternative for developing programs that must be quick to startup, and also to improve the user experience.
edit: what takes time in that example in the Julia version is the loading of the plotting library and plotting for the first time. Without that the “script” runs in 3s standalone (including julia startup and compilation), and runs “instantly” the second time if loaded from the same Julia section.
3 Likes
Note that the compilation does not need to be repeated each session necessarily. Julia has had ahead of time compilation through PackageCompiler.jl for some time now, it just takes a long time to compile and produces rather large binaries. People have been using this for ages to bake packages like Plots.jl and DifferentialEquations.jl into a sysimage so that there’s no startup overhead with them.
There’s an open PR currently to make the process of incrementally building images much much faster Faster incremental sysimg rebuilds by Keno · Pull Request #40414 · JuliaLang/julia · GitHub, which will make interactively using PackageCompiler.jl more reasonable.
On the other end of the spectrum, the groundwork has already been laid for small static binaries by our GPU compilers which must operate in this way, and there have been proof of concepts that just reuse the GPUCompiler.jl infrastructure for small binary static compilation to CPU targets.
2 Likes
I still see people point at a poorly written F77 code with moutains of GOTO and blaming Fortran for being very difficult to understand. That is, for sure, unfair to Fortran, especially modern Fortran.
I feel the problem is really how to really make Fortran popular again, among majority of people. It is like a tree, the more broad and the deep the roots, the more robust and stable the tree will be.
Because being superior alone does not guarantee success. Like, perhaps there are basketball players in history who are more capable than Michael Jordan, but no one can compete MJ in terms of popularity. I wish Fortran can be more like MJ than some higher level but less known figures.
If I put my feet in the shoes of the majority average users, by majority I would say that let me just limit the OS at Windows. Also, let me look at the thing from a very regular user’s point of view, who is by no means an expert. Things that preventing Fortran from being popular to the majority of people are the following,
-
Lacking a Killer application that can catch majority people’s interest. If we have things like TensorFlow, Pytorch, scikitlearn, etc, or at least some packages used by Python or other language are mostly/exclusively written by Fortran, which people have to reply on, then it will be difficult for Fortran not to be popular.
-
Lacking a good and easy to use “One-Stop” solution: IDE+Compiler. Visual studio is the best, but in the past people have to purchase Intel Fortran or find some ways to use Intel Fortran for free. Gfortran on windows does not have very powerful IDE. But overall, installing a good Fortran environment itself is not very convenient for everyone. However, ever since Intel decided to make the Compiler and MPI free, as Intel OneAPI, I feel things has changed. This is a positive signal. Now it is easy to install IntelAPI with a free Visual studio version. So this problem can be solved.
-
For average user, using Fortran vs Python/Julia is very like using Latex .vs. Word. People might want to prefer to see what they typed on the fly instead of waiting the whole article being compiled. They have the need to check the smaller pieces of code on the fly. So from this point of view, Fortran is not the most convenient. However, I saw there are very interesting and promising project like LFortran or something similar, even Intel perhaps is trying the same thing, that is, making Fortran more interactive. I do wish they success and they have to. I assume this problem can be solved.
-
Easy-to-install packages. There are many good Fortran packages requires the use of Cmake, which perhaps is still a little too difficult for average user who just want a button, so that installing packages is just as easy as Python/Julia. If it can be more easy to install and use packages/libraries, Fortran have decent chance of rising again. Because it is just convenient.
Finally, what I said can be wrong, but I do wish modern Fortran can be more and more popular, simply because I love Fortran.
Actually I am not qualified to make any comments. Because in order to answer this question in a meaningful way, it is will be best to have decent experience on Julia too, so that can make more useful comparison.
3 Likes
SciPy is still about 20% Fortran. You can browse Fortran applications here: Packages — Fortran Programming Language.
VASP is one such killer application, it is usually the most used code by CPU-hours on many HPC clusters.
But I see what you mean: Fortran does not have a vastly popular, open source application written in modern Fortran which is developed in Fortran and receives lots of contributions. SciPy has lots of Fortran code, written in F77. Lapack is probably the most popular, relatively actively developed, but unfortunately still in fixed form and not modern Fortran.
The electronic structure codes such as Quantum Espresso are relatively modern Fortran, decently popular and developed at GitHub.
As the Fortran Package Manager (see below) becomes more popular, we will see an uptick of solid libraries being developed, and the ecosystem will expand.
GFortran and LFortran are now in Conda I believe. We still need to create a solid language server for code completion and symbol lookups inside VSCode, and provide simple step by step instructions at fortran-lang.org.
That is precisely the goal of LFortran. We are making good progress. You can follow us on https://lfortran.org/. If anybody is interested in helping, please let me know, we are looking for new contributors.
This will be fixed by the Fortran Package Manager: GitHub - fortran-lang/fpm: Fortran Package Manager (fpm)
5 Likes
Besides this, Python, R and Julia have data frames that can store a “matrix” of data where the columns can be of different types, such floats, integers, booleans, character strings, and dates. In Fortran 95 you can define a derived type with 1D array components to store such data, but this is brittle. David Duffy has suggested that stdlib have data frames:
What should be part of stdlib?
I think a good implementation of a data frame. That is, a rectangular array where each column is one homogenous type - integer, character etc, but each column can be of any type. These exist in R, Pandas, Julia etc, and are the workhorse for statistical analysis. One will encounter arguments about whether this should all be in a “real database”, and so you just need to provide appropriate Fortran interfaces, but the continued success of R, Pandas etc is a potent counter. For speed, there do have to be indices and hashes under the bonnet, and optimized sorts, joins, Fortran array type slices, and so on.
5 Likes