Let’s get them to be used by other projects. In particular SciPy, that is the most high profile “user”. Unfortunately they already replaced fftpack with a C alternative, we were too late. However they still use minpack. Let’s get our modernized version into SciPy:
and as part of this, start contributing to SciPy in modern Fortran. If we can pull this off, and convince the SciPy community that this is a worthwhile goal and worth the extra hassle and complexity that this will inevitably bring, then we have turned the tide for Fortran.
What this will bring are possible issues on Windows, Conda possibly using too old of a Fortran compiler, pyodide using an old compiler (that doesn’t support modern Fortran) to convert to WebAssembly, and so on. So the hard work that we need to do is help SciPy fix all these issues. I don’t know what the technical problems will be, but I know there will be some. We need to overcome them. The hardest problem might be getting the SciPy community on board, and again, we have to do our best, I think it can be done.
Is anybody here interested in making this happen? If we can have a few people willing to help out, I think we can pull this off.
My question is: if we provide a C-style interface with iso_c_binding, do we need f2py?
Disclaimer: I literally know nothing about how SciPy is built, but f2py always seemed to me something holding back their use of modern fortran…since I think it was mostly designed for f77? How do they interface with C code now? If we provide a C compatible interface, it should be no different right?
Since @rgoswami is one of the maintainers of f2py now, he is the perfect person to answer.
From the Fortran side, we should provide C interface using iso_c_binding I think. It is needed for other languages too. How should f2py take it from there?
Well, we also have a new pywrap backend in LFortran that we are working on, that might help: it parses the C iso_c_binding modules and produces a C header file and Python bindings. We can adapt it / merge it with f2py, whatever makes the most sense.
Yup, f2py is still needed. The issue is that even with a compatible C-style interface it still needs to be wrapped into Python. That is the right Python-C API needs to be generated and for this SciPy exclusively uses f2py (even C codes are provided with f2py signature files for wrapping purposes). So in particular, mapping modules to Python modules, compatibility with the NumPy-C API and other object representations are handled by f2py.
However, f2py does support iso_c_binding now (to some extent, actively improving) and it does make it easier… So by all means a C-style interface is a good thing for the library to provide.
It would be good to iterate on this, but AFAIK the LFortran-Python support is via Cython which may not be feasible for SciPy (will need to check that though).
I had a look into how SciPy interfaces with minpack. The interface is mainly low-level plumbing via the Python and numpy header, i.e. defining the Python module directly in C, neither f2py nor Cython is involved at this point. This means for a minimum effort solution we only have to define a C API to make minpack usable by SciPy.
I will try to collect all the references from the SciPy source and post them in the tracking issue.
where the versions of MINPACK are discussed, including modernization efforts, and where the author finds speed improvements by using LAPACK:
lapackified version
The code in the lapackified folder differs from the base/ code as follows:
It uses LAPACK to compute QR factorizations and related operations. This reduces the amount of code (completely removes the qrfac function) and makes it much faster on large instances.
It uses some BLAS functions when applicable, notably to reduce code size.
The mldif and lmder functions are very similar; they have been merged.
The hybrid and hybrj functions are very similar; they have been merged.
Some functions have been converted to zero-indexed arrays.
All-in-all, the lapackified code is about 25% smaller and much faster than the base code.
