Say I have a very large project that has these characteristics:
Based on this, I would like to write the code in Python/Julia/Matlab and outsource the “slow” part to a Fortran subroutine that is called in the main code.
Any idea which language is better for calling/interfacing Fortran between Python and Julia? (I guess Matlab mex is a bit outdated and works moslty with C)
Thanks
In my list of Fortran codes there are far more cases where Python is used to wrap Fortran code than Julia or Matlab – using grep gives 143, 19, and 30 projects respectively. A non-technical consideration is that Matlab is proprietary.
Python has the largest community and is the easiest to use/interface. MATLAB’s visualization tools and language stability are unparalleled (which is why it has survived as a proprietary software despite fierce competition). However, given its presence on virtually all systems, Python would be the best choice.
Thanks for the feedback @shahmoradi and @Beliavsky.
Regarding Matlab, let me point out that they provide a good documentation of the MEX interface to C/C++ but the MEX Fortran is really outdated. On their website they use an example written in FORTRAN 77 and you can’t even use not fixed-format unless you change some internal Matlab files. For numerical computing Fortran is so much easier to learn than C… Of course I am preaching to the choir here 
https://www.mathworks.com/help/matlab/matlab_external/create-fortran-source-mex-file.html
Thanks! I was wondering if things are changing and Julia is increasing its users base fast. If so, it might be worth it to invest in Julia
For both Python and Julia you would still need to generate a shared library which would then be loaded on the Python/Julia side. I have never gone the Julia way but I’ve read a little bit after some discussions about comparisons between Julia and Fortran implementations, you can find some info here on how to load a library in Julia Calling C and Fortran Code · The Julia Language
For Python, you would have many choices, go through ctypes, cffi, f2py, and there might be a few more I’m forgetting. My preference is to write a thin iso_c_binding compatible list of wrapper procedures in a module which I would use to generate the .so/.dll, and on the Python side I’ll write a ctypes compatible wrapper using Numpy’s foreign function interface ctypes foreign function interface (numpy.ctypeslib) — NumPy v2.2 Manual.
I think that doing it that way you could actually use the same shared object from Julia, as it would not be tied to the Python interpreter version and test which approach you like the most.
You would not necessarily need MATLAB’s Fortran interface. You can use the iso_c_binding module of Fortran and other Fortran CFI features to bridge Fortran (as C) and MATLAB. Here is an example. But again, Python offers the highest flexibility, ease of use, and interoperability, with reasonable speed and time to start.
Java developers dominate MATLAB and, lately, C++ developers. That explains the heavy bias toward interoperability with C++. However, they are beginning to learn from MATLAB’s ancestor (Fortran). For example, they recently added the optional keyword argument capability to MATLAB (Incidentally, I suggested that MATLAB developers add this feature to MATLAB a few years ago).
Python is for sure a better choice for wrapping Fortran code.
As already mentioned, I use iso_c_binding module to write a thin C wrapper. Then I compile a shared library that you can plug into a Python C extension.
I personally prefer to write by hand the C extension but I can use other tools that automates the generation of C extensions.
See this Example
There was a discussion about calling Fortran from python on this forum a little ago (below, the last post is the most succint). This works well for my use-cases thus far. I find the real benefit that I only need to write Fortran & Python (it uses f2py underneath).
I understand this is a Fortran forum, but combining multiple languages always involves a lot of extra programing/debugging/documentation/etc effort. In my opinion, for the overall effort to pay off, the Fortran/C/C++ part must be significant enough. However, you said:
- Computational bottleneck in a relatively small part of the entire code
In these cases, I find it it is preferable to stick to one language. For instance, if you chose Python, you can intentionally design the code such that the time-critical functions are well isolated and can be jitted with Numba. You get very decent performance and you don’t have to worry about writing bindings, compiling them, etc.
If you still want to go for Python+Fortran, here is a recent example of mine.
HugoMVale/odrpack-python: Python bindings for the modernized version of odrpack95.
Great discussion, I will add my .02$.
There are many layers, but the common ground is that Fortran only talks to C by default (maybe other languages in the future).
So if you have a nice C interface, you can talk to any other language relatively easily.
Just note that there will be a lot of boilerplate if you use object-oriented patterns, that C doesn’t have.
So try to understand your scope: if you’re just trying to just speed up some code chunk, just go for a simple bind(C) subroutine and try to minimize the interface:
subroutine fast_code_chunk(a,b,c,z) bind(C,name='fast_code_chunk')
use iso_c_binding, only: RK => c_float, IK => c_int, etc.
real(RK), intent(inout) :: a(etc.)
end subroutine
you can just compile it as a single source, easy stuff.
If you start having large amounts of Fortran, I suggest to wrap everything (including temporary memory, pointers, etc). into a derived type and write a C API to the derived type itself. The way you do it is a matter of preference. I wrap a pointer into another bind(C) derived type, because for me it’s safer and the code is more clear:
type :: my_fortran_code
real, allocatable :: whatever(:,:,:,:)
contains
! Fortran interface
procedure :: new
procedure :: destroy
end type
type, bind(C) :: my_fortran_code_c
type(c_ptr) :: cptr = c_null_ptr
end type
! C api
function f_associate(self) result(fself)
type(my_fortran_code_c), value :: self
type(my_fortran_code), pointer :: fself
if (c_associated(self%cptr)) then
call c_f_pointer(self%cptr,fself)
else
nullify(fself)
endif
end function
! void do_something(my_fortran_code_c self);
subroutine do_something(self) bind(C)
! Anywhere you're not allocating/deallocating, you can pass `self` by value
type(my_fortran_code_c), value :: self
type(my_fortran_code), pointer :: fself
fself => f_associate(self)
if (.not.associated(fself)) stop 'variable does not exist'
! Use fself
call fself%blabla
end subroutine do_something
! void new(my_fortran_code_c* self);
subroutine new(self) bind(C)
type(my_fortran_code_c), intent(inout) :: self
type(my_fortran_code), pointer :: fself
fself => f_associate(self)
! Allocate if necessary
if (.not.associated(fself)) then
allocate(fself)
self%cptr = c_loc(fself)
endif
call fself%destroy()
end subroutine
! void destroy(my_fortran_code_c* self);
subroutine destroy(self) bind(C)
type(my_fortran_code_c), intent(inout) :: self
type(my_fortran_code), pointer :: fself
fself => f_associate(self)
! Allocate if necessary
if (associated(fself)) then
! Finalize Fortran
call fself%destroy()
! Clear pointer
deallocate(fself)
self%cptr = c_null_ptr
endif
end subroutine
And remember to always free the memory when you need the variable no more.
A fun discussion.
Yet Another Alternative: I have a modest-size Fortran code I’ve split into a Modern Fortran front-end to handle user interface and a set of computational kernels that do the heavy lifting. From the beginning I’ve made the interfaces to the kernels C-compatible (i.e. only atomic? arguments, array sizes passed explicitly, etc.) and given them C-bindings.
To use the Fortran kernels from Python (and from C++) I have a set of C++ header files that describe the C interface. The project is mature enough that I maintain these by hand but I’d use a tool if it was worth the investment.
The folks at Makepath are building me a Python front end; they are using Pybind 11 to describe the interface on the Python side. They link to pre-compiled kernel libraries.