Long time lurker’s first post. I’ve started an autodiff library called Fazang. The project is heavily influenced by the Stan/Math project that I’ve also been contributing to.
Currently Fazang supports some intrinsic Fortran functions such as sin, log, pow for its dedicated derived type var. Array functions such as sum, dot_product, matmul are also supported. The goal is to build an AD infrastructure to support algorithm studies in fields such as Bayesian inference and inverse problems. Admittedly it’s still at a very early stage.
Do you want to add fpm build system to your Fazang package? It would make it really easy to try out. If you do, you can ping me and I’ll be happy to try your library out!
Hi @yizhang, thank you for this library. I’m interested in it because of neural-fortran. Though initially a toy/learning project for me, it’s been successfully applied in several works, so I have a renewed interest to try to get funding to work more on it. It seems to me that Fazang could be used to make something akin to TensorFlow’s gradient tape, to avoid hand-coding backpropagation for different kinds of networks.
For adapting to fpm, at a first glance, I suggest:
One module per source file (it’s possible that fpm supports multiple modules now, but I don’t think so)
While not necessary, I suggest prefixing all module names by fazang_, to emulate a namespace and minimize risk of name clash. For example, Fazang has a matmul_mod module, and it’s quite possible that an application or library that wants to use Fazang may already have a matmul_mod (e.g. if they hand-coded their matmul).
If you follow fpm’s convention for file layout (module sources in src/, programs in app/, executable tests in test/), then things should work out of the box with a minimal fpm.toml file, for example like this one from neural-fortran.
Thanks for the ping, looks like an interesting project. I already like the build system you are using, meson served me well for my Fortran projects so far.
I had a quick glance over the build files and spotted a few things which can be improved to make the library more accessible for other meson projects, as well as CMake, autotools and co. Instead of listing those here, I went ahead and sent you a patch: https://github.com/yizhang-yiz/fazang/pull/3.
Regarding the -cpp flag I think you only need it because you are using preprocessor in .f90 files, if you rename those or all to .F90 the compiler will automatically preprocess them and you can save yourself the extra logic for compiler detection in your meson build file.
Sorry, I was not clear. I think the use of -cpp/-fpp based on the file name does not work on Windows as it doesn’t distinguish uppercase from lowercase letters in file names. At least according to my tests, but I could be wrong.
That is an interesting question. So far I didn’t have any problems with case-insensitivity on Windows. Here is my take on it:
I would claim while the file system is case-insensitive, it does still preserve the actual casing for the individual files, as does git. While you would be able to run both gfortran abc.F90 and gfortran abc.f90, the command line driver will still see the actual casing of the file name and can make decisions based on it. Further, every trick you can think about exploiting this behavior will only work on Windows and is therefore not cross-plattform compatible.
As a slight aside, I’ve just starting using meson and your PR @awvwgk was really useful for me to see some Fortran build best practices. Especially the script to install .mod files. I’m wondering if it would be worth expanding some of the docs at Page Redirection to include this kind of info, e.g. installing .mod files, dependencies, pkg-config? It’s quite easy to get bogged down in the meson docs (and none of it is Fortran specific).
In principle I think you are correct, but then there is the question if the compilers use that. I tried ifx and ifort and had to give the explicit /fpp flag for it to work. For gfortran on the other hand it worked without any flag, probably consistent with your experience.
I was planning to write something about best-practice meson for the webpage, but got distracted with other things. I found the docs for meson to actually be quite complete, still it takes some effort to read through all the pages. Secondary literature, in small tutorials and recipes tailored for Fortran, is something we definitely should aim for.
I’ll be putting up a rather complete guide for using meson in mixed Fortran/C/Python projects soon for an upcoming seminar talk (will be announced here once it is ready).
This is the first time that I’ve read library codes of automatic differentiation, so it is very interesting… By the way, I am wondering if the origin of the name “Fazang” shown at the bottom of the Github page
The library is named after ancient Chinese philosopher Fazang (法藏), who views the cosmos “as an infinite number of interdependent and interpenetrating parts” (一法为因，万法为果；万法为因，一法为果).
is related to something like s(i, j)%set_chain(chain_matmul_vd) (e.g. matmul.F90, Line80)? (so inter-var relations)
I am also interested in whether implementing AD codes in Fortran (involving pointers, derived types, overloaded assignment, element-wise operations, etc) was tricky or rather straightforward, e.g., as compared to other languages (Stan/Math project).
For the name I’m merely riding the trend of naming software after somebody (Stan, Edward, Turing, etc), and the referenced philosophy seems relevant to the abstraction of reverse AD (see also doc ch3.3) including the forward pass and reverse pass that the process goes through. You’re right that everything is tied together by the chain rule.
Many design decisions in Fazang were made based on how Fortran is different from C++. What I miss most are templates and arguably safer type cast & pointers. I can’t use pointer (to the underneath tape structure) because move_alloc messes with the address space, and Fortran doesn’t offer “smart pointers” that would safe guard against accidentally pointing to garbage. On the other hand I like Fortran’s derived types. It is less powerful but also with fewer foot guns. elemental attribute is a also huge win.
In general Fortran is more verbose but nothing tricky. Being verbose tends to keep devs away. But I work with biotech scientists. C++ is something most of them would shun while Fortran is friendly enough for formula translation.
By the way, I am wondering if the origin of the name “Fazang” shown at the bottom of the Github page