So I am working on a software written in fortran and my aim is to include machine learning models to this code.
so far I am thinking to train my models with python’s keras or Julia’s Flux library and I have seen that neural-fortran seems to be able to import a pretrained model saved in HDF5 and use it in a fortran code, which is exactly what I need. I have three questions concerning neural-fortran :
Moreover this solution is great for keras models, but so far, I have seen that Julia’s Flux models are saved in the .bson format. Does anyone know if there would be a way to save a Flux pretrained model in HDF5, so that it could be compatible with neural-fortran?
Finally, neural-fortran doesn’t seem to take into account batchnormalisation layers. Is it going to be included in a future version of neural-fortran?
Can you open an issue in the repo describing your set-up and the issues that you encounter? Even an “I don’t know where to start is valid”, but it would be helpful to know more about the environment (compilers, OS, build system, etc.)
I’m not familiar with Flux, but I suppose bson → json (using some non-Fortran tool at least) should be straightforward, and from there writing an adapter for a model stored in json to neural-fortran model constructor should also be a data plumbing kind of work.
The Fortran-Keras Bridge project (GitHub - scientific-computing/FKB: A two-way deep learning bridge between Keras and Fortran) has/had minimal support for both a batch normalization and a dropout layer. Batch normalization was only supported for prediction using pre-trained models. No support for training in Fortran. In my most recent attempt to develop a NN model with Keras, batch normalization was the only technique that allowed me to get anything approaching a converged solution most of the time.
So I will use neural-fortran in a supercalculator with OS linux-gnu, CentOS Linux
release 7.8.2003 and mpif90 uses the compiler ifort 19.0.5.281. As for the build
system, I don’t know what it is. Could you give me some examples or explain me
how to search for the information?
Hum, does neural-fortran also read json files then?
Hum, yes, I could try to implement it. Yet I am mostly a machine learning user so I’ll
have to do this in my free time. I’ll look into this.
Hum, very interesting, thank you very much @rwmsu !
Yet I then have the same questions as for neural-fortran :
How can you include this code into your fortran project?
Is it possible to use this code with Julia’s Flux pretrained models?
For the last question I am investigating on the possibility of saving Flux pretrained models in a keras compatible format. If you have any other options, I would be more then happy to hear them.
How do you compile your supercalculator? Is it with CMake, Make, or perhaps a custom build script? If you can get your application to build with fpm, that would be the easiest way to add neural-fortran as a dependency.
Neural-fortran uses json-fortran to parse a JSON from a string attribute of a Keras HDF5 file. So JSON reading is accessible, but of course the code to map the JSON fields to an internal representation of the model would need to be written separately. Here’s an example for Keras:
so I am trying to use FKB in my project, since it is lighter and has more options that are useful for me. Yet I am struggling with the use of convert_weights.py, since it is searching for a ‘model_config’ key in the HDF5 saved model and my version of tensorflow doesn’t produce this key. Which version of tensorflow is required for FKB? I am using the 2.21.0.
Sorry. I haven’t actually used FKB, just took a look at what it could do and decided to use the Intel oneAPI AI toolkit implementation which I think is threaded (at least on Intel processors). It ran fast enough to handle all but the biggest cases I was trying to run in a reasonable length of time. Fortunately, I was able to get a small allocation on a cluster with several V100s so I ran the largest cases there.
FKB was based on TensorFlow v1, before Keras became part of TF. There’s an open PR from 2021 that updates it to TF v2, however, it has not been reviewed or merged. You can try using the branch from that PR directly. Unfortunately, I don’t think FKB is maintained anymore. It was part of Jordan Ott’s PhD thesis work and I believe Jordan has since moved on to other work in the industry. Nevertheless, it would at this point probably be more trouble than worth it to try to use FKB. Most features that FKB brought to the table via glue scripts has been implemented in neural-fortran thanks to a NASA contract that asked for those features specifically. Plus, FKB uses an early version of neural-fortran that has since been significantly improved (correctness and compute performance-wise) in the upstram repo.
I think the shortest path to batchnorm layers in Fortran is for us to implement them together. It shouldn’t be too much trouble.
It seems to work on my computer. Yet I the calculation space I am working on, hdf5 isn’t installed, so I have to figure out a way to do this. Thank you very much for your help !
I think the issue is that you only ask the linker to link against neural-fortran (-lneural), but you also need to link against all dependencies (HDF5, h5fortran, json-fortran, functional-fortran).
I suggest trying fpm which takes care of all the tedious compilation and linking details for you.
I don’t have any errors anymore, comming from jsonfortran and functional, but I still get errors from h5fortran.
When I compile with cmake, no .mod is generated in the include for h5fortran.
To sum up all of my remaining issues seem to come from the h5fortran library. What could I do to solve this issue? Could it be due to the gfortran version I am using? (gfortran 11.3.0 > gfortran 9.4.0 tested according to the github)
In the process of trying to get neural-fortran to compile and run on a Cray with Cray’s compilers along with gfortran and ifort, I had issues with h5fortran’s use of unlimited polymorphic values and select type with one compiler (don’t remember which) so I just created my own set of hdf5 utitlities (based on h5fortran) but replaced the UP logic with type specific functions and used a generic interface. If you look at the hdf5 routines actually used in neural-fortran there are only a relative few so bringing in all of h5fortran is a bit of overkill. The same with functional because only one function (reverse) is used.
I’ve only played around a little with fpm but I second Milan’s suggestion you try that instead of cmake.
Thank you so much, it works now with the fpm compilation of neural-fortran.
Yet it doesn’t with the cmake compilation. So far I’ve been
tying both, since I’m planing to use this code in the computation space of my institute and they have cmake, but not fpm. I’ll have to try using the binary of fpm in this computation space to make this compilation work.
Anyways thank you very much for the help. For those who may be interested by the solution, here is the command line I use to compile a code with neural-fortran
Everything compiles until the point where the examples get compiled. I then get the
errors :
ifort: command line warning #10006: ignoring unknown option ‘-fno-frontend-optimize’
build/ifort_1D32607880784D78/neural-fortran/libneural-fortran.a(src_nf_nf_network_submodule.f90.o) : In the function « nf_network_mp_train_ » :
nf_network_submodule.f90:(.text+0x900b) : undefined reference to « for_rtl_ICAF_BROADCAST_DESCR »
nf_network_submodule.f90:(.text+0x904e) : undefined reference to « for_rtl_ICAF_BROADCAST_DESCR »
build/ifort_1D32607880784D78/neural-fortran/libneural-fortran.a(src_nf_nf_dense_layer_submodule.f90.o) : In the function « nf_dense_layer_mp_update_ » :
nf_dense_layer_submodule.f90:(.text+0x24c8) : undefined reference to « for_rtl_ICAF_BROADCAST_OP_DESCR »
nf_dense_layer_submodule.f90:(.text+0x251c) : undefined reference to « for_rtl_ICAF_BROADCAST_OP_DESCR »
build/ifort_1D32607880784D78/neural-fortran/libneural-fortran.a(src_nf_nf_dense_layer_submodule.f90.o) : In the function « nf_dense_layer_mp_init_ » :
nf_dense_layer_submodule.f90:(.text+0x3077) : undefined reference to « for_rtl_ICAF_BROADCAST_DESCR »
build/ifort_1D32607880784D78/neural-fortran/libneural-fortran.a(src_nf_nf_conv2d_layer_submodule.f90.o) : In the function « nf_conv2d_layer_mp_update_ » :
nf_conv2d_layer_submodule.f90:(.text+0x43af) : undefined reference to « for_rtl_ICAF_BROADCAST_OP_DESCR »
nf_conv2d_layer_submodule.f90:(.text+0x440d) : undefined reference to « for_rtl_ICAF_BROADCAST_OP_DESCR »
Compilation failed for object " simple "
stopping due to failed compilation
STOP 1
What happens if you disable coarrays via compiler flag? I don’t recall the exact syntax from the top of my head, but I think the neural-fortran README has examples for compiling with ifort in serial mode.
-fno-frontend-optimize is a gfortran option. Don’t use it when compiling with ifort.
Ok, I spoke too fast. The compilation works, but the executables don’t
do anything. From what I saw on the internet, this should be
due to memory issues, which shouldn’t be the case for me.
So I didn’t see any examples in the README about compiling
neural-fortran with intel in fpm. How would I need to do that?