I would like to announce a small tool which extract the module/submodule declaration and dependencies across multiple Fortran sources and generates recipes to be used in Makefiles. Here is a sample output:
It is currently capable of parsing code in free format and supports preprocessing. I haven’t tested it in large codebases, any feedback is appreciated. Here is the repository link for more details: GitHub - lsmenicucci/mk-fdeps
Nice work!
I’ve try to use it with my code and I’ve found some issues. One is easy to solve:
The keywords (use, module, submodule …) should be case insensitive. I’ll make a pull request to fix that.
In my codes, the object and module files (.o and .mod) are stored in a specific directory. I’ve managed to use “–with-parent”, but cannot remove the path of the tree source files (I’ve tried with “–strip-parents” without success). I’ve managed to use correctly “–strip-parents”.
Finally, my code can be compiled with the dependencies generated with your code.
@lsmenicucci nice, I see you used Fortran to implement it!
If you wanted, you could help us fix this issue in fpm: Have CMake and Make backends · Issue #69 · fortran-lang/fpm · GitHub to generate Makefiles for any fpm project, with dependencies, automatically. If your project would work with fpm, then you could just make fpm spit out makefiles and use make as you do with mk-fdeps.
It would be a nice project to expose the fpm module scanner as a Make target dependency generator, or vice-versa, allow mk-fdeps to be used from fpm. In fpm the module dependency scanning is accomplished here: fpm_source_parsing.f90
There is a long “tradition” of writing tools like makedepend or mkdep. Major Make texts dedicate entire chapters to the use of such tools:
The make dependency topic was a perennial topic on comp.lang.fortran in the 90s (when Fortran modules were introduced). It also pops up regularly here on Fortran Discourse. See some of my previous posts:
A couple more tools are listed at the Fortran Wiki Build Tools page and @Beliavsky’s Fortran Tools list. guess that SCons and Meson also have their own Fortran module dependency scanners. AFAIK, IDEs like SimplyFortran and Code::Blocks (CBFortran?) also support Makefile generation. Many large Fortran projects (e.g. CPMD) still maintain custom scripts. It’s a bit of a tragedy that this has been re-invented so many times, likely due to a mix of “Not Invented Here” syndrome and also a historical lack of centralized tooling in the Fortran world.
My concern with trying to parse the module function grammar is that it is huge and any partial implementation will inevitably run into cases it doesn’t handle and then the solution will be “just add this one little bit more” repeated over and over.
Eventually the tools fall into disrepair or fail on newer features like submodules. In the same thread, but a different post, Brad King noted:
In the long term, I’d like to see Fortran compilers adopt the approach we’ve proposed for C++ module dependencies in P1689r4.
The situation in Fortran mirrors the C/UNIX world. As Eric Raymond writes in TAOUP,
Each different makefile generator tackles these objectives in a slightly different way. Probably a dozen or more generators have been attempted, but most proved inadequate or too difficult to drive or both, and only a few are still in live use.
Will we ever reach that “long-term” solution, where the module dependency scanners are flexible, robust, and widely available? What can we do differently as a community?
A number of compilers have options to generate the dependencies as well.
ifx -gen-dep=dep.mk -syntax_only *90 */*90
complications always include handling cpp pre-processor directives and #include, recognizing intrinsics (making it a good rule to include the intrinsic descriptor in USE statements if anyone is keeping a “best practices” list; which makes that easy to add; as many vendors consider their proprietary libraries “intrinsics”, which is actually useful when figuring out what to exclude) plus what you already listed (“new” features such as submodules, …).
Now that the fpm model is much more accessible and is becoming more powerful and flexible a plugin using the model dump or API from fpm seems the best place to concentrate community efforts; although fpm makes using make less and less necessary.
The biggest think I miss about pre-module Fortran is being able to compile things in just about any order I wanted independently of other files. Maybe the only thing I miss, but that was nice.
A standard format for .mod files that could be included in a an archive (.a) file or equivalent would sure be nice too.
Your entry looks to me like a good mini-book for the fortran-lang site or an entry on the Fortran Wiki page.
It would be nice if lists like this could be “starred” to indicate popularity and rated, perhaps like movies often are on “rotten-tomatoes” and similar sites. In the not too distant past is was very hard to obtain open-source Fortran modules and libraries; the good news is that more and more are appearing in the github/gitlab/… and fortran-lang/discourse/ Fortran/Wiki/ @Beliavsky lists such that it is getting very time-consuming to sort through them. Perhaps a good problem in some respects, but one that it would be nice to have addressed.
A particularly nice thing to me about your utility appears to go unmentioned. No additional infrastructure is required as it is written in Fortran. That has an appeal to this audience that should not be overlooked. If you also released it in a single-file format that just required a simple compile that would be an additional nicety. So many tools end up needing a large amount of infrastucture and a lot of Fortran developers often work on large platforms with limited access to the WWW or (because they are centrally administered, etc. …) the user cannot easily install additional tools.
I think you should mention it is a Fortran-based solution on your site. If I am developing Fortran codes and all I need is the source and a Fortran compiler to add your tool (given how portable standard Fortran is) I have far less reasons to hesitate becoming dependent on it; and that audience is also more confident they can modify it or assume support of it in some distant future if support is dropped. Those are not inconsequential features for a lot of Fortran developers.
This is still true of submodules if I’m not mistaken. But the parent modules must be processed first.
This wouldn’t make much sense, as pointed out in the four replies starting here: Containers using F202Y's generic programming - #29 by ashe. As Themos pointed out, even if the module format was shared, there is a realistic chance of incompatibilities in the runtime support library. It is similar to how you can’t do a malloc in C followed by a deallocate in Fortran. It wouldn’t work to have a object compiled with gfortran call allocate, and let the deallocate happen in an ifx-compiled object. The compilers use different array descriptors, different conventions, etc.
Yep. Include files can be recursive, so one has to process them as they come. Also the tool will need to correctly mimic the compiler include folder search logic to resolve preprocessor symbols.
The sequence of commands to test this from the root project directory is:
make all # original makefile
cp build/mk-fdeps . # copy executable into current folder
make -f Makefile2 clean # get rid of build folder
make -f Makefile2
To ease the bootstrapping process one idea would be to create an ordered build list (see Module dependencies ). This is supported by the nagfor =depend -otype=blist command or the compile-order.txt file generated by FF08Depends.
Perhaps the build list could be checked into the repository under compile-order.txt:
Afterward you can keep using the Makefile with auto-generated dependencies. You must however make sure the compile-order.txt is valid when you check-in any changes.
Edit: I just realized the generated dep.mk fulfills the same role as compile-order.txt. So you could just check that file in and make sure it remains valid when files are added/removed.
The Cray Fortran compiler, invoked using the ftn command, is designed to find Fortran *.mod files within archive (*.a) files. This behavior is a key feature of the HPE Cray Compiling Environment (CCE).
I seem to remember that the early versions of CCE did not directly support module files with a .mod extension. Modules were compiled to a .o like other binary objects (no separate .mod file) therefore they would be added to an archive like other .o files. I think there may have been a compiler option to use .mod but its something the user had to add to their compiler statements. Backwards compatability might have been a reason Cray chose to allow .mod files in archives.
I built the exe using Intel fortran on Windows. When I ran the exe with “*.f90” as the argument, the response was: forrtl: severe (43): file name specification error, unit -129, file S:\MKMF\mk-fdeps-main\src\*.f90 <...stack trace...>
When I ran with all the .f90 names as a sequence of arguments, I was given the contents of the resulting makefile on the console. Perhaps some guidance could be given on how to use the tool on Windows.
I also found that I had to change “popen” to “_popen” and “pclose” to “_pclose” before building the utility using Intel Fortran.
It would then be up to the application to call the Win32 procedures, FindFirstFileA, FindNextFileA, FindClose. Presumably, Intel Fortran provides interface blocks in the ifwin extension module. With gfortran on Windows, it might be easier to run the executable under a Cygwin shell, where the expansion is done by the shell.
Here’s a quick program to test the shell expansion behavior:
! test_args.f90
program test_args
character(len=100) :: arg
integer :: i
do i = 1, command_argument_count()
call get_command_argument(i, arg)
print *, "Argument ", i, ": ", trim(arg)
end do
end program test_args
Hi all, is very nice to come back from a vacation and see all these interesting replies. I was definitely NOT familiar with the big family of Makefile generators we have spread in the wild. My initial intention, besides the Makefile problem, was to experiment some data structures in Fortran and prepare the grounds to write a full recursive descent parser for a future LSP.
Also, I just updated the tool so it uses a state machine based lexer (generated by re2c) which is orders of magnitude faster than the original handwritten one. Unfortunately, this adds a C file which, as far as I know, prevents the tool to be packed into a single source file, as suggested by @urbanjost.
That’s a very relevant issue, I’ve abandoned fpm particularly because of the benefits of Make (parallel builds, only build dependent files, predictable performance).