Module dependencies

1

Do people have tools to update make files (or another build system) when a new module is to be USEd in a program? That module may depend on other source files, so its dependencies need to be added in the proper order.

For example, I can create an executable with

gfortran kind.f90 first_last_true.f90 print_array.f90 error.f90 get_unit.f90 cholesky.f90 util.f90 correlation.f90 statistics.f90 random_normal.f90 random_mv_normal.f90 acf.f90 var_sim.f90 pythag.f90 svbksb.f90 svdcmp.f90 reg_stats.f90 var_fit.f90 xvar_fit_print.f90

and also with

gfortran kind.f90 statistics.f90 qsort.f90 date_mdy.f90 util.f90 qsort_good.f90 open_file.f90 get_unit.f90 replace_string.f90 error.f90 read_mdy_alloc.f90 data_frame_full.f90 data_frame_full_stats.f90 xvar_data_frame.f90

The file var_fit.f90 contains a module var_fit_mod that I want to use in xvar_data_frame.f90, so the question is which files in addition to var_fit.f90 do I need to add to the second compilation command above. A script tells me that the files in the first compilation list but not the second are

first_last_true.f90 print_array.f90 cholesky.f90 correlation.f90 random_normal.f90 random_mv_normal.f90 acf.f90 var_sim.f90 pythag.f90 svbksb.f90 svdcmp.f90 reg_stats.f90 var_fit.f90 xvar_fit_print.f90

I need to add a subset of this list of files to the second compilation list, but which ones and in what order? I created a main program dummy_var_fit.f90 that is just

use var_fit_mod
end

and ascertained that

gfortran kind.f90 error.f90 util.f90 print_array.f90 first_last_true.f90 acf.f90 svbksb.f90 pythag.f90 svdcmp.f90 reg_stats.f90 var_fit.f90 dummy_var_fit.f90

compiles. That narrows the list of files I need to add to

print_array.f90 first_last_true.f90 acf.f90 svbksb.f90 pythag.f90 svdcmp.f90 reg_stats.f90 var_fit.f90 dummy_var_fit.f90.

After some experimentation I find that

gfortran kind.f90 print_array.f90 first_last_true.f90 acf.f90 svbksb.f90 pythag.f90 svdcmp.f90 statistics.f90 qsort.f90 date_mdy.f90 util.f90 qsort_good.f90 open_file.f90 get_unit.f90 replace_string.f90 error.f90 read_mdy_alloc.f90 reg_stats.f90 data_frame_full.f90 data_frame_full_stats.f90 var_fit.f90 xvar_data_frame.f90

compiles. Creating dummy main programs and associated make files for each module can clarify what dependencies are, but I would like to automate the process of creating compilation comands.

4 Likes
2

Once fpm can generate Makefile and CMake build system, you can just rerun fpm and it would update your make files. There are also other tools that can generate the dependencies in a makefile form.

1 Like
3

makedepf90 is a choice. makedepf90(1) - Linux man page (die.net)

4 Likes
4

Thank you @Euler-37 for the tip and welcome!

5

I grabbed a version from GitHub - outpaddling/makedepf90: Generate make dependencies for Fortran code

1 Like
6

I’m not sure if it’s suitable for your needs, but long time ago, before switching to SCons and (then) CMake, I used to use mkmf. It might worth checking it out until fpm matures (as suggested).

1 Like
8

I once worked with the developer of fake. I wasn’t a fan, but it might suit your needs well.

9

Sounds like what fpm is for.

Here are a few more relevant threads and tutorials:

The Fortran Wiki also has a list of tools. I believe some of these tools probably haven’t kept up with language features like block statements and submodules. The Fortran Wiki list highlights one of the problems of Fortran, which is the amount of reinventing the wheel that goes on. If we could join forces and build a robust tool that given a list of source files outputs module dependencies in a chosen format, it could benefit all build systems.

The Ninja build sytem manual contains a short section about Fortran modules, and introduces a “hypothetical fscan tool [that] scans the source files” and writes an output file with the dynamic dependency information.

I guess all of the tools actually follow the same logic, which would be nice to summarize in pseudo-code:

Input: List of Fortran source files
Output: Fortran dependency graph

For File in List of Source Files:
  Process(File)...

If we could agree on a good notation for the dependency graph, than any existing or future build tool could use it. The logic fpm uses to resolve dependencies currently is contained in fpm_targets.f90, while the derived type describing source files are defined in fpm_model.f90. The actual parsing is defined in fpm_source_parsing.f90.

It would be great to discuss what are the possible caveats in building a module dependency graph (cyclic dependencies, preprocessing,…) and how to resolve each of them?

Another important question for a scalable tool is how to update the dependency graph given a previous version, and a list of modified files? The author of the tup build system, wrote an interesting document about Build System Rules and Algorithms which I believe are relevant also to the design of fpm.

3 Likes
10

Browsing through the C++ 2020 proposals I discovered:

Format for describing dependencies of source files

Unsurprisingly the authors are from Kitware (company behind CMake).

Maybe it is worth taking a look at to get some ideas for having fpm emit such a dependency file and in combination with a Universal Fortran compiler interface.

2 Likes
11

I see that Intel Fortran has a -gen-dep option to generate dependency information. For module files it shows the source file dependencies, and for object files it shows the source file and module file dependencies. Here is an example of its use:

c:\fortran\2015>ifort -nologo -gen-dep kind.f90 qsort.f90 date_mdy.f90 util.f90 qsort_good.f90 open_file.f90 get_unit.f90 replace_string.f90 error.f90 read_mdy_alloc.f90 data_frame_full.f90 xxdata_frame_full.f90
kind_mod.mod : \
  kind.f90

kind.obj : \
  kind.f90

qsort_mod.mod : \
  qsort.f90

qsort.obj : \
  qsort.f90 kind_mod.mod

date_mdy_mod.mod : \
  date_mdy.f90

date_mdy.obj : \
  date_mdy.f90

util.f90(4962): remark #8291: Recommended relationship between field width 'W' and the number of fractional digits 'D' in this edit descriptor is 'W>=D+7'.
   write (outu_,"('#>=thresh, total, thresh, abs_min_ =',2(1x,i0),g12.6)") count(xx(:,1) > abs_min_),n,abs_min_
-------------------------------------------------------------------^
util_mod.mod : \
  util.f90

util.obj : \
  util.f90 kind_mod.mod

qsort_good_mod.mod : \
  qsort_good.f90

qsort_good.obj : \
  qsort_good.f90 qsort_mod.mod util_mod.mod kind_mod.mod

open_file_mod.mod : \
  open_file.f90

open_file.obj : \
  open_file.f90

get_unit_mod.mod : \
  get_unit.f90

get_unit.obj : \
  get_unit.f90

replace_string_mod.mod : \
  replace_string.f90

replace_string.obj : \
  replace_string.f90

error_mod.mod : \
  error.f90

error.obj : \
  error.f90

read_mdy_alloc_mod.mod : \
  read_mdy_alloc.f90

read_mdy_alloc.obj : \
  read_mdy_alloc.f90 error_mod.mod util_mod.mod replace_string_mod.mod open_file_mod.mod \
  kind_mod.mod get_unit_mod.mod date_mdy_mod.mod

data_frame_full_mod.mod : \
  data_frame_full.f90

data_frame_full.obj : \
  data_frame_full.f90 get_unit_mod.mod util_mod.mod kind_mod.mod

xxdata_frame_full.obj : \
  xxdata_frame_full.f90 data_frame_full_mod.mod
2 Likes
12

The Python tool run-fortran works on some programs I tried, except for codes that use intrinsic modules with statements such as use iso_fortran_env. Sample usage is just

python3 path_to_script\run-fortran.py run

in the directory of the Fortran source files. It lists the files in a proper order for compilation.

13

I’ve found one more tool which is not listed on the Fortran Wiki pages:

fdep is a small set of scripts to teach autoconf/automake (using GNU make)
about the additional dependencies in Fortran 90 files due to modules.

It appears to be developed within the ELPA (Eigenvalue soLvers for Petaflop Applications) project.

14

Every compiler should have a mode for producing dependency information. The traditional format to describe graphs is a “dot” file. It seems fairly simple to derive a to-be-included make fragment from this.

15

Thanks for that information.

By default, dependencies are printed on standard output but you can also specify a filename. And you can also list the intrinsic modules with -gen-depshow=intr. For example:

$ ifx -gen-dep=dependencies.txt -gen-depshow=intr main.f90

Strangely, if you use several times that option, the output is appended to the text file, instead of being overwritten.

16

I am looking for something similar to initialise the linter of VS Code. When fpm is fully supported we can just run fpm build with some additional settings, but for the time being I am looking for something that can give me a list with the order the compilation is supposed to happen, the main limitations is that it needs to be cross platform and easy to install.

I have been eyeing @zedthree’s GitHub - ZedThree/fort_depend.py: A python script to generate dependencies for Fortran projects although I would have to heavily modify it for be usable by vscode. Is there anything else we could potentially use?

17

CMake has a built-in dependency tree generator. I’m not sure how easy it is to expose that information directly though

1 Like
18

A tool for obtaining file dependency information is FF08Depends.
It is available for download at:

https://www.megms.com.au/ff08depends.htm

<Quote
FF08Depends is a command line tool for obtaining file dependency information for Fortran 2008 source files.
The tool itself is written in Fortran 2003. A zip archive of the latest source release (revision 2960 from 2020-04-17), available under an Apache 2.0 license, can be found at www.megms.com.au/download/FF08Depends.zip.

-t or --tree:
Display the dependency tree rather than the dependency order. The optional argument specifies the format for the tree. The default format is make, which is a format that is suitable for many “make” programs. The other available format is dot, which specifies a format suitable as input into the dot graph tool (graphviz).
/>

The announcement post is here:

FF08Depends author is IanH:

The archive contains other utilities written in Fortran, in particular a Generic command line parsing module (CmdLine.f90)

4 Likes
19

This is a very useful tool. Thank you for sharing. Do you know if it can also detect and analyze included files via processor #include?

20

The answer to my question is given in the Intel forum link you shared.

But if you are doing anything outside the remit of the standard (like using the C preprocessor to #define or #include stuff) then the utility will probably just get confused. Note the qualifiers “should”, “perhaps”, etc are all in there for good reasons - this might also just waste your time.