Module dependencies

Do people have tools to update make files (or another build system) when a new module is to be USEd in a program? That module may depend on other source files, so its dependencies need to be added in the proper order.

For example, I can create an executable with

gfortran kind.f90 first_last_true.f90 print_array.f90 error.f90 get_unit.f90 cholesky.f90 util.f90 correlation.f90 statistics.f90 random_normal.f90 random_mv_normal.f90 acf.f90 var_sim.f90 pythag.f90 svbksb.f90 svdcmp.f90 reg_stats.f90 var_fit.f90 xvar_fit_print.f90

and also with

gfortran kind.f90 statistics.f90 qsort.f90 date_mdy.f90 util.f90 qsort_good.f90 open_file.f90 get_unit.f90 replace_string.f90 error.f90 read_mdy_alloc.f90 data_frame_full.f90 data_frame_full_stats.f90 xvar_data_frame.f90

The file var_fit.f90 contains a module var_fit_mod that I want to use in xvar_data_frame.f90, so the question is which files in addition to var_fit.f90 do I need to add to the second compilation command above. A script tells me that the files in the first compilation list but not the second are

first_last_true.f90 print_array.f90 cholesky.f90 correlation.f90 random_normal.f90 random_mv_normal.f90 acf.f90 var_sim.f90 pythag.f90 svbksb.f90 svdcmp.f90 reg_stats.f90 var_fit.f90 xvar_fit_print.f90

I need to add a subset of this list of files to the second compilation list, but which ones and in what order? I created a main program dummy_var_fit.f90 that is just

use var_fit_mod
end

and ascertained that

gfortran kind.f90 error.f90 util.f90 print_array.f90 first_last_true.f90 acf.f90 svbksb.f90 pythag.f90 svdcmp.f90 reg_stats.f90 var_fit.f90 dummy_var_fit.f90

compiles. That narrows the list of files I need to add to

print_array.f90 first_last_true.f90 acf.f90 svbksb.f90 pythag.f90 svdcmp.f90 reg_stats.f90 var_fit.f90 dummy_var_fit.f90.

After some experimentation I find that

gfortran kind.f90 print_array.f90 first_last_true.f90 acf.f90 svbksb.f90 pythag.f90 svdcmp.f90 statistics.f90 qsort.f90 date_mdy.f90 util.f90 qsort_good.f90 open_file.f90 get_unit.f90 replace_string.f90 error.f90 read_mdy_alloc.f90 reg_stats.f90 data_frame_full.f90 data_frame_full_stats.f90 var_fit.f90 xvar_data_frame.f90

compiles. Creating dummy main programs and associated make files for each module can clarify what dependencies are, but I would like to automate the process of creating compilation comands.

2 Likes

Once fpm can generate Makefile and CMake build system, you can just rerun fpm and it would update your make files. There are also other tools that can generate the dependencies in a makefile form.

makedepf90 is a choice. makedepf90(1) - Linux man page (die.net)

3 Likes

Thank you @Euler-37 for the tip and welcome!

I grabbed a version from GitHub - outpaddling/makedepf90: Generate make dependencies for Fortran code

1 Like

I’m not sure if it’s suitable for your needs, but long time ago, before switching to SCons and (then) CMake, I used to use mkmf. It might worth checking it out until fpm matures (as suggested).

1 Like

I once worked with the developer of fake. I wasn’t a fan, but it might suit your needs well.

Sounds like what fpm is for.

Here are a few more relevant threads and tutorials:

The Fortran Wiki also has a list of tools. I believe some of these tools probably haven’t kept up with language features like block statements and submodules. The Fortran Wiki list highlights one of the problems of Fortran, which is the amount of reinventing the wheel that goes on. If we could join forces and build a robust tool that given a list of source files outputs module dependencies in a chosen format, it could benefit all build systems.

The Ninja build sytem manual contains a short section about Fortran modules, and introduces a “hypothetical fscan tool [that] scans the source files” and writes an output file with the dynamic dependency information.

I guess all of the tools actually follow the same logic, which would be nice to summarize in pseudo-code:

Input: List of Fortran source files
Output: Fortran dependency graph

For File in List of Source Files:
  Process(File)...

If we could agree on a good notation for the dependency graph, than any existing or future build tool could use it. The logic fpm uses to resolve dependencies currently is contained in fpm_targets.f90, while the derived type describing source files are defined in fpm_model.f90. The actual parsing is defined in fpm_source_parsing.f90.

It would be great to discuss what are the possible caveats in building a module dependency graph (cyclic dependencies, preprocessing,…) and how to resolve each of them?

Another important question for a scalable tool is how to update the dependency graph given a previous version, and a list of modified files? The author of the tup build system, wrote an interesting document about Build System Rules and Algorithms which I believe are relevant also to the design of fpm.

2 Likes

Browsing through the C++ 2020 proposals I discovered:

Format for describing dependencies of source files

Unsurprisingly the authors are from Kitware (company behind CMake).

Maybe it is worth taking a look at to get some ideas for having fpm emit such a dependency file and in combination with a Universal Fortran compiler interface.

1 Like

I see that Intel Fortran has a -gen-dep option to generate dependency information. For module files it shows the source file dependencies, and for object files it shows the source file and module file dependencies. Here is an example of its use:

c:\fortran\2015>ifort -nologo -gen-dep kind.f90 qsort.f90 date_mdy.f90 util.f90 qsort_good.f90 open_file.f90 get_unit.f90 replace_string.f90 error.f90 read_mdy_alloc.f90 data_frame_full.f90 xxdata_frame_full.f90
kind_mod.mod : \
  kind.f90

kind.obj : \
  kind.f90

qsort_mod.mod : \
  qsort.f90

qsort.obj : \
  qsort.f90 kind_mod.mod

date_mdy_mod.mod : \
  date_mdy.f90

date_mdy.obj : \
  date_mdy.f90

util.f90(4962): remark #8291: Recommended relationship between field width 'W' and the number of fractional digits 'D' in this edit descriptor is 'W>=D+7'.
   write (outu_,"('#>=thresh, total, thresh, abs_min_ =',2(1x,i0),g12.6)") count(xx(:,1) > abs_min_),n,abs_min_
-------------------------------------------------------------------^
util_mod.mod : \
  util.f90

util.obj : \
  util.f90 kind_mod.mod

qsort_good_mod.mod : \
  qsort_good.f90

qsort_good.obj : \
  qsort_good.f90 qsort_mod.mod util_mod.mod kind_mod.mod

open_file_mod.mod : \
  open_file.f90

open_file.obj : \
  open_file.f90

get_unit_mod.mod : \
  get_unit.f90

get_unit.obj : \
  get_unit.f90

replace_string_mod.mod : \
  replace_string.f90

replace_string.obj : \
  replace_string.f90

error_mod.mod : \
  error.f90

error.obj : \
  error.f90

read_mdy_alloc_mod.mod : \
  read_mdy_alloc.f90

read_mdy_alloc.obj : \
  read_mdy_alloc.f90 error_mod.mod util_mod.mod replace_string_mod.mod open_file_mod.mod \
  kind_mod.mod get_unit_mod.mod date_mdy_mod.mod

data_frame_full_mod.mod : \
  data_frame_full.f90

data_frame_full.obj : \
  data_frame_full.f90 get_unit_mod.mod util_mod.mod kind_mod.mod

xxdata_frame_full.obj : \
  xxdata_frame_full.f90 data_frame_full_mod.mod
1 Like

The Python tool run-fortran works on some programs I tried, except for codes that use intrinsic modules with statements such as use iso_fortran_env. Sample usage is just

python3 path_to_script\run-fortran.py run

in the directory of the Fortran source files. It lists the files in a proper order for compilation.

I’ve found one more tool which is not listed on the Fortran Wiki pages:

fdep is a small set of scripts to teach autoconf/automake (using GNU make)
about the additional dependencies in Fortran 90 files due to modules.

It appears to be developed within the ELPA (Eigenvalue soLvers for Petaflop Applications) project.

Every compiler should have a mode for producing dependency information. The traditional format to describe graphs is a “dot” file. It seems fairly simple to derive a to-be-included make fragment from this.