How to correctly write Makefile for code with submodules?

Latest Update,
Problem fixed.
For someone who want to use flint as ODE, you can take my Makefile.win in my gitlab folder as a reference. It run with Intel OneApi in windows correctly.
The build.bat, rebuild.bat, clean.bat may also help you do makefile in windows + Intel OneAPI using make.
Modify the Makefile to work with gfortran, and in linux is trivial.

D:\flint_OneAPI>rebuild.bat
:: initializing oneAPI environment…
initializing Visual Studio command-line environment…
Visual Studio version 16.11.4 environment configured.
Visual Studio environment initialized for: ‘x64’
: advisor – latest
: compiler – latest
: dal – latest
: debugger – latest
: dev-utilities – latest
: dnnl – latest
: dpcpp-ct – latest
: dpl – latest
: inspector – latest
: intelpython – latest
: ipp – latest
: ippcp – latest
: itac – latest
: mkl – latest
: mpi – latest
: tbb – latest
: vpl – latest
: vtune – latest
:: oneAPI environment initialized ::
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c FLINTUtils.f90
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c FLINT_base.f90
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c ButcherTableaus.f90
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c StepSz.f90
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c ERK.f90
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c FLINT.f90
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c test_CR3BP.f90
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c ERKInit.f90
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c ERKIntegrate.f90
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c ERKInterp.f90
ifort.exe /nologo /MP /O3 /QxHost /assume:buffered_io /heap-arrays0 /Qipo /libs:static /threads /Qmkl:cluster /c ERKStepInt.f90
ifort.exe /exe:flint_test.exe test_CR3BP.obj FLINT.obj FLINT_base.obj FLINTUtils.obj ERK.obj ERKInit.obj ERKIntegrate.obj ERKInterp.obj ERKStepInt.obj ButcherTableaus.obj StepSz.obj /link /out:flint_test.exe /INCREMENTAL:NO /LARGEADDRESSAWARE
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.4.0 Build 20210910_000000
Copyright (C) 1985-2021 Intel Corporation. All rights reserved.

Microsoft (R) Incremental Linker Version 14.29.30136.0
Copyright (C) Microsoft Corporation. All rights reserved.

-out:flint_test.exe
-subsystem:console
/out:flint_test.exe
/INCREMENTAL:NO
/LARGEADDRESSAWARE
C:\Users\Rong\AppData\Local\Temp\ipo_2010434.obj
D:\flint_OneAPI>flint_test.exe
60.0104184754309 471.636527263228 -1019.06865347391
1454.33725487003 1.09976187882064 -1.46242207746315
-1.35142355895415 829 46 25


I have small question, if say, ERK.f90 is a module, all the ERKInit.f90, ERKIntegrate.f90, ERKInterp.f90, and ERKStepinit.f90 are submodule of ERK.f90. Should I write in my Makefile things like below?

ERK.obj: ERKInit.obj ERKIntegrate.obj ERKInterp.obj ERKStepinit.obj ERK.f90
	$(FC) $(FFLAGS) /c ERK.f90
	
ERKInit.obj: ERKInit.f90
	$(FC) $(FFLAGS) /c ERKInit.f90	
	
ERKIntegrate.obj: ERKIntegrate.f90
	$(FC) $(FFLAGS) /c ERKIntegrate.f90
	
ERKInterp.obj: ERKInterp.f90
	$(FC) $(FFLAGS) /c ERKInterp.f90	
	
ERKStepinit.obj: ERKStepinit.f90
	$(FC) $(FFLAGS) /c ERKStepinit.f90

It seems not exactly correct.

Slightly more detail.
I am using @prince_mahajan flint ODE solver which I believe is a very good one. His code runs perfectly.

Now I want to write a Makefile for flint, so that it can be used in other package.
I put the source code here,

Inside the folder,
test_CR3BP.f90 is the main program, other f90 are the flint library files.
I am writing Makefile to compile and link all the f90 files.
build.bat is made for doing make.
clean.bat is made for doing make clean.
rebuild.bat is made for doing clean first and then make.
Makefile.win is the makefile for windows.

I use intel OneAPI in windows. All the f90 file can be added in visual studio with OneAPI integrated, and compile and run perfectly.

I have no problem writing the Makefile in windows or linux for code without submodules.
In windows, just install gnu make in windows, and do make, and code will compile and run, with ifort or gfortran.

However, Flint is with submodule, I am not sure how to correctly write makefile.

The Makefile I wrote produce an error

ERKInit.f90(26): error #8765: Error in opening the compiled ancestor module file.   [ERK]
submodule (ERK) ERKInit
---------------^

I know it is somehow due to ERKInit.f90 is a submodule of ERK.f90.
It seems indicating that the submodule ERKInit.f90 cannot find its master module ERK.f90 when compiling.
But currently I do not exactly sure how to fix it. Not sure how to compile the main module ERK.f90 before compiling the submodules.

I just wonder,

  1. does anyone know how to write the relations between submodule and the main module in makefile?
  2. Or in short, could anyone if have time could show a Makefile which can compile all the f90 files in my Gitlab folder correctly? The f90 files absolutely have no problem, Just lacking a Makefile.

Thank you very much in advance!

Since the submodules depend on the module, you should have a rule to describe that relationship. The module should be compiled first and then the submodules. Anything using the module should simply depend on that, not on the submodules, as one of the design criteria was that the submodules contain the implementation and the module the interface, so that they can be compiled independently.

2 Likes

Thank you so much @Arjen , yeah I figured it out and uploaded the correct Makefile for ifort on windows and gfortran on linux in my gitlab folder.
When you have time, would you mind slightly look at another heap array issue of the same code? Thank you very much indeed!