I am seeking existing discussions about living with processes that call each other.
I have 2 processes that call each other that have not ever caused recursion in the 14 years of its executable existence. I am having severe problems updating them to gfortran 12+, because MODULEs must not call each other.
This project has 700+ files, many with more than 1 routine in them.
Making a static library of a “service routines subset” was straight forward. But upgrading to “modern” fortran that could attract programmers has introduced administrative problems. I embraced MODULEs and, indeed, doing so revealed coding problems that were (are) resolved that demonstrated its sensibility. It is greatly welcomed.
After I got to having converted 200+ of them to MODULEs and thought I would get aid on how to order compilation for “make” by adding each (or many at least) MODULE’s requirements. And doing the same for those files not yet converted to MODULEs.
It seems this last paragraph is what has brought me Circularity issues. I fought them by rewriting processes to make them go away. But, it was easy to see that Circularity just got “deeper” and difficult to locate the processes that constituted that “deeper”.
Trying to find help has wasted a lot of time. No “standards” references can be expected to go into this issue. Help with this is not going to be found via StackOverflow (easily, that is for sure) because those 2 processes are not simple, and SO wants a best answer for a single question.
Hence this “topic” to which I am clueless to add anything to (yet).
1: actual circular dependencies among modules; these are not allowed AFAIK, but submodules can be used to bypass the issue in practice
module a
use b
end module
module b
use a
end module
2: build dependencies as required by (GNU) Make. For these you need a tool (let’s call it the module scanner) which determines the dependencies between modules and expresses them in the Make syntax:
target: ... dependencies...
rule
For the second type of “circularity” you may want to review the following threads:
I have zero problem with gFortran and linux “make” telling me there is Circularity. I believe it. I have not heard of any other type.
I do have many <object_needed>.o list_of_prerequisites
And “make” will tell me “<one_of_prerequisites> needed by <some_object_needed>.o. Prerequisite dropped”.
Then, not surprisingly, “no rule of how make (that) one_of_prerequisites”.
I read once about “Grouped targets” and decided I did not need yet more “convenience” help.
I have SUBMODULEs, but have yet to find a place in these numerous old files still yet to be converted to MODULE-ness where that SUBMODULE “B” and SUBMODULE “C” of MODULE “A” allows “B” to call a process in “C” and “C” can do the same in “B” is helpful when it comes to a practical situation where the code of “B” is so different from the code in “C”.
The “service” routine pair I have “Circularity” trouble with will probably not be the only pair. This particular service routines pair appears in most of the files. That is what finally has me seeking help. I really do not want to modify a hundred files only to discover that my change was NOT the answer. Right now I get an idea (like programming "more’ to seemingly solve the Circularity issue) only to discover it is still there… only “deeper”.
Most of my MODULEs do work with other MODULEs just fine. It was the <needed_module>.o: <prerequisite_“one_of_a_few”> that “got me” as the number of such lines I created became numerous. Adding even more of them is not going to get rid of the “make”-voiced complaint.
Taking a step back, I suppose your 700+ files were not module-based.
If so, you can avoid the circular dependency by using just ONE module, with multiple submodules.
You probably already know which procedures must be made public, but exposing some interfaces from one submodule to another might require some trial-and-error (since the one usually complaining about missing symbols is the linker).
For example:
module bigmod
implicit none
private
! public interfaces
interface
module subroutine a()
end subroutine
module function calc() result(x)
...
end subroutine
...
end interface
public a
public calc
...
! private interfaces exposed among submodules
interface
module subroutine c()
end subroutine
module function proc() result(y)
...
end function
...
end interface
end module
submodule (bigmod) sm_public
implicit none
contains
module subroutine a()
...
end subroutine
module function calc() result(x)
...
end function
...
end submodule
submodule (bigmod) sm_private1
implicit none
contains
module subroutine c()
print *, proc()
end subroutine
...
end submodule
submodule (bigmod) sm_private2
implicit none
contains
module function proc() result(y)
...
y = calc()
end function
...
end submodule
submodule (bigmod) sm_private3
...
end submodule
...
With a simple module-submodules hierarchy, complilation is pretty straightforward —compile module first, then all submodules in any order.
If your Makefile doesn’t capture the module dependencies across files correctly, it’s always at risk of breaking, especially in parallel builds with -jN.
The introduction of modules forces builds to follow a particular order, in contrast to the “flat” compilation model of F77 where suffix rules like .f.o (obsolete) or pattern rules%.o : %.f are sufficient. Unfortunately these rules can’t distinguish if a file actually contains modules or just a bunch of external procedures.
People started to tackle this problem as soon as Fortran 90 modules became available. There are numerous threads about this issue at comp.lang.fortran:
True enough. When I got ahold of the project it was all Solaris F77 + a some c routines. I broke that thing into 2 big pieces. I nursed it to compile as legacy – no small feat. THEN created a 20 line “2nd part” to substitute for the 2nd big piece. And got that combination to breathe. THEN I started on updating the1st piece concentrating on the intertwined, unapologetic spaghetti code. ALL of it is public.
Your suggestion of 1 MODULE and the rest SUBMODULEs would NEVER have occurred to me. I will give that a try using your example as a guide. First, I will read carefully about private | public. I did not suspect I might use it to my advantage as a programmer updating 100% public code.
When using modules (as opposed to the F77-style procedures), you get explicit interfaces —i.e., the procedure signatures and their invocations get proper/better checking from the compiler.
But in Fortran, modules also provide encapsulation (public | private and protected).
Of the 700+ procedures you have, probably only a handful need to be exposed for public USE, while the rest can remain private. If that’s the case, then the single-module-multiple-submodules approach works for you.
(As of now, gfortran has a submodule-related bug when submodules try to access symbols declared in the parent module)
Otherwise, if most or all of the 700+ procedures need to be made public, then you’re better off keeping them in separate files and maybe providing modules with just the interface blocks. This is what LAPACK95 does, btw.
When I have unexpected circular module dependencies, I try to resolve them in two seemingly contradictory ways.
One way is to combine the two (or more) modules into a single module. This allows the contained routines to call each other while eliminating the circular logic problem.
The other way, ironically, is to reorganize the two modules and create a third module that includes only the interdependent routines. Say the original modules A and B have a circular dependence because there is a routine in module A that is referenced by module B. Then create a new C with just that routine. The new modules A and B must then have USE C statements in the appropriate contained subprograms, and of course the makefile (or whatever build system you have) must be updated to reflect the new dependencies.
Of course, sometimes this is more complicated than just a single shared subprogram, and sometimes it involves also organizing shared data which presumably began its life in legacy common blocks.
I did try that, but only halfheartedly, because name changes would affect most of the project files. Making THAT just running a sed program would lead down a rabbit hole of assuring that each file’s references to such a prospective change would need to be marked in some fashion to surely should be changed, e. g. NOT in the !History: section and similar. I have been through similar and it just eats on me that I am not directly headed toward my goal for this project when I descend into the warren.
I respect and read nearly everything you post that I run across, so I will spend the time to make your suggestion a member of my ready at hand schemes when getting surprises as I update (and update the update, sigh) to an acceptable, working whole.
It has been a “few” days. I ran into question about recreating a generic with the one huge MODULE scheme.
I do (think I) understand why the huge MODULE with an interface for all these 150 source files + submodules for each of the 150 avoids “Module Circularity”. I narrowed down the program from 700+ by removing as many features as could and still have it put to work the code in all150.
All went as you outlined. I (foolishly?) could see no reason for
my MODULE top_mod to enclose each of the interfaces with INTERFACE … END INTERFACE, so there is only such enclosure for the myriad MODULE SUBROUTINE name() … END SUBROUTINE name, there by saving me 150*2 additional lines in the top_mod file.
I converted the code files to be SUBMODULEs as I did the above. It has been taking me about 7-10 minute per source file and “top_mod” file.
:**** this is supposed to be directed to Mr. Walrus
**** How do I change this post to do that?
**** I felt like I reread his post and clicked on REPLY, but as I see above Discourse says I replied to my self.
**** AND why was my edit jammed in the middle of my post?
**** Is that initial “:” some how responsible? Sigh. I will look external to discourse about how to Edit.
Declaring generic as been for me INTERFACE <generic_name>
followed by interfaces for the procedures, followed by END INTERFACE . I do not think an INTERFACE within an INTERFACE is proper.
It seems I have 3 choices and am asking if that it the way you see it:
Abandon my nature of putting all of these top_mod interfaces of procedures in alphabetical order and stick the few generic “named” INTERFACEs at the top or bottom of top_mod.
Byte the bullet and go back and separate every MODULE SUBROUTINE name() declaration with END INTERFACE, a blank line, and a INTERFACE, and adding to the few "one-line"ers as needed.
Maintain alphabetization by doing an END INTERFACE, then a blank line, then INTERFACE , the short list of which procedures get generisized, END INTERFACE , another blank line, the finally a plain INTERFACE to resume the “normal” MODULE SUBROUTINE declarations.
Are these my choices with the MODULE top_mod scheme?
I am not the walrus. The walrus was John. I am John.
In Fortran, the interface block is overloaded both to provide a bridge for an external procedure, and also to provide a generic name —the generic statement was later allowed outside derived types to try to overcome the confusion.
Your post is somewhat confusing in terms of what you’re trying to accomplish with interfaces, so I will… clarify with comments?:
module bigmod
implicit none
private
! the 'module' keyword implies that the code is elsewhere, but belongs to this module. This exposes procedures defined in submodules
interface
module subroutine a(i, j, k)
integer, intent(in) :: i, j
integer, intent(out) :: k
end subroutine
module subroutine fff()
end subroutine
end interface
! here, the code is elsewhere but does not belong to this module, so 'import' might be required
interface
subroutine bind() bind(C, NAME = 'bind')
end subroutine
end interface
! this is a generic interface, 'import' might be required
interface my_generic
subroutine sub1()
end subroutine
end interface
! I can use any procedure that belongs to the module, thus avoiding the function/subroutine block duplication
interface another_generic
module procedure a
end interface
public :: another_generic
! this is equivalent to the 'another_generic' above
generic, public :: same_as_above => a
end module
submodule (bigmod) sm_partial
interface
! procedures in this submodule need this, but the parent module doesn't
module subroutine ddd()
end subroutine
end interface
contains
! here, I'm avoiding the duplicate signature for the subroutine
module procedure a
! Implementation goes here
...
end procedure
! the outside world doesn't need to know about this, so there's no need for a module interface
subroutine some_sub()
...
end subroutine
end submodule
submodule (bigmod:sm_partial) sm_helpers
contains
module procedure ddd
...
end procedure
module procedure fff
...
end procedure
end submodule
You are allowed to go as crazy as you want/need in terms of submodule hierarchies. The thing to keep in mind is that each submodule has access to everything from its ancestors.
I apologize for just about it all. Each of 1), 2), and 3) are how to organize MODULE top_mod so some future programmer can find their way around in lots of repeating format language without getting cross eyed.
I will go the safe but laborious way 1). I have alphabetically, 150 SUBMODULEs to describe in MODULE top_mod. The minimum is
INTERFACE
MODULE SUBROUTINE source_001 (arg)
/declare arg
END SUBROUTINE source_001
END INTERFACE
/blank line\
Similar to the above will be 149 more. In the middle of that there will be some that are different like this:
Do you really have 150 submodules? Or is it 150 public functions and subroutines in a few submodules?
A submodule is just a hierarchical abstraction. Your API will not be affected by the amount of submodules, but by whatever is made public in the parent module.
Maybe I’m oversimplifying, but…
You said you originally had 700+ files. Applying some categorization, maybe you end up with at most 10 submodules. And for those submodules, only a few procedures need to be known by the top module —either because they’re needed among submodules or because they’re part of the public API.
I apologize in advance if I seemed to be making things more confusing than they need to.
This started seeking how others avoided “Module Circularity”. There were several that others have used. I had come upon an “MC” and with the advice … I chose to recode 1 source file and that occurrence went away, but soon there after another was complained about. It named the modules, but this turned out to be at a depth of more than 1 (it was explained to me). There were no known tools to tell where in the various MODULEs this circularity was being caused. At that point I began to worry.
Since I am early in this strategy of MODULES for every individual file and I could not produce an executable, AND relatively few legacy files having been converted to MODULEs, I was in a quandary of how to spend my time not going backward, but getting an executable which is necessary to at least properly perform the User Reference Manual’s many examples. The big choice I made was to start a subset of the 700+ files (many with helper subroutines) that could still present a prompt. That reduced the # to 150, but only after having taken a meat clever in the way of wielding “commenting out” (but adding a suitable meta-code so restoration would make healing quicker) a feature or 2 that permeate most of this source subset.
These 150 “old” files do allow creation of a viable (if lobotomized) executable. I call this SUBM_scheme.exe. The goal is that as I learn practical, real world, techniques for gFortran -std=f2008, they will become familiar enough to me that I can see ahead the likeliest alternatives where this legacy code MUST be changed if there is any hope of any Fortran programmer being a “programmer of record” for it.
Your “one big module”, 1st suggested back in October to avoid “Module Circularity”, took more than a “few days” to be the next scheme to try out. And here I am. Without something like this, the unknown more "Module Circularity"s are likely to change the old code more than a line or 2. Changes bring corner cases to face after QA of the program is rerun (and expanded for said corner cases).
I suspect that even with the “one MODULE to rule them all (SUBMODULEs)” resulting in no “Module Circularity”, other upgrades to gFortran -std=f2008, will require code changes that, conceivably, legacy constructs where algorithms called each other, but have (did have) ways to not result in infinite regress, … will regress. I was initially optimistic when the build complained of “Module Circularity”, because gcc has been wonderfully improving in pointing out flaws that have helped reduce the known number of bugs in that legacy executable. The paramount goal of handing this project to a programmer(s) is for “them” to not quit after a couple weeks of not even being able to read this Fortran 77 code (which had understandably been gamed to “be more like c”).
Much has been done, but not enough yet to expect any 2024 Fortran programmer to accept it as an enjoyable way to spend any part of their life on.