Ways to suppress Circularity

I am seeking existing discussions about living with processes that call each other.

I have 2 processes that call each other that have not ever caused recursion in the 14 years of its executable existence. I am having severe problems updating them to gfortran 12+, because MODULEs must not call each other.

This project has 700+ files, many with more than 1 routine in them.
Making a static library of a “service routines subset” was straight forward. But upgrading to “modern” fortran that could attract programmers has introduced administrative problems. I embraced MODULEs and, indeed, doing so revealed coding problems that were (are) resolved that demonstrated its sensibility. It is greatly welcomed.

After I got to having converted 200+ of them to MODULEs and thought I would get aid on how to order compilation for “make” by adding each (or many at least) MODULE’s requirements. And doing the same for those files not yet converted to MODULEs.

It seems this last paragraph is what has brought me Circularity issues. I fought them by rewriting processes to make them go away. But, it was easy to see that Circularity just got “deeper” and difficult to locate the processes that constituted that “deeper”.

Trying to find help has wasted a lot of time. No “standards” references can be expected to go into this issue. Help with this is not going to be found via StackOverflow (easily, that is for sure) because those 2 processes are not simple, and SO wants a best answer for a single question.

Hence this “topic” to which I am clueless to add anything to (yet).

What type of “circularity” are we talking of?

1: actual circular dependencies among modules; these are not allowed AFAIK, but submodules can be used to bypass the issue in practice

module a
  use b
end module

module b
  use a
end module

2: build dependencies as required by (GNU) Make. For these you need a tool (let’s call it the module scanner) which determines the dependencies between modules and expresses them in the Make syntax:

target: ... dependencies...
    rule

For the second type of “circularity” you may want to review the following threads:

Edit: some compilers come with a built-in module scanner you can use

however, the output slightly differs which makes it harder to write a generic Makefile supporting multiple compilers.

For external scanners you could check the one at the links below (both are in Awk and use “dependency” files):

Since GNU Make 4.3 it is also possible define grouped targets with the &: syntax, which could be used instead of dependency files.

2 Likes

I suppose my elaboration was TL;DR.

I have zero problem with gFortran and linux “make” telling me there is Circularity. I believe it. I have not heard of any other type.

I do have many <object_needed>.o list_of_prerequisites

And “make” will tell me “<one_of_prerequisites> needed by <some_object_needed>.o. Prerequisite dropped”.

Then, not surprisingly, “no rule of how make (that) one_of_prerequisites”.

I read once about “Grouped targets” and decided I did not need yet more “convenience” help.

I have SUBMODULEs, but have yet to find a place in these numerous old files still yet to be converted to MODULE-ness where that SUBMODULE “B” and SUBMODULE “C” of MODULE “A” allows “B” to call a process in “C” and “C” can do the same in “B” is helpful when it comes to a practical situation where the code of “B” is so different from the code in “C”.

The “service” routine pair I have “Circularity” trouble with will probably not be the only pair. This particular service routines pair appears in most of the files. That is what finally has me seeking help. I really do not want to modify a hundred files only to discover that my change was NOT the answer. Right now I get an idea (like programming "more’ to seemingly solve the Circularity issue) only to discover it is still there… only “deeper”.

Most of my MODULEs do work with other MODULEs just fine. It was the <needed_module>.o: <prerequisite_“one_of_a_few”> that “got me” as the number of such lines I created became numerous. Adding even more of them is not going to get rid of the “make”-voiced complaint.

Taking a step back, I suppose your 700+ files were not module-based.

If so, you can avoid the circular dependency by using just ONE module, with multiple submodules.

You probably already know which procedures must be made public, but exposing some interfaces from one submodule to another might require some trial-and-error (since the one usually complaining about missing symbols is the linker).

For example:

module bigmod
    implicit none
    private

    ! public interfaces
    interface
        module subroutine a()
        end subroutine
        module function calc() result(x)
            ...
        end subroutine
        ...
    end interface
    public a
    public calc
    ...

    ! private interfaces exposed among submodules
    interface
        module subroutine c()
        end subroutine
        module function proc() result(y)
            ...
        end function
        ...
    end interface
end module

submodule (bigmod) sm_public
    implicit none
contains
    module subroutine a()
        ...
    end subroutine
    module function calc() result(x)
        ...
    end function
    ...
end submodule

submodule (bigmod) sm_private1
    implicit none
contains
    module subroutine c()
        print *, proc()
    end subroutine
    ...
end submodule

submodule (bigmod) sm_private2
    implicit none
contains
    module function proc() result(y)
        ...
        y = calc()
    end function
    ...
end submodule

submodule (bigmod) sm_private3
    ...
end submodule
...

With a simple module-submodules hierarchy, complilation is pretty straightforward —compile module first, then all submodules in any order.

1 Like

If your Makefile doesn’t capture the module dependencies across files correctly, it’s always at risk of breaking, especially in parallel builds with -jN.

The introduction of modules forces builds to follow a particular order, in contrast to the “flat” compilation model of F77 where suffix rules like .f.o (obsolete) or pattern rules %.o : %.f are sufficient. Unfortunately these rules can’t distinguish if a file actually contains modules or just a bunch of external procedures.

People started to tackle this problem as soon as Fortran 90 modules became available. There are numerous threads about this issue at comp.lang.fortran:

@ jwmwalrus

True enough. When I got ahold of the project it was all Solaris F77 + a some c routines. I broke that thing into 2 big pieces. I nursed it to compile as legacy – no small feat. THEN created a 20 line “2nd part” to substitute for the 2nd big piece. And got that combination to breathe. THEN I started on updating the1st piece concentrating on the intertwined, unapologetic spaghetti code. ALL of it is public.

Your suggestion of 1 MODULE and the rest SUBMODULEs would NEVER have occurred to me. I will give that a try using your example as a guide. First, I will read carefully about private | public. I did not suspect I might use it to my advantage as a programmer updating 100% public code.

I will report back in a few days.

1 Like

My searches never took me to comp.lang.fortran
There is a lot there it seems for updating.

I am exhausted searching. Fortran-lang.discourse is giving me the help I thirst for. Mr. Walrus’ 1st.

When using modules (as opposed to the F77-style procedures), you get explicit interfaces —i.e., the procedure signatures and their invocations get proper/better checking from the compiler.

But in Fortran, modules also provide encapsulation (public | private and protected).

Of the 700+ procedures you have, probably only a handful need to be exposed for public USE, while the rest can remain private. If that’s the case, then the single-module-multiple-submodules approach works for you.

(As of now, gfortran has a submodule-related bug when submodules try to access symbols declared in the parent module)

Otherwise, if most or all of the 700+ procedures need to be made public, then you’re better off keeping them in separate files and maybe providing modules with just the interface blocks. This is what LAPACK95 does, btw.

1 Like

When I have unexpected circular module dependencies, I try to resolve them in two seemingly contradictory ways.

One way is to combine the two (or more) modules into a single module. This allows the contained routines to call each other while eliminating the circular logic problem.

The other way, ironically, is to reorganize the two modules and create a third module that includes only the interdependent routines. Say the original modules A and B have a circular dependence because there is a routine in module A that is referenced by module B. Then create a new C with just that routine. The new modules A and B must then have USE C statements in the appropriate contained subprograms, and of course the makefile (or whatever build system you have) must be updated to reflect the new dependencies.

Of course, sometimes this is more complicated than just a single shared subprogram, and sometimes it involves also organizing shared data which presumably began its life in legacy common blocks.

I did try that, but only halfheartedly, because name changes would affect most of the project files. Making THAT just running a sed program would lead down a rabbit hole of assuring that each file’s references to such a prospective change would need to be marked in some fashion to surely should be changed, e. g. NOT in the !History: section and similar. I have been through similar and it just eats on me that I am not directly headed toward my goal for this project when I descend into the warren.

I respect and read nearly everything you post that I run across, so I will spend the time to make your suggestion a member of my ready at hand schemes when getting surprises as I update (and update the update, sigh) to an acceptable, working whole.

It has been a “few” days. I ran into question about recreating a generic with the one huge MODULE scheme.

I do (think I) understand why the huge MODULE with an interface for all these 150 source files + submodules for each of the 150 avoids “Module Circularity”. I narrowed down the program from 700+ by removing as many features as could and still have it put to work the code in all150.

All went as you outlined. I (foolishly?) could see no reason for
my MODULE top_mod to enclose each of the interfaces with INTERFACE … END INTERFACE, so there is only such enclosure for the myriad MODULE SUBROUTINE name() … END SUBROUTINE name, there by saving me 150*2 additional lines in the top_mod file.

I converted the code files to be SUBMODULEs as I did the above. It has been taking me about 7-10 minute per source file and “top_mod” file.

:**** this is supposed to be directed to Mr. Walrus
**** How do I change this post to do that?
**** I felt like I reread his post and clicked on REPLY, but as I see above Discourse says I replied to my self.
**** AND why was my edit jammed in the middle of my post?
**** Is that initial “:” some how responsible? Sigh. I will look external to discourse about how to Edit.

Declaring generic as been for me INTERFACE <generic_name>
followed by interfaces for the procedures, followed by END INTERFACE . I do not think an INTERFACE within an INTERFACE is proper.

It seems I have 3 choices and am asking if that it the way you see it:

  1. Abandon my nature of putting all of these top_mod interfaces of procedures in alphabetical order and stick the few generic “named” INTERFACEs at the top or bottom of top_mod.

  2. Byte the bullet and go back and separate every MODULE SUBROUTINE name() declaration with END INTERFACE, a blank line, and a INTERFACE, and adding to the few "one-line"ers as needed.

  3. Maintain alphabetization by doing an END INTERFACE, then a blank line, then INTERFACE , the short list of which procedures get generisized, END INTERFACE , another blank line, the finally a plain INTERFACE to resume the “normal” MODULE SUBROUTINE declarations.

Are these my choices with the MODULE top_mod scheme?

I am not the walrus. The walrus was John. I am John. :laughing:

In Fortran, the interface block is overloaded both to provide a bridge for an external procedure, and also to provide a generic name —the generic statement was later allowed outside derived types to try to overcome the confusion.

Your post is somewhat confusing in terms of what you’re trying to accomplish with interfaces, so I will… clarify with comments?:

module bigmod
    implicit none
    private

    ! the 'module' keyword implies that the code is elsewhere, but belongs to this module. This exposes procedures defined in submodules
    interface
        module subroutine a(i, j, k)
            integer, intent(in) :: i, j
            integer, intent(out) :: k
        end subroutine
        module subroutine fff()
        end subroutine
    end interface

    ! here, the code is elsewhere but does not belong to this module, so 'import' might be required
    interface
        subroutine bind() bind(C, NAME = 'bind')
        end subroutine
    end interface

    ! this is a generic interface, 'import' might be required
    interface my_generic
        subroutine sub1()
        end subroutine
    end interface

    ! I can use any procedure that belongs to the module, thus avoiding the function/subroutine block duplication
    interface another_generic
        module procedure a
    end interface
    public :: another_generic

    ! this is equivalent to the 'another_generic' above
    generic, public :: same_as_above => a
end module

submodule (bigmod) sm_partial
    interface
        ! procedures in this submodule need this, but the parent module doesn't
        module subroutine ddd()
        end subroutine
    end interface
contains
    ! here, I'm avoiding the duplicate signature for the subroutine
    module procedure a
        ! Implementation goes here
        ...
    end procedure

    ! the outside world doesn't need to know about this, so there's no need for a module interface
    subroutine some_sub()
        ...
    end subroutine
end submodule 

submodule (bigmod:sm_partial) sm_helpers
contains
    module procedure ddd
        ...
    end procedure
    module procedure fff
        ...
    end procedure
end submodule

You are allowed to go as crazy as you want/need in terms of submodule hierarchies. The thing to keep in mind is that each submodule has access to everything from its ancestors.


John M.

I apologize for just about it all. Each of 1), 2), and 3) are how to organize MODULE top_mod so some future programmer can find their way around in lots of repeating format language without getting cross eyed.

I will go the safe but laborious way 1). I have alphabetically, 150 SUBMODULEs to describe in MODULE top_mod. The minimum is
INTERFACE
MODULE SUBROUTINE source_001 (arg)
/declare arg
END SUBROUTINE source_001
END INTERFACE
/blank line\

Similar to the above will be 149 more. In the middle of that there will be some that are different like this:

INTERFACE <generic_name>
MODULE PROCEDURE :: source_075, source_111
END INTERFACE
/blank line\

then back to more of the not generic ones.

INTERFACE
MODULE SUBROUTINE source_blah (arg)
/declare arg
END SUBROUTINE source_blah
END INTERFACE

and so on until finally END MODULE top_mod

Do you really have 150 submodules? Or is it 150 public functions and subroutines in a few submodules?

A submodule is just a hierarchical abstraction. Your API will not be affected by the amount of submodules, but by whatever is made public in the parent module.

Maybe I’m oversimplifying, but…

You said you originally had 700+ files. Applying some categorization, maybe you end up with at most 10 submodules. And for those submodules, only a few procedures need to be known by the top module —either because they’re needed among submodules or because they’re part of the public API.

I apologize in advance if I seemed to be making things more confusing than they need to.

This started seeking how others avoided “Module Circularity”. There were several that others have used. I had come upon an “MC” and with the advice … I chose to recode 1 source file and that occurrence went away, but soon there after another was complained about. It named the modules, but this turned out to be at a depth of more than 1 (it was explained to me). There were no known tools to tell where in the various MODULEs this circularity was being caused. At that point I began to worry.

Since I am early in this strategy of MODULES for every individual file and I could not produce an executable, AND relatively few legacy files having been converted to MODULEs, I was in a quandary of how to spend my time not going backward, but getting an executable which is necessary to at least properly perform the User Reference Manual’s many examples. The big choice I made was to start a subset of the 700+ files (many with helper subroutines) that could still present a prompt. That reduced the # to 150, but only after having taken a meat clever in the way of wielding “commenting out” (but adding a suitable meta-code so restoration would make healing quicker) a feature or 2 that permeate most of this source subset.

These 150 “old” files do allow creation of a viable (if lobotomized) executable. I call this SUBM_scheme.exe. The goal is that as I learn practical, real world, techniques for gFortran -std=f2008, they will become familiar enough to me that I can see ahead the likeliest alternatives where this legacy code MUST be changed if there is any hope of any Fortran programmer being a “programmer of record” for it.

Your “one big module”, 1st suggested back in October to avoid “Module Circularity”, took more than a “few days” to be the next scheme to try out. And here I am. Without something like this, the unknown more "Module Circularity"s are likely to change the old code more than a line or 2. Changes bring corner cases to face after QA of the program is rerun (and expanded for said corner cases).

I suspect that even with the “one MODULE to rule them all (SUBMODULEs)” resulting in no “Module Circularity”, other upgrades to gFortran -std=f2008, will require code changes that, conceivably, legacy constructs where algorithms called each other, but have (did have) ways to not result in infinite regress, … will regress. I was initially optimistic when the build complained of “Module Circularity”, because gcc has been wonderfully improving in pointing out flaws that have helped reduce the known number of bugs in that legacy executable. The paramount goal of handing this project to a programmer(s) is for “them” to not quit after a couple weeks of not even being able to read this Fortran 77 code (which had understandably been gamed to “be more like c”).

Much has been done, but not enough yet to expect any 2024 Fortran programmer to accept it as an enjoyable way to spend any part of their life on.

1 Like