Does SPAG really convert COMMON blocks into modules?

Help
1

This question belongs to SPAG, a plusFORT utility.

Now I’m really confused. I’ve prepared the COMMON blocks free of EQUIVALENCE coverage, that was a problem earlier. No more issue with this, and I get the modules generated in files called C_XYZ.f90. But… they all are empty! Only module header and footer. Is it expected? I mean, the .fig file does not really emphasize that COMMON contents are transferred to the modules. So @apple3feet , does SPAG automatically transfer COMMON block contents to the modules or not?

2

Hi

I’m in the process of preparing version 8.01 for release - within 2/3 weeks. I’ll do an announcement when it’s ready. The new version will be free for open source and non-commercial users using the licensing data (plusFORT.fig) below.

Open source Personal, Educational or Academic User
+ NON-COMMERCIAL USE - Not for use on proprietary or closed source code
ZFEANZAHNOHGELCHEO

A pre-release version is available at www.fortran.uk/download-pre-release-version. There are still some loose ends, so let me know if you have any problems.

This is a big update, and there are bug-fixes in the area you’re looking at, so could I ask you to take a look with the new version?

A summary of the changes is below:

-------------
Version 8.01F                                          November 2023
-------------

* Major new restructuring functionality (using dispatch loops and/or 
  internal subprograms.) allows SPAG to remove all GOTOs in even the
  densest spaghetti code,   New configuration options for this 
  functionality, and corresponding updates to PFFE.
* Improved restructuring using EXIT and CYCLE with nested named 
  DO loops.  SPAG adds an invented name to DO loops if required.  
* A major internal upgrade has removed a long-standing limitation to
  SPAG's symbolic analysis, allowing it to "look ahead", and 
  reliably identify the characteristics of subprograms with and 
  without explicit interfaces.  This change has many knock-on 
  effects in improved reporting and error detection.
* SPAG's prescan, which allows it re-order input files so that 
  MODULES are processed before code that uses them without the use
  of makefiles or external props, has been extended and improved.
* SPAG now uses the standard ISO_FORTRAN_ENV module to allow it to 
  translate legacy non-standard types, such as INTEGER*1 and REAL*8, 
  to standard Fortran.  This supersedes and improves upon the
  previous treatment using the proprietary F77KINDS module.
* By default, SPAG now inserts a PROGRAM statement at the beginning 
  of code with no subprogram statement, and adds an invented  name 
  to un-named BLOCK DATA subprograms.
* SPAG switches to "module-maker" mode when an INCLUDE file (no
  subprogram or END statement) is encountered.
* Bug in analysing symbol usage within some TYPE statements fixed.  
  This was a regression from version 7.60, and could have caused
  SPAG to insert incorrect type declarations.
* Bug in analysing symbol usage within WHERE and NULLIFY statements
  fixed.
* Bug fixed in declaration rewriting when adding INTENT to 
  declarations of POINTER variables.
* Bug when applying a global SAVE attribute - SPAG now excludes
  automatic arrays.
* Bug fixed in order of declarations when rewriting declarations 
  from scratch, in cases where the dimension of an array valued 
  FUNCTION or any argument depends on other arguments.
* Declaration rewriting now disabled for subprograms with ENTRY
  statement.
* New option to enter user specified prefix for names and labels
  invented by SPAG.
* Numerous other minor bugs and infelicities fixed.
* Manual updated to revision P.
7 Likes
3

Thank you for this update!

I’ve found that the rewrite of COMMON into modules does not happen when setting 4 is less or equal to 44: 4=44. But with 4=54 I get a segmentation fault. I’ve realized that it is due to the subroutine having an alternate return. If I remove the ‘*’ asterisk from the arguments, SPAG works.

4

This is amazing news!

5

segfaults are never good! Can you send an example?

6

@apple3feet , the file below reproduces the segfault. Please change the extension back from .f90 to .f, as this forum does now allow Fortran 77 uploads:

bishel.f90 (1.2 KB)

7

Very pleased to hear about the forthcoming new release and the new features. I tried the pre-release 8.01 version on some F77 files, and no GO TO statements remained after conversion!

However, there are some kinks to be ironed out. The attached F77 source file (please change the file suffix from .txt to .f), after converting using SPAG 7.61 and run using Gfortran, produced normal output. With SPAG 8.01, on the other hand, two problems occurred:

  • The “external fcn2” statement was dropped in the conversion process. No big problem, I manually added it back to the F90 output file.

  • The resulting F90 program apparently goes into an endless loop and no output is produced. As far as I can see, SPAG has created a large DO loop starting with

    SPAG_DispatchLoop_1: DO

    There is no statement (other than RETURN) in the body of the loop that causes the loop to be exited!

SPAG options used: fig=tof90.fig 121=100

xdverk.txt (41.4 KB)

8

I am very interested. I want to try it on nastran-95.

9

I want to try it on nastran-95.

Will not work without EQUIVALENCE elimination prior to SPAG processing.

10

Thanks - the fix is in the upcoming release.

1 Like
11

Thanks for this. Your diagnosis is spot on, and the fix (for both issues) is in the upcoming release.

1 Like
12

@apple3feet , could you please increase the maximum size of symbol table to avoid Symbol Table Overflow, and the maximum number of input sources? We have a lot more memory these days, ideally could you please make both parameters limitless?

13

Coming back to my original question: SPAG is supposed to convert the COMMON blocks into modules, but it does not always happen correctly due to a regression bug:

Although the modules for COMMON blocks are successfully created and used in the function, the old COMMON blocks are not always removed. The variable declarations for the COMMON blocks are also not removed.

For example:

SUBROUTINE xipfl
   USE C_PASSER
   USE C_XGPI2
   USE C_XGPI4
   USE C_XGPIC
   USE C_ZZZZZZ
   IMPLICIT NONE
   INTEGER Bcdcnt , Dmap(1) , Dmpcnt , Dmppnt , Iallon , Ichar , Icold , Icomon , Icrdtp , Idmpnt , Iequl , Insert , Irturn ,       &
         & Isavdw , Iseqn , Isgnon , Islsh , Istopf , Ldmap , Length , Lmpl , Maskhi , Masklo , Masks(1) , Modidx , Modnam , Mpl(1) &
         & , Mplpnt , Nbegin , Nblank , Nbpc , Nchkpt , Ncond , Ncpw , Ndiag , Ndmap , Nend , Nequiv , Nestm1 , Nestm2 , Newcrd ,   &
         & Nexit , Njump , Nosgn , Noutpt , Npurge , Nrept , Nsave , Nsol , Ntime , Nwpc , Nxequi , oscar(1) , ospnt
   REAL Core(1)
   COMMON /passer/ Istopf , Modnam , Icomon
   COMMON /xgpi2 / Lmpl , Mplpnt , Mpl
   COMMON /xgpi4 / Irturn , Insert , Iseqn , Dmpcnt , Idmpnt , Dmppnt , Bcdcnt , Length , Icrdtp , Ichar , Newcrd , Modidx , Ldmap ,&
                 & Isavdw , Dmap
   COMMON /xgpic / Icold , Islsh , Iequl , Nblank , Nxequi , Ndiag , Nsol , Ndmap , Nestm1 , Nestm2 , Nexit , Nbegin , Nend ,       &
                 & Njump , Ncond , Nrept , Ntime , Nsave , Noutpt , Nchkpt , Npurge , Nequiv , Ncpw , Nbpc , Nwpc , Maskhi ,        &
                 & Masklo , Isgnon , Nosgn , Iallon , Masks
   COMMON /zzzzzz/ Core
   INTEGER andf , orf
   INTEGER i , iofl , ityp , j , k , k1 , k2 , k3 , l
   EXTERNAL andf , orf
   INTEGER :: spag_nextblock_1
   spag_nextblock_1 = 1
...

Note: this bug is a regression against SPAG 7.61. Version 7.61 is able to properly remove all COMMON blocks always, however it crashes sometimes due to other reasons.

The old COMMON blocks are supposed to be removed, but it happens only for some files, and does not happen for the others. I was not able to track down the reason of this. There is no warning or code content that obviously triggers COMMON blocks removal to cease. One clear observation is: the COMMON blocks are either all removed, or all persist.

This is not a critical issue, I should still be able to run SPAG-7.61 for the problematic sources, and in those that crash 7.61 cut the remaining COMMON blocks with some extra scripting. The harder part is to parse and remove the declarations for the COMMON blocks as well.

But this issue produces an invalid source code that could not be compiled anymore due to conflicting declarations. Most notably, it’s a regression!!

@apple3feet I’m attaching the sample sources for investigation: in 8.01 xipfl.f90 has a bug, pktri1.f90 does not have a bug. In 7.61 both do not have a bug.

xipfl.f90 (4.3 KB)
pktri1.f90 (6.4 KB)

14

The problem with xipfl arises because of the ENTRY statement. The plan was for version 8 to disable declaration rewriting (which includes conversion of COMMON to MODULE) in routines with ENTRY statements - mostly because it does not work for ENTRY statements with arguments.

There is actually a message (<>W Declaration re-writing disabled) but I agree it’s not particularly explicit, and the output code is invalid. I’ll see what can be done.

1 Like
15

I’ll increase the symbol table size. I’m not aware of any limit within SPAG on the number of input files - can you give more detail?

16

Do you mean ENTRY somehow impacts the possibility of COMMON → MODULE rewrite? Could you please explain how?

17

If I’m entering 9,151 source files into a single invocation of SPAG, it crashes. If I squeeze multiple sources into one to overcome it, I get symbol table overflow.

I’m not yet sure though whether my attempts to feed SPAG with all source files at once make any sense. Without having the code, I can only guess the design decisions that you could make.

One reason to deploy all sources at once is to reveal all different representations of the same COMMON block to fortrtl, and generate the most unified one. But it would only be meaningful for a very smart ultimate solution for COMMON blocks conversion. After you said the plan is to disable it, I’m not sure to which extent it was meant to work. And I strongly disagree that COMMONs elimination is good to be disabled.

I’m puzzled by the fact that there is no way to disable the declarations sorting/reordering in the alphabetical order. Given that this is always done by default, the original COMMON block data flow is not preserved in the module and cannot be matched. I hope I’m missing something…

18

It’s not a language restriction - it’s the way SPAG is structured.

  • “COMMON → MODULE rewrite” requires “declaration rewriting” (for example to remove COMMON block variables from other declarations)
  • “Declaration rewriting” does not work correctly in the presence of “ENTRYs with arguments”.

The latter is a limitation which may be removed in time. In the mean-time, I prefer to intercept the problem before it happens, rather than have SPAG produce invalid code. I know that in your case the ENTRY statement has no arguments, and it’s possible we may allow that case in some future release.

19

@apple3feet ,

Thanks for your feedback. I’ve ended up having a fault-tolerant combination of SPAG 7.61 and 8.01. 7.61 is able to process some COMMON blocks that 8.01 can’t or gets crashed. Whenever 7.61 also crashes, I revert to 8.01 and have to fix the remaining issues manually. This approach maximizes the yield of good output.

I think this discussion has become very biased by many different aspects. Among these, I think entry points is not the reason of the crash, or there are more reasons. If you wish to get this sorted out clearly, please open a full-featured bug tracker and point me to it.

As an author of SPAG, you have been doing a great work. I do see how drastically SPAG improves the readability of the Fortran code thanks to fixing the labels, GOTOs and loop styles. It’s especially honorable that for doing all these you deploy a classic tech, that is you have developed parser, lexer and code emitter, in fact making 50% of a Fortran compiler by your own. I suppose it’s not easy to maintain and fix bugs in this code. Lots, lots of effort!

I don’t know what is your plan about the commercial license, but I have doubts that enterprise would buy anything but a fully robust code modernization tool. Enterprises such as simulations, oil&gas, CAD, anyone who still deploys old Fortran code would fail to complement the output of SPAG with skills needed to complete the effort to 100%. They are simply neither qualified nor managed well enough to do the extra work around SPAG output that I’m pursuing for here. The product, which leaves the codebase in a state that requires unknown amount of work of a qualified engineer is NACK to buy - alas this is how decision making works. I believe there is only one way to pass: complete the automatic transition of COMMONS. I know it’s hard, but I don’t see another way. I sincerely wish you best of luck!!

20

Well thank you for the kind words and wishes, but I’m bound to say I think your final paragraph is a bit unfair.

You say: “I believe there is only one way to pass: complete the automatic transition of COMMONS. I know it’s hard,”

Personally, I think that the general case (which has arrays and variables of different types and sizes effectively equivalenced because the COMMON statement is different in every subprogram) is so hard, and with uncertain benefit, that I doubt it’s worth the effort. There are other ways to improve modularization.

So does that mean SPAG is useless for enterprises? SPAG’s default setting is not to convert isolated COMMON to MODULE, and this functionality is very low on the list of things we get asked about. As you say yourself, SPAG can “drastically improve readability of Fortran code”, and it can chomp through millions of lines of code without a hiccup and without this functionality.

The manual warns “… it is strongly recommended that a better way is to put COMMON blocks into INCLUDE files before translation is attempted. This is because the conditions required for correct translation of an isolated COMMON are stricter than for INCLUDE files.” That seems fair comment - I suspect you will end up doing less work overall if you follow the recommendation.

My inclination now is to remove the “COMMON → MODULE” functionality altogether (but of course keep “INCLUDE file → MODULE”), precisely because of what you say about engineers being “neither qualified nor managed well enough to do the extra work”.