Passing in a longer array than a function expects

1

In Fortran one is allowed to pass a longer array than a function expects:

program array_test
implicit none
integer, allocatable :: x(:)
allocate(x(10))
x = 1
call f(5, x)

contains

    subroutine f(n, x)
    integer, intent(in) :: n
    integer, intent(in) :: x(n)
    print *, x
    end subroutine

end program

This will pass with bounds checking on (e.g., gfortran -W -Wall -fcheck=all) and print:

           1           1           1           1           1

If the array is shorter, say, you do allocate(x(4)), then it fails bounds checking.

Does anyone know the use case when you want to pass longer arrays to functions like this? Was this needed in F77 somehow? Is it still needed in modern Fortran?

The reason I am asking is that we are implementing bounds checking in LFortran and we first implemented a requirement that the array has to be exactly the same size, which triggered failures in some 3rd-party codes, but it seems that every time it failed it was what I would consider an actual bug in the code, typically an off-by-one error. Technically it’s valid Fortran (I think), but it didn’t seem intended.

So I was hoping to understand what use cases people have for this feature. For my codes I can’t think of one right now, I would like the compiler to tell me if I am passing in a longer array by a mistake.

3 Likes
2

F77 and before did not have any kind of dynamic memory allocation, so it was typical to use fixed dimension arrays that were large enough to handle any practical size calculation. F90 added dynamic memory allocation with allocate() and with automatic arrays, so many of these legacy applications could be replaced with this newer functionality.

However, that memory allocation comes at a cost, so even now it is sometimes better to overallocate workspace arrays and then reuse that workspace for multiple passes of the algorithm using different subsets of those arrays.

Here is a paper I published in 2023 that demonstrates this. DOI:

It is not open source, but here is the abstract.

“A new implementation of a recursive pairwise merge algorithm to construct a GCF from a list of CSF expansion coefficients is presented. The essential new feature is the preallocation of some work arrays used within the intermediate steps of the merge procedure. This results in roughly an order of magnitude improvement in overall efficiency and also approximately eliminates a factor of n, the molecular orbital dimension, from the original implementation. Initial application of this merge procedure to a series of Hm molecules shows that the GCF wave functions can be represented well both with delocalized canonical Hartree–Fock orbitals and with localized molecular orbitals. For a given wave function complexity, as measured by the average facet count, f̅, the delocalized Hartree–Fock orbitals show smaller errors for small f̅ values, while the localized orbitals show smaller errors for larger f̅ values.”

Another class of applications that use this functionality is linear algebra. Here you often take a large matrix, divide it into subblocks, and operate on those subblocks separately (sometimes in parallel). The LAPACK library is an example of using this technique, although it does not use modern fortran features so the code itself is not a good example in that respect. But if LAPACK were rewritten from scratch using modern fortran, the technique of divide-and-conquer on subblocks of vectors and matrices would still play an important role.

Since this is a standard fortran feature, I would recommend that any warning messages for its use be optional and not enabled by default. I cannot tell you how frustrating it is to a programmer for a compiler to print hundreds of lines of false-positive warnings for using intentionally a standard feature.

5 Likes
3

Yes, work arrays are needed and allocation should avoided.

However, why cannot you pass a section (slice) of a workarray? In my example above it would look like this:

call f(5, x(:5))

Then you satisfy all requirements and yet all passed arrays have exactly the right length.

Yes, in the --std=f23 mode we don’t print any warnings for any standard feature.

1 Like
4

In F77, one was allowed to change the parameters of an array (including rank) “between” the actual and dummy arguments, provided that the dummy array is not bigger than the actual. Also, you could pass an array element instead of the whole array as the actual argument to make the dummy array start at that particular element.
In the absence of sum() etc. it made some of the whole-array operations much easier, like in

program summation
  integer t1(20), t2(10,20), t3(10,20,10)
  integer sum
  ! [ initialize arrays ]
  print *, sum(t1,20), sum(t2,10*20), sum(t3,10*20*10)
  ! to compute partial sums one could write e.g.
  print *, sum(t1, 10)        ! first half of t1
  print *, sum(t1(11), 10)    ! second half of t1
  print *, sum(t2, 10*2)      ! first two columns of t2
  print *, sum(t2(1,3), 10*2) ! third and fourth columns of t2
end
integer function sum(t, n)
integer n, t(n), i
sum = 0
do i=1,n
  sum = sum+t(i)
enddo
end function

In modern Fortran, with the assumed-shape arrays, pointers etc. it is probably much less needed/used. But surely requiring the exact same size would break quite a few codes. Also, the Standard still explicitly allows that, stating in 15.5.2.12 Sequence association:

An actual argument that represents an element sequence and corresponds to a dummy argument that is an array is sequence associated with the dummy argument. The rank and shape of the actual argument need not agree with the rank and shape of the dummy argument, but the number of elements in the dummy argument shall not exceed the number of elements in the element sequence of the actual argument.

3 Likes
5

You can do that, right? There are more than just one way to do the same thing. However, the legacy code already exists, and it was likely written at the time before array notation. If you want to change calling conventions in legacy codes, then make the interface explicit and the simpler call f(x(1:5)) is sufficient because the subprogram can extract the length if necessary.

Other cases are more complicated, particularly where the rank changes between the actual and dummy arguments. In some cases, you can use pointers to change the rank, but again, that requires modifying the legacy code.

1 Like
6

Thanks a lot @msz59 and @RonShepard for the examples.

Yes, legacy codes need to stay working, that’s given.

I was wondering for new codes in modern Fortran, when you would like to use this feature. It seems that the use cases are:

  • Want to pass in the whole array f(x) instead of an exact section f(x(1:5)).
  • I want to specify array slicing using sum(t1, 10), sum(t1(11), 10), sum(t2(1,3), 10*2), etc. instead of using sections sum(t1(1:10)), sum(t1(11:20)), sum(t2(1:10,3:4)).
  • Legacy codes that I do not want to modify

It seems that I would always prefer the sections, as to me at least it seems much more readable. I can definitely see how it was convenient in legacy codes however.

Note: changing of the rank (while keeping the total size the same) is a separate feature — that has several good use cases when you need to index an array in few different schemes, that otherwise one would have to emulate using associate and pointers, and I am not sure if there could be a performance penalty.

3 Likes
7

It is also more versatile as the sections can be defined (in 2D terms) raw-wise, column-wise, every Nth row/column etc. while the old F77-way is restricted to contiguous, memory-order subparts of an array.

4 Likes
8

Yes, contiguous arrays were an essential requirement for the old “F77-way”.

When using the Silverfrost FTN95 compiler for bounds checking, it provides / has available both the array size, as defined in the calling routine and the array size when it was origionally allocated, as these can differ for code that uses the “F77-way”.

Prior to ALLOCATE, this was an important issue for FORTRAN users.

A lot of lost knowledge if “Was this needed in F77 somehow?” is being asked.

1 Like
9

It seems this feature has nothing to do with allocate, but rather with the fact that in F77 you couldn’t pass a non-contiguous array section into a function as in f(A(3:5,3:5)), and rather you had to pass the full array via one of the approaches above.

Consequently it seems this feature is indeed not needed since Fortran 90, which allowed passing array sections to function calls, which makes it easy to pass exactly an array of the right size.

10

I would be a bit careful here: with 1D arrays it is indeed always possible to pass a contiguous section if one wants to pass the exact size. With nD arrays, as you are noticing, this may imply passing non-contiguous sections, and consequently triggering a copy-in/copy-out. So, for nD arrays, it looks to me still desirable to be able to pass actual arrays that are longer than the declared size of the dummy argument.

11

Ordinary bounds checking should not invalidate perfectly standard F77 code. You might want some kind of super bounds checking that did so, but bounds checking should not invalidate code merely because the compiler writer knows a better way to accomplish the same effect using more modern code.

You aren’t thinking of rejecting the practice of passing a portion of an array to a function with a lessor rank, are you? I pass a 2d array of the tax brackets for a single year and filing status to a function from a 4d array that includes all years and filing statuses from hundreds of places in my US income tax calculator. It saves a lot of code compared to a separate named array for each. No doubt there is a better way to do this now, but there wasn’t back in 1979 when we started out.

Maybe this is an appropriate time for me to advocate run-time checking for uninitalized variables. The additional runtime is very much worth it when a program suffers from mysterious failures, especially random ones. It should include checking integers and characters. Just initializing reals to NaN is not sufficient. Optimization can move many of the checks out of loops, minimizing the slowdown if the check is even left on in the production version of the code.

12

In pre-Fortran-77 codes, when the size of the array was unknown, it was also common to define the dummy array argument with a size of 1 (or in the case of multi-dimensional arrays, the final dimension). Unfortunately this had the side effect of triggering bounds checking errors almost immediately. Fortran-77 introduced using the asterisk as an alternative.

13

As already mentioned above, in modern code, subblocks of larger matrices can be passed using array notation. The example given above by @certik was f(A(3:5,3:5)). If the subprogram interface is implicit, then this does trigger copy-in/copy-out argument association when this subblock is not contiguous (it would be contiguous if, for example, the actual array were declared as (3:5,*)). Even if the interface is explicit, there are still cases where copy-in/copy-out is triggered, but if the dummy argument is assumed shape, then most compilers will pass the original data along with the metadata that accounts for the spacings between elements of rows and columns. In f77 code, this subblock would be passed as something like f(3,3,A(3,3),lda). See for example any of the dozens of LAPACK routines that work with matrices that use this convention. The first two arguments indicate to the subroutine that a 3x3 subblock of the matrix is being passed. The A(3,3) argument is the first element in the subblock. The lda argument is the leading dimension of the actual matrix argument and is used within the subroutine as the spacing between elements within a row of the dummy argument. This is all based on storage sequence association, which was defined early on within fortran, before even f66 I think, and still applies in modern fortran for explicit shape dummy arguments (m,n), assumed size dummy arguments (m,*), and for arguments with the contiguous attribute.

I think the original proposal in this discussion was to print warnings whenever this argument passing convention is used by a programmer. Although I do think this would be a good option for the programmer to be able to invoke from time to time during debugging, I do not think it should be the default because 1) it is a common coding convention in legacy code, and 2) it is standard fortran (perhaps with some corner case exceptions). I do agree that the modern conventions have advantages, which is why they were introduced into the language in the first place, but as I stated previously, it is very frustrating to a programmer for a compiler to print false positive warnings for standard features that the programmer intentionally uses.

I don’t think I have written any new code in the last 30 years that uses this convention, but of course I use some half a million to a million lines of legacy code where it is used throughout, and even new code that references one of these legacy routines would still use this convention.

The corner case exceptions include the following situation. Suppose the actual argument array is dimensioned A(6,3). That array has 18 contiguous elements in the storage sequence. Suppose the subroutine is invoked as above with actual arguments f(3,3,A,6) and that it is written as

subroutine f(m,n,D,ldd)
integer :: m, n, ldd, D(ldd,n)

The dummy array will have dimension D(6,3), and if written correctly, only the leading 3x3 subblock of elements of that array will be referenced. The actual matrix storage locations of those nine elements are (4,5,6,10,11,12,16,17,18), and they correspond to the dummy matrix storage locations (1,2,3,7,8,9,13,14,15). If only those elements are referenced, then all is fine. But the dummy array actually has 18 elements in its scope. Of those 18 elements, the dummy storage locations (4,5,6,10,11,12) map back to the actual array elements (7,8,9,13,14,15). So even if those elements were referenced (presumably by mistake, because they are outside the intended 3x3 subblock), it would not violate the standard because those dummy elements have a defined association with actual matrix elements that exist. (This shows one advantage of the newer A(3:5,3:5) actual argument convention, which would allow the compiler to detect these out of bounds errors.)

However, there are three more dummy array elements with dummy storage sequence (16,17,18) that map back to actual storage sequence locations (19,20,21). These elements are beyond the range of the actual array argument. In practice, if those dummy elements are not referenced, then the code works as intended. This has been the case for all versions of fortran, at least back to f66. But it is unclear if this convention actually satisfies the letter of the standard. It is possible that even the dummy declaration itself, D(6,3) in this example, violates the standard, even if those trailing three elements in the storage sequence are never referenced. Maybe someone who is familiar with the standard wording on this issue can comment on this?

The workaround for this corner case is to declare the dummy argument instead as D(lda,*), which is D(6,*) in the above example. I think this is what LAPACK does, although earlier libraries such as EISPACK and LINPACK could not do that prior to f77. This basically tells the compiler that the programmer is taking all responsibility to stay within the actual argument storage locations, and to not do any bounds checking at all on that last dimension. This has some disadvantages when debugging the code, and even during production runs, because the compiler cannot detect at run time any array bounds violations for that second dimension, and that can cause undetected program errors or even memory access faults during execution of the code.

14

Intel ifort, with all its majesty, used to do copy in/out even when a full array was passed with explicit full bounds. I reported it to Intel several years ago as a bug, but learned the lesson of not specifying (even the full) bounds on the leading dimensions, and use : instead.

1 Like
15

In an entirely Fortran program, the compiler is in charge and can keep track of the sizes of all arrays if it chooses to. The NAG Compiler does that with the -C=calls -C=array option. The caller generates code to write actual argument array extents in some static area and the callee updates its view of the extents of the dummy arrays based on the contents of same static area.

If a human can keep track of the sizes (and we can otherwise why would we even talk of a bug), the compiler certainly can. If the compiler documents how it’s doing it, even a companion processor can be taught how to do it (or the person writing code for the companion). Everything is checkable, in principle, with some effort.

2 Likes
16

Correct. If we do stricter bounds checking than the standard allows, it needs to be a separate option.

As I said above (Passing in a longer array than a function expects - #6 by certik) I think that’s a separate feature and I think it doesn’t have an equivalent, so indeed I am not thinking of rejecting it. And everything has to be configurable anyway.

1 Like