Pointer to a growing allocatable array in gfortran error

Help
1

I am using an allocatable array to append an array of data onto it multiple times. Each time the data is added to the array, I am creating a pointer array to the added data in the array. I noticed that after the allocatable array size goes over 4, the next time I add data to the array, the pointer to the first four values in the array is a random large number.

This is only happening with the gfortran compiler. When I use the ifort compiler I do not have the same problem. At this point I plan on using a different method to build this array, however, I’m curious on why this is happening in the first place.

Example code is below.

program example
  implicit none
  integer, parameter :: nApp = 5
  integer, dimension(:), allocatable, target :: TargetArray
  integer, dimension(nApp), target :: AppendArray
  integer, dimension(:), pointer :: p1, p2, p3
  integer :: nn, ii

  nn = nApp
  allocate(TargetArray(0))
  AppendArray = [(ii, ii = nn-(nApp-1), nn)]
  TargetArray = [TargetArray, AppendArray]
  p1 => TargetArray(nn-(nApp-1):nn);
  print *, p1

  nn = nn + nApp
  AppendArray = [(ii, ii = nn-(nApp-1), nn)]
  TargetArray = [TargetArray, AppendArray]
  p2 => TargetArray(nn-(nApp-1):nn);
  print *, p1
  print *, p2

  nn = nn + nApp
  AppendArray = [(ii, ii = nn-(nApp-1), nn)]
  TargetArray = [TargetArray, AppendArray]
  p3 => TargetArray(nn-(nApp-1):nn);
  print *, p1
  print *, p2
  print *, p3
  print *, TargetArray

end program example

The output when using gfortran compiler:

           1           2           3           4           5
   995220502           0  2105687195   419310127           5
           6           7           8           9          10
   995220502           0  2105687195   419310127           5
           6           7           8           9          10
          11          12          13          14          15
           1           2           3           4           5           6           7           8           9          10

The output when using ifort compiler:

           1           2           3           4           5
           1           2           3           4           5
           6           7           8           9          10
           1           2           3           4           5
           6           7           8           9          10
          11          12          13          14          15
           1           2           3           4           5           6
           7           8           9          10          11          12
          13          14          15
2

Hi @MattG great question and welcome to the Discourse.

It’s yet another unintuitive result of automatic LHS reallocation. The second TargetArray = [TargetArray, AppendArray] makes the subsequent print *, p1 non-conforming (invalid Fortran), because per 9.7.3.2 Deallocation of allocatable variables (page 150 of the 2023 standard):

Deallocating an allocatable variable with the TARGET attribute causes the pointer association status of any pointer associated with it to become undefined.

So p1 becomes undefined (dangling) because TargetArray is automatically reallocated and most compilers then typically return some uninitialized memory when you print p1.

P.S. We are planning to add dangling pointer checks to LFortran so that you would get a nice runtime error that p1 is dangling, thus avoiding this confusion what happened; but we haven’t got to it yet.

3

How would you modify the code to maintain the consistent association of p1 with the same elements after reallocation of the target? Would you have to store the bounds in temp variables and then reassociate the pointer with an array slice after reallocation?

4

What at first sight seems like a feature of ifort, actually looks like a failure to call free as soon as possible (maybe as an optimization?).

In the OP’s example. after p2 has been assigned a target, p1 may still point to the old memory region. And after p3 has been assigned a target, both p1 and p2 may still point to old memory regions.

The language already provides a way to mitigate the issue, through the associate-construct —you just do the corresponding associations right before usng p1, p2, p3.

But if the associate-construct is not deemed suitable (or is undesired), then something like

p1 => pointer_to_region(TargetArray, nn-(nApp-1), nn)

Should be reissued after every TargetArray increase.

(Btw, Go has a similar issue, somewhat mitigated by array capacity, when you have slices s1 and s2 pointing to the same underlying array, and you do s1 = append(s1, ...), then s1 and s2 might end up pointing to different underlying arrays.)

5

Ifx shares the ifort bug that @certik and @jwmwalrus diagnosed.

6

The Gfortran problem is that p1 and p2 need to be reset after re-allocate.
The following code appears to identify and fix this problem with Gfortran (or potentially the original code ?)

program example
  implicit none
  integer, parameter :: nApp = 5
  integer, dimension(:), allocatable, target :: TargetArray
  integer, dimension(nApp), target :: AppendArray
  integer, dimension(:), pointer :: p1, p2, p3
  integer :: nn, ii, n0, kk

  call report_version

  write (*,fmt='(/a)') 'First'
  n0 = 0
  allocate(TargetArray(0))
  AppendArray = [(ii+n0, ii = 1,nApp)]
  TargetArray = [TargetArray, AppendArray]

  nn = n0 + nApp
  p1 => TargetArray(n0+1:nn)
  print *, 'p1 ', p1

 do kk = 1,4
  write (*,fmt='(/a,i0)') 'Next ',kk
  n0 = n0 + nApp
  AppendArray = [(ii+n0, ii = 1, nApp)]
  TargetArray = [TargetArray, AppendArray]

  nn = n0 + nApp
  p2 => TargetArray(n0+1:nn)
  print *, 'p1x', p1
  print *, 'p2x', p2

  p1 => TargetArray(1:nApp)
  p2 => TargetArray(nApp+1:nApp*2)
  p3 => TargetArray(n0+1:nn)

  print *, 'p1f', p1
  print *, 'p2f', p2
  print *, 'pef', p3

  write (*,fmt='(/a)') 'Full Array'
  print *, TargetArray
 end do

end program example

    subroutine report_version
     use iso_fortran_env , only : compiler_version, compiler_options
     implicit none

     write (*,*) 'Version : ', compiler_version ()
     write (*,*) 'Options : ', compiler_options ()

    end subroutine report_version

I also tried using a temporary array which may tidy up memory management ?
integer, dimension(:), allocatable :: Temp
Temp = [TargetArray, AppendArray]
TargetArray = [Temp]

7

Here is what I think is happening.

After that first statement, the old TargetArray is deallocated. The pointer p1 was associated that now-deallocated array, so it is now a dangling pointer. The fortran language does not define what happens to deallocated arrays, so they can have any value, including the original value or new values if, for some reason, the compiler decides to reuse that memory for something else.

You are doing i/o. so it is common for the i/o library to allocate temp workspace and possibly reuse the previously allocated memory.

The same thing applies to the last append and pointer assignment sequence. After that sequence, both p1 and p2 are dangling pointers.

8

In the original example code with some added prints, gfortran 13.1.0 reports the associated() status as .true. for the pointers even after the automatic deallocation. Is this correct behavior?

program example
  implicit none
  integer, parameter :: nApp = 5
  integer, dimension(:), allocatable, target :: TargetArray
  integer, dimension(nApp), target :: AppendArray
  integer, dimension(:), pointer :: p1, p2, p3
  integer :: nn, ii

  nn = nApp
  allocate(TargetArray(0))
  AppendArray = [(ii, ii = nn-(nApp-1), nn)]
  TargetArray = [TargetArray, AppendArray]
  p1 => TargetArray(nn-(nApp-1):nn);
  print *, p1
  print *, associated(p1), associated(p2), associated(p3)
  
  nn = nn + nApp
  AppendArray = [(ii, ii = nn-(nApp-1), nn)]
  TargetArray = [TargetArray, AppendArray]
  p2 => TargetArray(nn-(nApp-1):nn);
  print *, p1
  print *, p2
  print *, associated(p1), associated(p2), associated(p3)
  
  nn = nn + nApp
  AppendArray = [(ii, ii = nn-(nApp-1), nn)]
  TargetArray = [TargetArray, AppendArray]
  p3 => TargetArray(nn-(nApp-1):nn);
  print *, p1
  print *, p2
  print *, p3
  print *, TargetArray
  print *, associated(p1), associated(p2), associated(p3)
  
end program example

Output:

           1           2           3           4           5
 T F F
  1258871088       25915  1258864656       25915           5
           6           7           8           9          10
 T T F
  1258871088       25915  1258864656       25915           5
           6           7           8           9          10
          11          12          13          14          15
           1           2           3           4           5           6           7           8           9          10          11          12          13          14          15
 T T T

And strangely, the value of p2 is never junk, it still points to the correct memory even after the automatic reallocation.

Edit: Even stranger if I add another line to the end of the program so the final two lines are:

  print *, associated(p1), associated(p2), associated(p3)
  print *, associated(p1,target=TargetArray(1:5)), associated(p2,target=TargetArray(6:10)), associated(p3,target=TargetArray(11:15))

Here’s the output of the final two lines:

T T T
F F T
9

The runtime output from NAG compiler (7.2) with -C=all -C=undefined leaves no doubt:

1 2 3 4 5
Runtime Error: mattg.f90, line 20: Reference to dangling pointer P1
Target was deallocated by automatic reallocation at line 18 of mattg.f90

10

1 2 3 4 5
Runtime Error: machalot.f90, line 15: Undefined pointer P2 used as argument to intrinsic function ASSOCIATED

POINTER objects have 3 states: associated, disassociated and undefined. ASSOCIATED is only legal to call when the POINTER is in one of the first two states. Annoying, perhaps, but necessary in the interests of runtime performance and simplicity of implementation.

11

Thanks @themos, I was hoping NAG already does this. And it does.

12

You would use different allocatable arrays, one for the p1 pointer assignment, another for the p2 assignment, and another for the p3 assignment.

13

Yes. p1 being undefined at that point, associated(p1) has an undefined behavior, meaning that it can return anything, or crash, or start WW III.

14

I thought the whole point of associated was to check if the pointer had been pointed at something.

because “growing the allocatable” caused reallocation, the original memory pointed to by pointer has no bearing on the post-growth memory containing the new allocatable.

the pointer has still been given a target, and thus associated is true. However, other actions have caused the target to no longer be the same as when the pointer was originally pointed.

15

Maybe this is the conceptual problem. When the old target becomes deallocated and possibly reallocated at a new memory location, the various pointers (there may be several) to the old memory do not get modified or reassigned to point to that new location, and there is no way to tell from the pointer metadata that the old memory is no longer defined, so their status simply becomes undefined. The pointer metadata includes the memory location, rank, bounds, and strides. That implies that it is the programmer’s responsibility to know when that occurs, not the compiler’s.

When the pointer is nullified, then the metadata does change. The pointer is not in an undefined state, it is now in a defined state of being dissociated, and the associated() intrinsic can correctly report that state.

16

That’s actually half the point.

The ASSOCIATED() intrinsic has a second, optional argument, to help you verify if the pointer (when used as an alias) is actually pointing to the intended target.

17

Only if the pointer is not undefined. Look in 19.5.2.5 “Events that cause the association status of pointers to become undefined” of the Standard. Seventeen cases are listed, including

(3) the target of the pointer is deallocated other than through the pointer

The reason for this is runtime efficiency. The responsibility is on the programmer, not on the run-time system. Of course, some compilers might, if asked to, emit less efficient code but provide help to the programmer by issuing a runtime termination message about the offending pointer.

18

That’s not guaranteed to help with a pointer with undefined association status. You might get a wrong answer, crash the program, etc.

19

Here is an example of how associated() might fail in this case.

integer, allocatable, target :: a, b
integer, pointer :: ap
allocate(a)
ap => a
write(*,*) associated(ap), associated(ap,a), associated(ap,b)  ! should be T T F.
deallocate(a)                                                  ! ap is now undefined.
allocate(b)                                                    ! this could reuse the old memory.
write(*,*) associated(ap), associated(ap,a), associated(ap,b)  ! undefined, but might be T F T.
end program

$ flang  upt.f90 && a.out
 T T F
 T F T

I tested this with gfortran and flang, and they both print T F T at the end, which is one of several incorrect outputs. None of the associated() references is standard conforming in that last write statement because ap is undefined, and it is the programmer’s responsibility to avoid invoking the function with an undefined pointer argument. The first T is printed because ap was previously assigned to something, and its metadata has not changed since that time. The second F is the correct result because a is now deallocated, and the associated() intrinsic knows that ap and a are not associated; the function call itself is still nonconforming because ap is undefined. The last T is incorrect because ap and b were never associated with each other; it is printing T here presumably because the allocation of b reused the memory from the previous allocation. The old metadata for ap results in that function is being tricked.

There are several other ways to use pointer trickery to fool the associated() intrinsic into returning incorrect results when the pointer argument is undefined, or if two pointer arguments are given and at least one of them is undefined.

You can also use move_alloc(from=a,to=b) to move the allocation status from a to b, and in this case the pointer association moves with the new variable. associated(ap,b) is now T for the right reason, and it is defined by the standard to be T. This is a defined feature of move_alloc(), and the reason it works is because the pointer metadata is unchanged. Ironicaly that is the same reason the above code fools the associated(ap,b) call.