Formatted stream read

Help
21

The problem I have observed is when combining FORMATTED STREAM with List Directed I/O reading.
With Gfortran, List Directed I/O reading appears to not stop at the end of the number (identified by a valid delimiter), but steps to an <lf>.
I wonder what would happen if no <lf> exists ?

I note that list directed I/O is not permitted unless the stream file is opened as ‘FORMATTED’. This puzzles me, as the format can be used to generate a packet of text. (could be achieved as an internal write then transfer the character buffer to the stream file)

In Modern Fortran explained, there is very little describing this combination.

There may be other problems that I am not aware of!

22

I have seen this file truncation problem before, but it does not make any sense to me to truncate a formatted stream file, especially if you can position before the end of file to overwrite text.
Stream files should not be truncated.

23

So both the gfortran and ifort versions I am running appear to have the file in the same state until program termination, where they do not truncate; but overwrite. But at program termination ifort truncates the file at the current position; and gfortran adds an end-of-line at the current position,
even if the current position was set with the non-advancing I/O statement with no parameters; which the standard indicates per the above reference should do nothing but change the position in the file.
So I would say they are both wrong at program termination, and that the files should NOT be truncated
except perhaps if a call to ENDFILE was made(?).

program t
implicit none
integer,parameter :: isz=10
integer :: i, iu, ipos(isz)=1, isize
open(newunit=iu, file='overwrite.txt', access='stream', form='formatted')
do i=1,isz
  inquire(iu,pos=ipos(i))
  write(iu,*)'write=',i,'ipos=',ipos(i)
enddo

inquire(iu,size=isize)
flush(iu)
call execute_command_line('cat overwrite.txt')
write(*,*)'file size is',isize

write(iu,*,pos=ipos(8))'new eigth line'
inquire(iu,size=isize)
flush(iu)
call execute_command_line('cat overwrite.txt')
write(*,*)'file size is',isize

write(iu,'(a)',pos=ipos(3),advance='no')'new third line'
inquire(iu,size=isize)
flush(iu)
call execute_command_line('cat overwrite.txt')
write(*,*)'file size is',isize

write(iu,'(a)',pos=ipos(6),advance='no')
inquire(iu,size=isize)
flush(iu)
call execute_command_line('cat overwrite.txt')
write(*,*)'file size is',isize


end program t

$ gfortran --version
GNU Fortran (Ubuntu 13.1.0-8ubuntu1~20.04.2) 13.1.0
$ ifort --version
ifort (IFORT) 2021.10.0 20230609

So after the program terminates the output file should contain 10, 8, 6, or 3 lines? and what should the file contents be? Both compilers truncate to the 8th line on the advancing I/O and overwrite on the 3rd,
but then change the file on program termination differently. The standard implies I should be able to return to any position returned by INQUIRE, which creates an interesting catch-22 with regards to the truncation. And then if I return to the position captured in IPOS(10) where am I? I have changed the file since I recorded the position. I found that even though both compilers appear to have truncated the file to line eight because of the advancing I/O that I can still return to POS=IPOS(10) without an error but if I then write it is appended to line eight and does not restore the filesize to what it was when I recorded IPOS(10); so I find the positions confusing; particularly if I have written other positions. Is anyone else as confused as I am as to what the output of the file should be after running the following, after each stop?

test program 
   program t
implicit none
integer,parameter :: isz=10
integer :: i, iu, ipos(isz)=1, isize, qpos
open(newunit=iu, file='overwrite.txt', access='stream', form='formatted')
do i=1,isz
  inquire(iu,pos=ipos(i))
  write(iu,*)'write=',i,'ipos=',ipos(i)
enddo

inquire(iu,size=isize,pos=qpos)
flush(iu)
call execute_command_line('cat overwrite.txt')
write(*,*)'file size is',isize,qpos

write(iu,*,pos=ipos(8))'new eigth line'
inquire(iu,size=isize,pos=qpos)
flush(iu)
call execute_command_line('cat overwrite.txt')
write(*,*)'file size is',isize,qpos

write(iu,'(a)',pos=ipos(3),advance='no')'new third line'
inquire(iu,size=isize,pos=qpos)
flush(iu)
call execute_command_line('cat overwrite.txt')
write(*,*)'file size is',isize,qpos
! stop 1
! position with no I/O

write(iu,'(a)',pos=ipos(6),advance='no')
inquire(iu,size=isize,pos=qpos)
flush(iu)
call execute_command_line('cat overwrite.txt')
write(*,*)'file size is',isize,qpos
! stop 2

write(iu,'(a)',pos=ipos(10),advance='no')
inquire(iu,size=isize,pos=qpos)
flush(iu)
call execute_command_line('cat overwrite.txt')
write(*,*)'file size is',isize,qpos
! stop 3

write(*,*)'BACK TO ',ipos(10)
write(iu,'(a)',pos=ipos(10),advance='no')'EOF'
inquire(iu,size=isize,pos=qpos)
flush(iu)
call execute_command_line('cat overwrite.txt')
write(*,*)'file size is',isize,qpos

!stop 4

end program t

The more I played with the program and those two compilers the more uncertain I became as to what I should expect in the file. Is this worth asking for clarification from J3 or does someone know for certain
what should be truncated and where output should go for this little test case?

24

As the OP, I thank all you guys for explanations. Still, I am not convinced that the reality of the formatted stream access is what we should expect, either at the Standard level or the implementations, or both. If it, however, really must be as it is, I consider it a lost opportunity of providing the Fortran programmers with truly effective way to do the I/O (mostly read).

Again, IMHO treating a stream file, sequence of file storage units, in a record-oriented way seems self-contradictory.

What this statement is supposed to mean? If taken literally, I would say that we could, in principle, have a true stream access (in the C-way) if only a processor provided it, still staying standard conforming. A processor may allow a file to be viewed both as a record file and as a stream file. The word may, AFAIU English, ist not must. So, I guess, it could treat a file opened with access='stream', just as most people (or me, at least) understand stream - a sequence of file storage units a.k.a. bytes, as opposed to a sequence of records, a.k.a. lines.

What we have now, is a dysfunctional hybrid of record- and stream-oriented I/O, requiring advance='no' to come a bit closer to what stream access should be, at the cost, however, of excluding list-directed input. And so, 34 years after introducing the Modern Fortran, one still cannot read freely formatted file of, say, numbers just by using repeated read statement, without having to write parsers, modules and libraries. This is what I call a lost opportunity.

25

If the program is reading (not writing) the file, then what is the practical difference between record and stream access? You can use advance='no' one character at a time on both.

26

@urbanjost

I have always used open (unit=iu, file=‘overwrite.txt’, access=‘stream’), which appears to avoid the file truncation problem.
However, when I was experimenting with form=‘formatted’, Silverfrost FTN95 will not truncate a formatted stream file if pos= has been used to access the file. This solved this problem for my usage.
It really looks like a problem with the standard definition, especially where 3 compilers give different implementations.

I also find that use of List Directed I/O on a file with stream access is also a problem with standard definition or implementation, as it is based on an end of record marker.
Until this is clarified and corrected, access=‘stream’, form=‘formatted’ will not provide a good solution.

27

And that is my very point. Almost no difference. And the stream access should treat end-of-line marker as an ordinary character, just as it is done when the file is open as unformatted. Then we would not have to use advance='no' and could use list-directed input. Actually advance should be forbidden on stream files, as they have no records to advance.