For a small project (see Anecdotal Fortran... :-) - #8 by vmagnin), I need to write binary files in a little-endian order. In the following program, where I suppose the integers to be 4, 2 and 1 bytes (they are on my machine), I have successfully done what I want (but…):
program endian
implicit none
integer :: status
integer(4) :: four
integer(2) :: two
integer(1) :: one
four = 4
two = 2
one = 1
open(unit=1, file='endian.bin', access='stream', status='replace', &
action='write', iostat=status)
write(1, iostat=status) four, two, one
close(1, iostat=status)
end program endian
Now, my question is: will my program write in little-endian on all machines? I have not found the term “little-endian” in the Fortran 2018 draft. And I therefore guess it is not guaranteed by the standard… Probably the endianness used in files is just the same as in RAM?
I know that Intel allows you to specify a default endianess for all output on the OPEN statement ie OPEN(NEWUINT=iunit, CONVERT=“BIG_ENDIAN”, etc. I think this is an extension though and not part of the standard. I believe gfortran also supports CONVERT. If I remember the CONVERT extension has been around for a while in various compilers. I first saw it several years ago in the Cray compilers. Also, as far as I know, the IBM Power chips are big endian but just about everything Intel or AMD X86-64 is little endian so unless you are also targeting the POWER chips I doubt you need to worry about endianess. Just speaking for myself I would like to see CONVERT offically part of the standard as well as the ability to specify an encoding like base64 or XDR because one major application (VTK toolkit) requires big-endian format for all unformatted/binary files for the VTK legacy format and the newer VTK XML format needs base64 encoding to embed binary data into the XML files.
About the integer sizes.
Although must of compilers assume the kind is the size, but it is not the case for all of them, nagfor for instance. So yes, you should use ISO_FORTRAN_ENV, but I don’t know if int8 or int16 exist.
I routinely use -fconvert=big-endian with gfortran and -convert big_endian with ifort. I prefer using compiler options over compiler-dependent language extensions as it makes the code itself easier to manage.
Thanks @rwmsu and @milancurcic for the -fconvert flag. I think I will, as a first step, begin assuming the machine is little-endian and tell the other people to use the -fconvert=little-endian flag, if (lucky people) they have it!
INT8 , INT16 , INT32 , INT64 :
Kind type parameters to specify an INTEGER type with a storage size of 16, 32, and 64 bits. It is negative if a target platform does not support the particular kind. (Fortran 2008 or later.)
Re. finding endianess. Assuming your compiler supports INT8 and INT16, a quick way to check if you are doing big endian io is:
Create two INT8 variables (byte1 and byte2)
Create a INT16 variable (twobytes) and set it equat to 1_INT16
Open a scratch file for unformatted read/write and write out twobytes
ie
Open(newunit=nunit, STATUS=“scratch”, FORM=“unformatted”)
Write(nunit) twobytes
4 Rewind nunit and then read byte1 and byte2
Read(nunit) byte1, byte2
5 if byte1=0 and byte2=1 the file IO format is big_endian
This comes from some code found on the www.cfdbooks.com web site of Hiroaki Nishikawa (AKA Katate Masatsuka).
Just some background: Typically endian (I/O for unformatted files) uses the native endian assumed by the CPU hardware. That way the I/O amounts to a block move of bits with no rearrangements. If you want the endian reversed (typically BIG and LITTLE are the only options), you are usually writing a file intended to be read by a different system that uses the other endian. Most compilers provide a compiler option or that, or some extension to the OPEN statement. (As others have noted above.)
I’m curious as to if Cray or other vendors ever considered adding a base64 (or other) encoding IO option as a compiler extension. This would be a big help for writing things like the VTK XML format files where you want to mix base64 encoded binary with standard text. I presume it would have to be an option on the read and/or write statement. something like
Having that capability in a compiler would have saved me the time and the almost 1000 lines of code it took to develop my own base64 encoding/decoding package.
Thanks everybody,
yesterday I have successfully generated my first sinusoidal waves in Fortran:
$ file output.wav
output.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, stereo 44100 Hz
$ play output.wav
output.wav:
File Size: 21.2M Bit Rate: 1.41M
Encoding: Signed PCM
Channels: 2 @ 16-bit
Samplerate: 44100Hz
Replaygain: off
Duration: 00:02:00.00
In:2.32% 00:00:02.79 [00:01:57.21] Out:123k [!=====|=====!] Hd:0.0 Clip:0
I will push on GitHub a first version of that personal project, using of course fpm, within one week. Maybe some people here are interested to (modestly) walk with me in the steps of Stockhausen, Kraftwerk, Jean-Michel Jarre, Daft Punk, and other giants! I will make a post in the Discourse when it is ready.
Thanks @ivanpribec data sonification has been in my thoughts for many years (although I didn’t know the term), but I have never done it. It could be another serious motivation to work on that geek toy project (but I believe music is also something serious!).
I often have to read and write data in the big-endian order. I do not trust options like -fconvert because they either apply to all external units and that is completely unacceptable, or one has to specify the units to apply at compile time and that is not acceptable for me either.