The data used in my programs usually starts out as CSV files. If they are large and reading them takes a long time, I can read the data and then write it to a file using an unformatted stream. Reading such a file is very fast if you know the data types of the data written. How do you document this? The unformatted stream files I have created are only intelligible if I remember what programs wrote them, and that concerns me.
Maybe whenever you create an unformatted stream file foo.bin you should create a file foo.fmt describing the contents, for example
Unformatted sequential access files are usually preferred over unformatted stream access files unless you need the arbitrary positioning capability of the stream access facility.
May I ask why are unformated sequential access files usually preferred?
I usually use unformatted stream access files because I must read C+±based files in which bits are stored. As a habit, I therefore also use unformatted stream access files to store integer/real arrays.
If your “data” have some "class"ification to them (e.g., particular results of a simulation at certain intervals), you can consider using a Fortran derived type as a “class” for such data and implement defined IO for reading them in (and writing them out as well).
The subroutines you write for defined IO can then include in them (inline) documentation and you can try to process the routines using FORD or some such system for HTML type output. This may make it easier for you to work with the data in your code as well as your clients and also help you manage what the data are all about.
Unformatted sequential read is best for reading a file written with unformatted sequential write because, assuming the read statement data lists are correct, the READ statement will read the data correctly without any further action. The length of each written record is embedded in the file. For files written by C++, the stream access method is usually better.
Simplest solution seems to me to start every file with a CHARACTER(LEN=512) and write in there whatever you would put in foo.fmt. You could even have a continuation marker at the last position of the CHARACTER, if you need to add another CHARACTER(LEN=512). Alternatively, you could start using netCDF or HDF libraries.