We’ve added this subroutine to our electronic structure code:
subroutine idle
use modmain
implicit none
! time interval between checking for the STOP file
integer, parameter :: tcheck=30
integer i
real(8) ts0,ts1
! set the stop signal to .false.
tstop=.false.
do i=1,tidle/tcheck
! wait for tcheck seconds
call timesec(ts0)
ts1=ts0
do while (abs(ts1-ts0) < tcheck)
call timesec(ts1)
end do
! check for STOP file
call checkstop
if (tstop) return
end do
end subroutine
What this does is idle the calculation for tidle seconds. This is useful on HPC clusters when it takes time for a job in the queue to start running because resources are unavailable. It can be time-consuming if you need to repeatedly adjust parameters and resubmit jobs.
This routine scans for a file named STOP every 30 seconds and if it’s found it deletes it and restarts the calculation with a new input file. It works very well and saves a lot of (human) time.
However, it also burns 100% CPU just idling and I was wondering if this could be reduced.
Now, I’m aware that the sleep command exists as an extension in both GNU and Intel Fortran (and possibly others). I’m also aware that there are ways to get Fortran to sleep by calls to C++ or POSIX functions. An interface to usleep and winsleep is also available in Fortran stdlib. However, I’d like to avoid mixing languages or adding an entire library for this single purpose.
I presume that it’s not possible in standard Fortran as it has to involve system interrupts in order to suspend a thread (although I’d be happy to be wrong). If so, then I think it would be useful to standardize the sleep subroutine in Fortran.
I guess, so far the best you can achieve in terms of Fortran-based solution involve the use of iso_c_binding. I reported a bug some time ago of stdlib related to sleep which may contain some useful pieces of information.
I am no export with OpenMP, but is it not possible in your case to fork a new thread and play with the wait policy. Your description makes me think of a passive wait policy.
Yes. Calling a script that handles the system-dependent parts is a good solution as long as it is not something called very very frequently, as spawning a process incurs a significant amount of overhead. Sounds like this can be a long-running command so that would suffice. If security is a concern you probably want to use a full pathname and take other precautions to make sure the correct command is called.
To select between other alternatives requires knowning how many platforms you need to support, including compilers and OS types. The advantage of a vendor-supplied extension is that it is generally OS-agnostic so you make the same call on MSWindows, Unix, Linux, OpenBSD, … You can make a wrapper file containing cpp preprocessing directives that provide you with a consistent interface to the vendor abstractions as well. sleep(3c) and usleep(3c) are both very easy to call from GNU/Linux and Unix platforms. See urbanjost · GitHub /M_system and github.com/urbanjost/M_vendor for some examples. The stdlib library has an interface to sleep as well; and there are several examples on the Fortran Wiki. Without knowing a little bit about what compilers and OS need supported it is hard to recommend one method over the other. I would recommend first looking at
It is kinda sad that the POSIX Fortran bindings were ignored by many Fortran vendors. Cray (now part of HPE), Intel, and in the past a handful of others have supported them. A simple call to PXFSLEEP is all that is needed.
Invoking a shell command (or script) via execute_command_line shouldn’t be that expensive on modern systems with decent implementations of fork/exec, COW virtual memory pages, and so on. Especially since in this case it is only called once every 30 seconds.
The argument on the other side is that the POSIX sleep command is accessible in a standard way from C, and the interface to fortran is now (since f2003) defined in a standard way, so it is straightforward, and just as portable, to do that as it would be to use the POSIX Fortran bindings.
This only applies to POSIX based operating environmants. It does not solve the problem with writing portble code for Windows and other nonPOSIX systems. It would require a standard fortran intrinsic to achieve that.
I usually associate the sleep() command with interactive codes, where the computer with a nanosecond clock needs to wait on a human with a second reaction time to do something, and you don’t want to waste cpu cycles in a timer loop. However, I think the scenario in the original post makes a good case for the need for a sleep() command in other situations too.
Normally the sleep() command takes an integer argument, which is the time in seconds. One criticism is that often a shorter sleep period is appropriate, say in the millisecond range, e.g. to wait for some network operation to complete or for data to be pushed through a channel to an external device. There is a nanosleep() C function for this too, but it takes a structure as an argument instead of an integer or a float, so it is a little more complicated to use from fortran.
As of today, are there any non-POSIX operating systems left standing other than Windows?
A lot more complicated, I may say, although not because of the timespec struct:
Even though, just like sleep(), the nanosleep() function is a cancellation point, the nanosleep-one has an argument to return the remaining time —and that, possibly combined with OpenMP support, makes things… interesting.
But in any case, the next standard revision could really benefit from reviewing/enhancing the time-related support. Maybe adding a flag to increase date_and_time resolution to nanoseconds; adding a sleep subroutine with an optional nsecs argument; etc.
About 25 years ago I wrote my own version of the PXF library. I developed it on Windows NT, using Cygwin as a POSIX layer. Then later ported it to ubuntu linux. It was almost complete. The remaining issues were where there needed to be some integration with the Fortran run-time for I/O. At the time, g95 was just getting off the ground. I never got around to modifying it, or later on, gfortran.
The test suite that I wrote for it found a number of bugs in both the Cray (MIPSpro by then), and Intel implementations. All long since fixed.
Just curious. I know Intel supported POSIX directly (at least on Linux) in ifort and I think that was carried over into ifx. What other Fortran compilers (if any) support POSIX, at least on Linux.
According to the ifposix.f90 file shipped with ifort, the standard is “IEEE POSIX Std 1003.9-1992 library routines”, so it’s clearly aimed at Fortran 77.
The Sun compiler used to ship with libFposix.so (and required the -lFposix flag), which seemed to conform to the PXF interface, but I have no idea if that’s true (things touched by Oracle are often awkward and exhausting, ).
The gfortran compiler has some POSIX support, but as an extension, and without the “PXF” prefix.
The non-Windows limitation is directed towards the infamous fork, but I think most of the other stuff works on Windows.
There were many fortran compilers that supported pxf, but mostly on various proprietary unix operating systems, before linux was popular. Sun, IBM, DEC, HP, and Cray come to mind. Also, I think there was a library for CVC fortran on CTSS, which was not a unix OS. Remember, this interface library was designed for f77, and we’re talking about the early 1990s, before even f90 was common. One of the criticisms of the library was that it was never updated to use f90 features (i.e. derived types, KINDs, etc.), it mostly remained just an f77 library all through the 1990s and early 2000s, until the f2003 C interop features became available.
The only syntax extension PXF required over Fortran 77 was long (up to 31 character) external names. For a complete implementation, a few mods are needed in the run-time library. E.g., allowing access to file descriptors and the mapping between them and Fortran unit numbers, streaming I/O, and a few other things. It also required writing a handle manager (PXFSTRUCTCREATE and friends) to create, modify, and free C structs for the system calls that need them.
F2003 C Interop isn’t a panacea - especially when trying to interoperate between C structs and Fortran derived types. It isn’t so bad within an application or library where both Fortran and C sides are under a single developers control. But given the variety of POSIX environments, where the contents of the C header files (structs, constants, etc) can vary considerably from system to system, C Interop can introduce as many problems as it solves. So one still needs to write an interface layer where both sides are under the same control.
The ‘idle’ feature in our code is just a convenience and not a critical part. It’s OK if it fails on some systems, the only consequence would be that the job will have to be resubmitted.
I agree. It would be a good feature to have as standard. There may be other situations such as waiting for a slow filesystem to catch up with the code, waiting for other independent codes to complete, or just performing a calculation at a particular time of the day like a cron job.
).