Global Ocean Modeling With GPU Acceleration in Python

The topic on focus should be heterogeneous computing: Especially Intel does currently promote this for emerging systems consisting of CPUs, GPUs, FPGAS, and upcoming new types of accelerators, for the exascale era.

The PGAS programming model was designed for such a purpose specifically: At the SPMD-level (Fortran: within coarray teams) for each specific part of the heterogeneous hardware, and at the APGAS-level (Fortran: across coarray teams) as the glue to allow access among the distinct parts of the heterogeneous hardware:

Fortran is not an island and if we take a look at other programming languages we may find channels and coroutines as an emerging and attractive trend for parallel programming. Obviously, Fortran has no built-in channels and no peculiar syntax for implementing and handling of coroutines. But Coarray Fortran is configurable through customization: We can implement channels based on coarrays (it requires a new type of synchronization method), and we can implement coroutines with Fortran as well.

Both, channels and coroutines may look different in Fortran though: Channels must be in two flavors, at the SPMD-level and at the APGAS-level. Coroutines can be implemented by (usually) merging two coroutines into a single subroutine (using Fortran block constructs) and even further by combining multiple such constructs and split them into module procedures (in submodules). A single coroutine is usually associated with a task thus, merging coroutines and further combining them leads into full featured (fault-tolerant) task-pools. We use the traditional control constructs to control execution flow of the coroutines, no special start-wait-stop syntax is required.

A promising strategy for heterogeneous computing should be to implement algorithms (usually) at the SPMD-level and to embed any required access through APGAS-level channels into this. (Data transfer through channels in one coroutine does not block further execution of other coroutines that use another channel). I am still working on that.

2 Likes