I am working with a code where do concurrent is used quite a bit for GPU acceleration. One interesting issue is that not everything needs to be necessarily done on the GPU, certain things would be nice to keep on the host but be able to parallelize on the host. Say initializing some arrays that will also have a host value, or something akin to that.
With do concurrent things will get offloaded to the GPU or stay on the CPU but there is now way to tell the runtime/compiler if something is to always be executed on the host/device without using openacc or openmp pragmas. Since we already have do concurrent it might be nice to have something that tells the runtime if we want it to be offloaded to communicate to the compiler backend. This, I hope, will make life easier for nvidia, amd, and intel and anyone that wants to accelerate do concurrent.
I had asked Mr. Claude for some opinions and it put together an interesting proposal. A keyword for do concurrent that ensures where the data is, something like do concurrent (..) resident(var1,var2) that tells the compiler that the DC loop will only access data from where it already is. Probably checking some pointer addresses, whatever. The full proposal written by Claude is here, I did not change anything Claude generated because I thought it is interesting to see what it proposed. I iterated a bit with it, talking about pointers, dervied types, etc. but I probably wasn’t exhaustive enough. I think this might be applicable and extendable to any separate memory accelerators, maybe FPGAs!
This mostly stems from having to do some things on the host (for convenience, or just not worth offloading) but they can be run in parallel. do concurrent is super nice to say that but it has its limitations in systems with separate memory address spaces.
I also thought Claude put together something very nice though. It is quite coherent.
DO CONCURRENT: locality specifier to keep operands co-resident (no implicit offload copy)
Motivation
DO CONCURRENT asserts that loop iterations are independent. On implementations that offload (e.g. nvfortran -stdpar=gpu, ifx -fopenmp-target-do-concurrent), that single assertion is also taken as permission to execute the loop against a copy of its operands placed in a separate memory. The offload decision is translation-unit-wide: there is no way, in standard Fortran, to keep one DO CONCURRENT loop parallel-on-host while another is offloaded.
This conflates two distinct properties:
- the iterations are independent, and
- the implementation may relocate/duplicate the operands into a distinct memory to run them.
There is a legitimate need to keep (1) while withdrawing (2) — for example, cheap initialization loops whose arrays are also used on the host, where offloading forces an unwanted migration and is slower than staying put. Today the only workarounds are splitting translation units by -stdpar mode, or dropping DO CONCURRENT for !$omp parallel do, which leaves standard, single-source Fortran.
Proposed solution
Add an opt-in locality specifier to the DO CONCURRENT header that asserts the loop executes against the storage its operands already occupy — the implementation may still parallelize the loop, but may not satisfy it by introducing a copy of those operands in a distinct memory.
Placeholder keyword: RESIDENT (bikeshed later — COLOCATED, IN_PLACE, etc.).
do concurrent (i = 1:n) local(tmp) resident(a, b)
tmp = b(i) * scale
a(i) = tmp
end do
Semantics
The specifier defines a simple, monotonic set of states:
- Unmarked — no constraint on operand location. The implementation may run the loop in place, on host threads, or by copying operands into a distinct memory and offloading, whichever it judges best. This is exactly today’s behavior.
RESIDENT(...)— for each named variable, the loop shall access that variable in the storage it already occupies; the implementation shall not, as a means of executing the loop, create a copy of the variable in physical memory distinct from that storage.
There is deliberately no third keyword to force offload. The permission to offload already exists — it is precisely what an unmarked DO CONCURRENT grants — so “offload this loop” is just the absence of the constraint together with an implementation that chooses to. RESIDENT only withdraws that permission for the named operands; it introduces no new freedom.
The actor throughout is the implementation, which subsumes both compiler and runtime; whether the placement choice is made at compile time or run time is not constrained.
The specifier constrains operand placement only. Iteration independence and the existing localization rules are unchanged, and it is intended to compose with DEFAULT(NONE).
Note this is deliberately phrased around access / no implicit copy, not around “same address space.” A single-address-space guarantee is too weak: managed/unified-virtual-address memory satisfies it while still physically migrating pages on access — the exact movement this feature exists to prevent. The no-implicit-copy formulation forbids both the discrete-memory copy and the managed-memory migration, and never needs the standard to define “address space.”
The words host, device, GPU, and accelerator do not appear. The desired behavior (parallel, on-host, no migration) arises as a consequence on hardware with distinct memories, not as a named target.
Indirection and derived types
For an entity reached through a descriptor or handle, the constraint applies to the storage that entity designates at the point of execution: for a pointer, its target; for an allocatable, its allocated storage. An unassociated pointer or unallocated allocatable designates no storage and the specifier is a no-op for it.
The constraint is transitive. For a derived-type entity, it follows POINTER and ALLOCATABLE components down the reachable graph, so that naming an aggregate pins the storage of its dynamically-sized components as well. This is the same reachability that OpenACC and OpenMP must already traverse for correctness: a shallow map of a derived type with allocatable components produces a valid top-level object whose components still designate host storage, which is why both models perform a deep copy — allocating device mirrors for the components and patching the on-device descriptors to point at them (attach/detach). A non-transitive RESIDENT would reintroduce exactly that shallow-map bug (a pinned aggregate designating data that moved), so transitivity is required for the specifier to be consistent, not merely convenient.
Crucially, RESIDENT is the prohibition side of deep copy, which is why the transitive scope carries almost no implementation cost. Deep copy is hard because it must walk the type, allocate mirrors, transfer data, and fix up descriptors. RESIDENT does none of that — it only forbids that walk-and-mirror from being performed for the named entity. The scope is the whole reachable graph; the obligation is to leave it alone.
The one genuine wrinkle is POINTER components: reachability through a pointer is a runtime property (what it is associated with when the loop executes), so the prohibition is well-defined but its extent is only known dynamically. ALLOCATABLE components do not have this issue — their storage identity is fixed while allocated.
Behavior across memory architectures
- Distinct physical memories (discrete GPU): offload requires a copy, which is forbidden; the loop runs with host parallelism. This is the motivating case.
- Managed / unified virtual memory: a migration is an implicit copy, so it too is forbidden; the loop stays put.
- Physically unified/coherent memory: no distinct memory exists, so the constraint is vacuously satisfied and the implementation is free to run the loop anywhere. This is correct — the reason to pin (avoiding migration) does not exist there.
- CPU-only: trivially satisfied; no effect.
This is the key advantage over an explicit OFFLOADABLE/NON_OFFLOADABLE flag: a device flag is wrong on unified memory, whereas a co-residency assertion is automatically right everywhere and its meaning simply evaporates when the hardware makes it moot.
Backwards compatibility
Purely additive and opt-in. Unmarked DO CONCURRENT retains today’s behavior exactly, and users annotate the exceptions rather than the common path.
The feature adds no new default behavior and changes no existing program. Every conforming implementation already gets to decide operand placement for unmarked loops; this proposal leaves that untouched and only gives the programmer a way to say “not this one.” Because it withdraws a permission rather than granting a new mode, it cannot alter the meaning of any existing code and is provably zero-cost to adopt.
Prior art
- F2018 locality specifiers (
LOCAL,LOCAL_INIT,SHARED,DEFAULT) and F2023REDUCE— the established mechanism for extending theDO CONCURRENTheader with data-semantic clauses. - C++ execution policies (
seq,par,par_unseq): parallelism expressed as permission/policy, with device mapping left to the implementation and never named in the language. - OpenACC/OpenMP deep copy (attach/detach): the established transitive-reachability semantics for derived types with allocatable/pointer components.
RESIDENTis the prohibition side of the same traversal.
Open questions
- Keyword.
RESIDENTis a placeholder. - Pointer-component extent. Transitivity through
POINTERcomponents makes the constraint’s extent a runtime property (the association at execution). The prohibition is well-defined; how, if at all, to characterize the extent statically is open.ALLOCATABLEcomponents are unaffected. - Interaction with localization. Kept strictly orthogonal here; should it be required to appear with
DEFAULT(NONE)to avoid stacking a placement guarantee on ambiguous data semantics?