Problems with coarrays

But the issue here is that CAF does NOT push to other images reliably.

If I understand you correctly, then see that document on page 26 for coarray get and put operations:
https://doku.lrz.de/download/attachments/43320950/Advanced_Fortran_Coarrays.pdf .

From my own experiences, ‘unreliable’ data transfer through coarray put operations are common if the SPMD coarray runtime (i.e. the runtime of the current coarray team) got ‘destroyed’ by violating the execution segment ordering in that coarray team, i.e. some programming mistake with a synchronization or atomic operation. Any following data transfer or synchronization may or may not succeed, the ‘destroyed’ runtime becomes unreliable.

The only way, that I am aware of, to recover from a ‘destroyed’ coarray runtime is to leave the current coarray team (END TEAM) and then to re-enter (CHANGE TEAM) the coarray team, if this is still possible. Of course this makes no sense if the execution segment ordering is violated again and again, it should not be violated at all.