Actually there are instances where FEM codes have used GPUs, usually not in single precision though. Most GPUs (even the ones in commodity graphics cards) can do FP64. Unfortunately, both NVIDIA and AMD throttle back FP64 performance to 1/16 to 1/32 FP32 performance so a 600 dollar card will not compete with a $10,000 to $15,000 Tesla A100 or Instinct MI250 which can run FP64 at either 1/2 or the same speed as FP32. In most cases they don’t even advertise FP64 performance and you have to enable it in the GPU kernel but its there. I’m working with a Fortran based FEM code designed for impact and penetration problems (and has been around for about 40 years now) that has $acc directives all over the code base.
Also, the issue with FEM and GPUs has as much to do with the fact that explicit algorithms don’t map well to GPUs (as compared to implicit ones) as it does with the precision you are using (the major reason LS-DYNA does not support GPUs for their explcit solver). GPU’s are great if you can recast most of your computational work as matrix-vector or matrx-matrix operations. If you can’t do that the overhead of moving data back and forth between the GPU and CPU can limit any performance gain you might get.