It is well known that
-ffast-math in GFortran (and other similar options in other compilers) that do not follow the IEEE standard exactly can change numerical results, including even making some codes to produce completely incorrect results, the most famous example is probably the Kahan summation algorithm.
-ffast-math always safe?
A: No, it can make your code return wrong answers, such as the Kahan algorithm.
Q: Can one design rules to follow in your code, so that
-ffast-math is safe, and define exactly what you mean by “safe”?
A: I think the answer might be yes, and that is what this post is about. I have made some progress towards this, but need help getting it over the finish line. If you are interested to help out, you can do so below.
Q: What is the point of trying to design such rules if we can simply turn off “-ffast-math” and be safe?
A: Turning off “-ffast-math” is definitely a valid approach. However, often the code might not be as performing. One can recover performance by being intentional about optimizations, for example by using “fast-math” to discover which optimizations might help, and then applying them explicitly, and testing the code that it still works with them (or applying such optimizations “by hand” in the source code itself). The alternative approach being proposed here is to program with a set of rules, that then give the compiler greater freedom to rearrange the computation to improve performance.
- Such “fast-math” rules, if they exist, must say that Kahan algorithm does not satisfy them (and thus such code might not work with “fast-math”).
- The rules are for the Fortran programmer to follow to ensure enabling “fast-math” in the compiler will still produce correct results; as well as for compilers to know what they can and cannot do. The idea of these rules is that they give compilers more freedom to rearrange the computation.
About 4 years ago I wrote this document:
That tries to come up with such rules and then shows on many examples how to apply such rules. Let me know what you think and if you have ideas how to make the rules more rigorous.
In particular, I am aware that when I say “well conditioned” in the document, it might not always be exactly equal to the condition number from numerical analysis, although it seems related.