Books about optimization of numerical software?

There are:

  1. Language level optimizations;
  2. Compiler pass optimizations;
  3. Things that basically have to be hand written in assembly (think OpenBLAS).

I would like resources on any of them but specially 2 and 3. I tried search engines but all results are on software for mathematical optimization (operational research).

I was able to find 3 books but they are all from ~2000 and I guess the basics should remain the same but CPUs have changed a bit since then: multi core is much more popular, performance and efficiency cores, vectorization, etc.

  1. Performance Optimization of Numerically Intensive Codes;
  2. Software Optimization for High-Performance Computers;
  3. Software Optimization Cookbook: High-Performance Recipes for the Intel Architecture;
1 Like

Honestly, one of the best resources here is Julia’s source code, libraries, and tutorials.

1 Like

Introduction to High Performance Computing for Scientists and Engineers | CRC Press

Also see the book pages at Georg Hager's Blog | HPC Book

1 Like

Georg Hager and Gerhard Wellein are very nice people, I attended their performance optimization course half a year ago. Since the course material is online accessible, maybe you want to check it out:


to how to write a LIBM in a high level language

I was not able to find this one. Could you link it please? Also will those talks be available on youtube afterwards?

Optimizing Floating Point Math in Julia | JuliaCon 2022 (it’s my talk). All talks are going on to youtube as soon as they go live. Also for the source code, julia/base/special at master · JuliaLang/julia · GitHub has most of the implementations (the code organization isn’t perfect but it’s mostly decent)

1 Like

These books also seem very readable and informative (though I have read only 10 % of each yet…). Both were published in 2021, so I guess they are relatively new.

1 Like

Goedecker, Goove names come to my mind. Two different authors.

1 Like