Benchmarking nested do loops, MATMUL and Blas DGEMM

Uncategorized
24

Yes, I think something like that is likely. It is not possible (e.g. using top) to see which core is running a given process. In fact, all of the efficiency cores in MacOS are treated as a group, and processes assigned to the efficiency cores are routinely swapped in and out among those cores. However, there is a taskpolicy command, which (I think) can be used to assign a process to the efficiency core group. This would be used normally to improve energy efficiency, reduce heat dissipation, or to conserve battery power. Here is the output of the code (with the sleep(1) call still within the dgemm() loop).

$ taskpolicy -b  a.out

 Matrix multiplication benchmark
 Matrix size:        1000 x        1000
 Runs:           5

 SYSTEM_CLOCK rate:           1000000000  counts/second

 Nested DO loops
 Run  1:   0.805514 seconds
 Run  2:   0.843484 seconds
 Run  3:   0.855970 seconds
 Run  4:   0.782877 seconds
 Run  5:   0.829418 seconds
 Checksum =     2.5008217646E+08

 Fortran MATMUL
 Run  1:   0.039520 seconds
 Run  2:   0.049923 seconds
 Run  3:   0.042240 seconds
 Run  4:   0.033030 seconds
 Run  5:   0.030708 seconds
 Checksum =     2.5008217646E+08

 BLAS DGEMM
 Run  1:   0.066994 seconds
 Run  2:   0.071541 seconds
 Run  3:   0.071792 seconds
 Run  4:   0.071853 seconds
 Run  5:   0.071933 seconds
 Checksum =     2.5008217646E+08

 ===============================================
 Average wall-clock execution times
 ===============================================
Nested loops :   0.823453 seconds
MATMUL       :   0.039084 seconds
DGEMM        :   0.070823 seconds

MATMUL speedup over loops:     21.069 x
DGEMM speedup over loops :     11.627 x
DGEMM / MATMUL ratio      :      0.552 x

If you compare this to the previous output, you can see that everything has slowed down. Compared to the previous timings, the do-loop timings are now about 8x slower, the matmul/dgemm timings are now some 15x slower, and the sleep+dgemm calls are now about 2x slower. So the sleep(1) call still has a 2x effect, even though everything is now being run (I think) on just the efficiency cores.

25

Here is the code I added for the busy-wait loop:

contains

   subroutine busy_wait(n)
      ! spin for n seconds.
      use, intrinsic :: iso_fortran_env, only: int64
      implicit none
      integer, intent(in) :: n
      integer(int64) :: count, count_rate, target
      call system_clock( count, count_rate )
      target = count + n * count_rate
      do
         call system_clock( count )
         if ( count >= target ) exit
      enddo
      return
   end subroutine busy_wait

Here is the summary output for a normal run:

$ gfortran -O3 -framework accelerate -fexternal-blas matmul_dgemm_benchmark.f90 && a.out
[...]
 ===============================================
 Average wall-clock execution times
 ===============================================
Nested loops :   0.119195 seconds
MATMUL       :   0.002862 seconds
DGEMM        :   0.003758 seconds

MATMUL speedup over loops:     41.644 x
DGEMM speedup over loops :     31.719 x
DGEMM / MATMUL ratio      :      0.762 x

Compared to the previous sleep(1) timings, the dgemm() times are now much faster, as predicted. They are still significantly slower than the matmul/dgemm times. I think this shows that sleep(1) probably does cause some process scheduling or some other type of overhead to occur that the busy_wait(1) does not cause. But the dgemm() times are still significantly longer than the matmul/dgemm times, so that is still evidence that the instruction cache and/or the data cache is being overwritten and must be refilled.

Here is the output for the efficiency core run.

$ taskpolicy -b  a.out
[...]
 ===============================================
 Average wall-clock execution times
 ===============================================
Nested loops :   0.823114 seconds
MATMUL       :   0.042803 seconds
DGEMM        :   0.072645 seconds

MATMUL speedup over loops:     19.230 x
DGEMM speedup over loops :     11.331 x
DGEMM / MATMUL ratio      :      0.589 x

These timings are very similar to the previous efficiency core timings. I’m surprised by that. When run on efficiency cores, there is no difference between sleep(1) and busy_wait(1).