Does heap arrays slow down the code a lot?

1

Deal all,

I am a little confused, under what condition, heap arrays will slow the code by a lot?

I am testing @prince_mahajan nice flint ODE code on my machine.
Using intel OneAPI on windows and gfortran on Linux.
code is here flint_OneAPI · main · Chen Rong / FLINT_Test · GitLab
putting all the f90 in visual studio with Intel OneAPI, you can compile and run.
Or if you are in linux with gfortran, you could just use the Makefile to build and run.

Let us just say in windows, VS + ifort,

The same code, if I always heap arrays, like below with /heap-arrays0, it took more than 5 minutes,

image
image2162×395 17.9 KB

However, if I do not heap arrays (simply leave it blank than setting it to 0), then it only took 10 seconds.

image
image2564×1185 165 KB

Guys, again, may I ask, under what condition, heap arrays will slow the code by a lot?

Thank you very much in advance!

PS.

I did a vTune analysis and found that it looks like allocate and deallocate arrays on the heap perhaps caused the slow down,

image
image2320×1858 314 KB

2

Depending on the size heap arrays, that is, automatic arrays that are taken from the heap rather than the stack, may influence the performance. Reserving memory from the stack is simply cheaper, because it is already there. However, the stack is a very limited resource, especially on Windows, and then you can run into stack overflow problems.
Whether the impact is noticeable depends on the number of times you enter a routine in which heap arrays are used. You might benefit from:

  • Setting a minimum size for heap arrays - then not every automatic array comes from the heap (it is a compiler option)
  • Using the SAVE attribute to reduce the number of times an array is to be allocated, but that means you may have to do some explicit management: allocate or reallocate the array when necessary
1 Like
3

Sometimes, common sense can provide an answer faster than fancy tools such as VTune, etc. Some of the subprograms in your source files have local arrays whose size is not known at compile time, since that size depends on a dummy argument or the size of a dummy argument. Some of these local arrays are declared even within a BLOCK…END BLOCK construct, and such blocks can be inside DO loops. During the execution of the test program, some subroutines/functions are executed over 150 million times. Unless you use a very smart compiler that knows that during the solution of a system of ordinary differential equations the order of the system is fixed, when you use options such as IFort’s /heap-arrays billions of heap allocations and deallocations will be performed, when declaring such arrays with a fixed size (such as 6, for the test problem with order 6) would have sufficed.

Sometimes, the price can be high when features such as OO code or heap arrays are used. Judicious use of heap arrays would occur if you were to select compiler options to let small local arrays (such as real arrays of size 6) be allocated on the stack (or even in the data section), and use heap arrays for large arrays that are allocated and deallocated a moderate number of times during a run of the program.

The short answer to the question that you used as the topic title: It depends.

2 Likes
4

@Arjen @mecej4 Thank you very much indeed! I appreciate that.

For now, I think that I have a quick workaround.

For example, I have two f90 files. A.f90 and B.f90. I need to compile and link them to build exe from them.
A is written by me and with heap arrays0 it works fines.
B is written by others and heap array0 it will slow down.

So, I can just do, when compiling A I add flag /heap arrays0, and when compiling B I do not add this heap arrays0 flag. I mean specify different compile flags for different files.
Do you think this work?

5

Thank you very much! @mecej4
Eh, do you know how to “select compiler options to let small local arrays (such as real arrays of size 6) be allocated on the stack”, like does gfortran has similar options?

Intel fortran has some options like set heap and stack reserve/commit size. But I am not sure it is useful,

image
image2568×1430 173 KB

It looks like the in source code, there are temporary array create constantly.
So, if I do /heap-arrays0, these temp array will be store to memory instead of CPU cache, that can significantly slow down the code.

forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B14FD  MAIN__                    363  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
   60.0104184754309        471.636527263228       -1019.06865347391
   1454.33725487003        1.09976187882064       -1.46242207746315
  -1.35142355895415              829          46          25
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
forrtl: warning (406): fort: (1): In call to DEFUNC, an array temporary was created for argument #3

Image              PC                Routine            Line        Source
FLINT_test.exe     00007FF6354F9ABC  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF6354BD187  ERK_mp_ERK_INT             89  StepSz.f90
FLINT_test.exe     00007FF6354B28CA  MAIN__                    449  test_CR3BP.f90
FLINT_test.exe     00007FF63569AAEE  Unknown               Unknown  Unknown
FLINT_test.exe     00007FF63569AEF8  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFF5D247C24  Unknown               Unknown  Unknown
ntdll.dll          00007FFF5D5AD721  Unknown               Unknown  Unknown
6

Is it possible that when the most convenient for you, you could please give a small example about how to use save to reduce the number of allocate arrays? @Arjen

Thank you very much!

7

@CRquantum Why don’t you give it a try yourself and see what you can come up with? Do it in a separate program, not your big one. See if you can understand when these temporary arrays are being created. Post here when you’ve got some results.

1 Like
8

To add to other comments:
By default;
Automatic arrays usually are created in the stack, while
Allocatable arrays usually are created in the heap
Creating an automatic array on the stack can be quicker than creating an array on the heap.
There is no significant performance difference between use of automatic arrays vs allocatable arrays in most cases ( exceptions can be use of OpenMP and memory<>cache transfers)

So I would target, frequently created small arrays as automatic ( as the stack has limited capacity, unless you change stack size when linking)
Larger arrays to be allocated on the heap.

The unknown is how frequently you are creating arrays; can some large arrays be retained and not re-created?

I did not see if multi-threading is used? for single thread there is little performance difference between heap and stack, but for multi thread, small private arrays can benefit from being on the (individual thread) stack, while for larger arrays that get their own memory pages, there is little difference. (lots of small private arrays on the shared heap can lead to memory coherence conflicts)

There is also a question about if the compile option /heap arrays can work if a non-zero option is used. eg if attempting to allocate automatic arrays larger than 10k to the heap. Size selection is a run time selection and may not be available for arrays of unknown size, as the decision between heap and stack is probably made at compile time.

A possible approach for your problem is to investigate the frequency of defining arrays and to possibly reduce this frequency for larger arrays going to the heap.

2 Likes