# Questions on variable scope in parallel computing

**URL:** <https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824>\
**Category:** Help\
**Created:** [May 31, 2025, 2:27pm UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824 "2025-05-31T14:27:17Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![fish830911](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/fish830911/32/2221_2.png) [@fish830911](https://fortran-lang.discourse.group/u/fish830911)\
**Post date:** [May 31, 2025, 2:27pm UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/1 "2025-05-31T14:27:17Z")

</div>

Dear all,

I recently facing some difficulties in parallelizing a loop using the `powell` methods in the `toolbox.f90` in Fabian Kindermann’s “Introduction to Computational Economics Using Fortran” as attached. I come up with an toy example in the attached `main.f90` to illustrate what I encountered.

In the line 48, the loop I want to parallelize looks like,

```fortran
    !$omp parallel do collapse(2) private(inda, indb, aval, bval, xout, yout)
    do inda = 1, anum
        do indb = 1, bnum
            aval = agrid(inda)
            bval = bgrid(indb)
            call fmintest(aval, bval, xout, yout)
            savemat(indb, inda, :) = [xout, yout]
        end do
    end do

```

If I comment out the `!$omp` line, then the code runs fine; if I uncomment that line, the code runs pretty randomly, sometime it works sometimes it is far.

Let me know what’s wrong with this way of parallelizing the code, and whether there’s a scope issue here. Thank you so much for your attention!

[toolbox.f90](https://fortran-lang.discourse.group/uploads/short-url/zvsEUsC1VnbCqDMUGld3OMxanMi.f90) (331.0 KB)  
[main.f90](https://fortran-lang.discourse.group/uploads/short-url/guNf6w2qmV4inIkBHpASp8I3nIa.f90) (1.6 KB)

---

<div class="post-metadata">

**Author:** ![fxm](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/fxm/32/5726_2.png) [@fxm](https://fortran-lang.discourse.group/u/fxm)\
**Post date:** [May 31, 2025, 3:54pm UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/2 "2025-05-31T15:54:09Z")

</div>

Hi @fish830911, what catches my eye first here is `fmintest` as a potential problem. Is this a pure subroutine (having no side effects)? Also, you want to make sure xout and yout are scalars when used in `savemat(indb, inda, : ) = [xout, yout]`.

---

<div class="post-metadata">

**Author:** ![aledinola](https://avatars.discourse-cdn.com/v4/letter/a/3ec8ea/32.png) [@aledinola](https://fortran-lang.discourse.group/u/aledinola)\
**Post date:** [May 31, 2025, 3:59pm UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/3 "2025-05-31T15:59:11Z")

</div>

Fmintest is just a wrapper that calls fminsearch. The issue here is that the user is calling fminsearch, which is a minimization routine and wants to pass the parameters aval and bval to the function to be minimized.

The code is like a sandwich:  
First the main calls fmintest (user-written) which in turn calls fminsearch (this is a toolbox routine) that in turn calls xyfunc (this is a user-written function).

Problem: how do we pass `aval` and `bval` to `xyfunc`?  
There are two solutions:  
(1) Declare aval and bval as module variables, with the attribute `threadprivate`  
(2) Use an internal function approach, which requires this extra fmintest

There is an example on the fortran-lang callback best practice

Hope this link is useful @fish830911

> **[Type Casting in Callbacks — Fortran Programming Language](https://fortran-lang.org/bn/learn/best_practices/type_casting/)**
>
> Fortran : High-performance parallel programming language

---

<div class="post-metadata">

**Author:** ![PierU](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/pieru/32/1848_2.png) [@PierU](https://fortran-lang.discourse.group/u/PierU)\
**Post date:** [May 31, 2025, 6:18pm UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/4 "2025-05-31T18:18:21Z")

</div>

It’s tricky, because `xyfunc` is not called by its host routine, which means that accessing `aval` and `bval` by host association looks unpredictable to me.

I would even tend to question the conformance of this code from a pure Fortran POV, but I won’t dig into the standard to find out.

---

<div class="post-metadata">

**Author:** ![ivanpribec](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/ivanpribec/32/3290_2.png) [@ivanpribec](https://fortran-lang.discourse.group/u/ivanpribec)\
**Post date:** [May 31, 2025, 6:50pm UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/5 "2025-05-31T18:50:29Z")

</div>

> [@PierU](#):
>
> It’s tricky, because `xyfunc` is not called by its host routine, which means that accessing `aval` and `bval` by host association looks unpredictable to me.

My understanding is the values of `aval` and `bval` remain defined until `fmintest` exits.

The part I think is worth questioning is the implementation of `fminsearch` from the `toolbox` module (a generic interface for the procedure `powell`). The procedure `powell` makes use of several of the private module variables prefixed with `tbox_`:

```auto
! should the random tbox_seed be set
logical, private :: tbox_seed = .true.

! Level of tolerance for all routines
real*8, private :: tbox_gftol = 1d-12

! Maximum number of iterations
integer, private :: tbox_itermax_min = 1000

! Boolean value for showing iterations for optimization
logical, private :: tbox_show_min = .false.

! Maximum number of iterations for brent_pow
integer, parameter, private :: tbox_tbox_itermax_pow_b = 150

! Level of tolerance for all routines
real*8, private :: tbox_gftol_root = 1d-8

! Maximum number of iterations for broydn
integer, private :: itermax_root = 200

! Boolean value for showing iterations for root finding
logical, private :: tbox_show_root = .false.

```

The procedure `powell` only references these values so I assume this should not be a problem.

Another potential source of problems is that `powell` calls `brent_pow` which uses `save`’d variables:

```fortran
        !##########################################################################
        ! FUNCTION brent_pow
        !
        ! Minimizes a one dimensional function.
        !##########################################################################
        function brent_pow(ax, bx, cx, tol, xmin, func)

            implicit none

            !##### INPUT/OUTPUT VARIABLES #########################################

            ! left, middle and right interval points
            real*8, intent(in) :: ax, bx, cx

            ! level of tolerance
            real*8, intent(in) :: tol

            ! minimum value found
            real*8, intent(out) :: xmin

            ! function value at minimum
            real*8 :: brent_pow

            !##### OTHER VARIABLES ################################################

            real*8, parameter :: cgold = 0.3819660d0
            real*8, parameter :: zeps=1.0e-3*epsilon(ax)
            integer :: iter
            real*8 :: a=0d0, b=0d0, d=0d0, e=0d0, etemp=0d0 ! SAVE 
            real*8 :: fu, fv, fw, fx, p, q, r, tol1, tol2, &
                u, v, w, x, xm

```

This means there is a race condition on the variables `a`, `b`, `d`, `e`, and `etemp`. Essentially, the `toolbox` is not written in a thread-safe way.

In general, the `toolbox` code is written a bit sloppily (for instance it uses single precision in literals in several places). I’m also a bit uncertain about skipping the dot in real literals. 👀 The callback function `func` is passed to internal procedures of `powell` as an argument, although it could be accessed through host-association.

---

<div class="post-metadata">

**Author:** ![RonShepard](https://avatars.discourse-cdn.com/v4/letter/r/a3d4f5/32.png) [@RonShepard](https://fortran-lang.discourse.group/u/RonShepard)\
**Post date:** [May 31, 2025, 8:38pm UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/6 "2025-05-31T20:38:22Z")

</div>

> [@ivanpribec](#):
>
> I’m also a bit uncertain about skipping the dot in real literals.

If there is a `d` exponent, then it is double precision even without a decimal point. But your general comments are valid, the nonstandard `*8` notation and `d` exponents should all be replaced with standard notation, preferably with parametrized kind values so that they could be easily changed. There are robust tools to do all of that, and it would only take a matter of a few seconds. Of course making them thread safe requires more than just a simple conversion step.

---

<div class="post-metadata">

**Author:** ![aledinola](https://avatars.discourse-cdn.com/v4/letter/a/3ec8ea/32.png) [@aledinola](https://fortran-lang.discourse.group/u/aledinola)\
**Post date:** [May 31, 2025, 9:03pm UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/7 "2025-05-31T21:03:53Z")

</div>

To fix the race condition, could we replace

```auto
real(8) :: a = 0d0

```

with

```auto
real(8) :: a
a = 0d0

```

and likewise for the other variables?

---

<div class="post-metadata">

**Author:** ![ivanpribec](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/ivanpribec/32/3290_2.png) [@ivanpribec](https://fortran-lang.discourse.group/u/ivanpribec)\
**Post date:** [May 31, 2025, 9:19pm UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/8 "2025-05-31T21:19:19Z")

</div>

That would be my recommendation, yes. Without the initialization, the variables will become [automatic variables.](https://en.wikipedia.org/wiki/Automatic_variable)

(In gfortran, `-fopenmp` implies [`-frecursive`](https://gcc.gnu.org/onlinedocs/gfortran/Code-Gen-Options.html#index-frecursive); this will apply to some other internal procedures which use local arrays.)

In fact, `brent_pow` already sets some of those variables, so one could just add the missing ones

```fortran
            a = min(ax, cx)
            b = max(ax, cx)
            v = bx
            w = v
            x = v
            d = 0.0d0 ! <--- added
            e = 0.0d0
            etemp = 0.0d0 ! <--- added

```

---

<div class="post-metadata">

**Author:** ![fish830911](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/fish830911/32/2221_2.png) [@fish830911](https://fortran-lang.discourse.group/u/fish830911)\
**Post date:** [June 1, 2025, 1:14am UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/9 "2025-06-01T01:14:31Z")

</div>

Dear @ivanpribec,

Thank you so much for spotting this! I applied the change to both the toy example and the project I am working on and now the loop paralleled perfectly.

To avoid myself from making this mistake, could you explain a little bit what does it mean a variable is `save`’d? And why such way of defining variables would create race conditions?

Thank you!

Hui-Jun Chen

---

<div class="post-metadata">

**Author:** ![fish830911](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/fish830911/32/2221_2.png) [@fish830911](https://fortran-lang.discourse.group/u/fish830911)\
**Post date:** [June 1, 2025, 1:33am UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/10 "2025-06-01T01:33:41Z")

</div>

Dear @aledinola

Thank you very much for the link, I will read it after I finish moving!

---

<div class="post-metadata">

**Author:** ![fish830911](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/fish830911/32/2221_2.png) [@fish830911](https://fortran-lang.discourse.group/u/fish830911)\
**Post date:** [June 1, 2025, 1:36am UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/11 "2025-06-01T01:36:05Z")

</div>

Dear @fxm

In this example, `xout` is a vector and `yout` is a scalar. Somehow in `gfortran` it just compiles and runs without an issue.

---

<div class="post-metadata">

**Author:** ![ivanpribec](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/ivanpribec/32/3290_2.png) [@ivanpribec](https://fortran-lang.discourse.group/u/ivanpribec)\
**Post date:** [June 1, 2025, 10:12am UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/12 "2025-06-01T10:12:55Z")

</div>

> [@fish830911](#):
>
> To avoid myself from making this mistake, could you explain a little bit what does it mean a variable is `save`’d? And why such way of defining variables would create race conditions?

For historical reasons, initializing a variable has an implicit save behavior:

```fortran
real(8) :: d = 0.0d0

```

is equal to

```fortran
real(8), save :: d = 0.0d0

```

This is different from a separate declaration and assignment, i.e.

```fortran
real(8) :: d
d = 0.0d0

```

The purpose of the `save` attribute is to save the value between calls of the function, implying the variables is placed in [static storage](https://en.wikipedia.org/wiki/Static_variable) and are persistent for the duration of the program. Another way I personally think of it is like a global variable, but limited to the scope of the function, if that makes sense. Or you could imagine it like a “function with memory”; the value of `d` is remembered across procedure calls.

Now in a multi-threaded program, each thread calling `brent_pow` will be referring to the same `save`’d variables thereby “stepping on each others toes”, i.e. modifying the values that another thread is still using. This is known as a data race. You could solve this issue by putting the function call in a [`single` construct](https://www.openmp.org/spec-html/5.0/openmpsu38.html):

```fortran
    !$omp parallel do collapse(2) private(inda, indb, aval, bval, xout, yout)
    do inda = 1, anum
        do indb = 1, bnum
            aval = agrid(inda)
            bval = bgrid(indb)
            !$omp critical
            call fmintest(aval, bval, xout, yout)
            !$omp end critical
            savemat(indb, inda, :) = [xout, yout]
        end do
    end do

```

but this would force the execution to become sequential (the threads would enter the function exclusively, i.e. one-by-one). Instead, by removing the saved variables, each thread keeps their own copy of `d` and the data-race is resolved.

Edit: The `single` construct, has been replaced with `critical`; this way all threads in the group will call `fmintest` exclusively. Sorry about the error.

The implicit `save` is a very common “gotcha” in Fortran. You can find other explanations at the following links:

- [Fortran assignment on declaration and SAVE attribute gotcha](https://stackoverflow.com/questions/3352741/fortran-assignment-on-declaration-and-save-attribute-gotcha)
- [Why is there an implied SAVE attribute in Fortran?](https://stackoverflow.com/questions/14565739/why-is-there-an-implied-save-attribute-in-fortran)
- [Fortran GOTCHAS: Implied save](https://fortran-lang.org/learn/quickstart/gotchas/#implied-save)

I’ve also shown some examples related to `save` here:

- [SAVE attribute scope - #2 by ivanpribec](https://fortran-lang.discourse.group/t/save-attribute-scope/8509/2)
- [SAVE attribute scope - #11 by ivanpribec](https://fortran-lang.discourse.group/t/save-attribute-scope/8509/11)

---

<div class="post-metadata">

**Author:** ![ivanpribec](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/ivanpribec/32/3290_2.png) [@ivanpribec](https://fortran-lang.discourse.group/u/ivanpribec)\
**Post date:** [June 1, 2025, 10:22am UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/13 "2025-06-01T10:22:01Z")

</div>

Perhaps a fitting saying in this context is

> Too many cooks spoil the broth.

Either you let each commis chef prepare their own broth, or the sous chef upgrades one commis into the potager (soup cook). The analogy being between threads and commis’.

---

<div class="post-metadata">

**Author:** ![themos](https://yyz2.discourse-cdn.com/free1/user_avatar/fortran-lang.discourse.group/themos/32/521_2.png) [@themos](https://fortran-lang.discourse.group/u/themos)\
**Post date:** [June 2, 2025, 11:48am UTC](https://fortran-lang.discourse.group/t/questions-on-variable-scope-in-parallel-computing/9824/14 "2025-06-02T11:48:22Z")

</div>

I find that your typical exposition of Fortran gives attention to some of the pillars of Fortran (the Expression, the Assignment, the DO loop, the Array) but leaves the Association under-explored.