Pitfalls of using chatgpt to generate Fortran code

AI
1

I asked ChatGPT-5 to generate a program to simulate an AR(1) process:

y(t) = rho*y(t-1) + eps(t), where eps(t) is iid Normal(0,sigma2)

The promt was:

Can you provide Fortran code to simulate an AR(1) process of the form y(t)=rho*y(t-1)+e(t), where e(t) is i.i.d. N(0,\sigma^2)?

and I got this code, which is OK but has a trivial mistake and will not compile:

!=============================================================
!  AR(1) Simulation: y_t = rho * y_{t-1} + eps_t
!  eps_t ~ N(0, sigma^2)
!=============================================================
module rng_normal_mod
  use iso_fortran_env, only: real64
  implicit none
  private
  public :: init_rng, randn_vec

contains

  !-------------------------------------------------------------
  subroutine init_rng(seed)
    ! Optional: initialize RNG with a deterministic seed
    implicit none
    integer, intent(in), optional :: seed
    integer :: n, i
    integer, allocatable :: seed_array(:)
    call random_seed(size=n)
    allocate(seed_array(n))
    if (present(seed)) then
       do i = 1, n
          seed_array(i) = seed + 37*i
       end do
    else
       call system_clock(count=i)
       do i = 1, n
          seed_array(i) = i*abs(mod(i*37, 7919))
       end do
    end if
    call random_seed(put=seed_array)
    deallocate(seed_array)
  end subroutine init_rng

  !-------------------------------------------------------------
  subroutine randn_vec(x)
    ! Generate vector x of N(0,1) random numbers using Box–Muller
    implicit none
    real(real64), intent(out) :: x(:)
    integer :: n, i
    real(real64) :: u1, u2, r, theta

    n = size(x)
    do i = 1, n, 2
       call random_number(u1)
       call random_number(u2)
       if (u1 == 0.0_real64) u1 = 1.0e-10_real64
       r     = sqrt(-2.0_real64 * log(u1))
       theta = 2.0_real64 * acos(-1.0_real64) * u2
       x(i)  = r * cos(theta)
       if (i+1 <= n) x(i+1) = r * sin(theta)
    end do
  end subroutine randn_vec

end module rng_normal_mod

!=============================================================
program simulate_ar1
  use iso_fortran_env, only: real64
  use rng_normal_mod
  implicit none

  ! ---- Parameters ----
  integer, parameter :: T = 500
  real(real64), parameter :: rho   = 0.8_real64
  real(real64), parameter :: sigma = 1.0_real64
  real(real64), parameter :: y0    = 0.0_real64
  integer, parameter :: seed = 123456     ! optional fixed seed

  ! ---- Variables ----
  real(real64), dimension(T) :: y, eps
  integer :: t
  integer :: unit

  ! ---- Initialize RNG and generate shocks ----
  call init_rng(seed)
  call randn_vec(eps)
  eps = sigma * eps

  ! ---- Simulate AR(1) ----
  y(1) = rho * y0 + eps(1)
  do t = 2, T
     y(t) = rho * y(t-1) + eps(t)
  end do

  ! ---- Print and save results ----
  print '(A)', 'First 10 simulated y values:'
  print '(10F10.4)', y(1:10)

  open(newunit=unit, file='ar1_vector.csv', status='replace')
  write(unit,'(A)') 't,y,eps'
  do t = 1, T
     write(unit,'(I0,1x,ES20.10,1x,ES20.10)') t, y(t), eps(t)
  end do
  close(unit)

  print *, 'Simulation complete. Results saved to ar1_vector.csv.'
end program simulate_ar1

Of course the bug is that the integer t and T are the same variable. It seems that the AI assumes that Fortran is a case sensitive language like e.g. Matlab

1 Like
2

I have not seen ChatGPT make that mistake, I have seen lots of others including it guessing that Fortran has intrinsics which do not actually exist, how did it respond when you pointed this out? I have been quite impressed with how it appears to learn from its mistakes.

1 Like
3

I gave this follow up:

There is a subtle mistake in your code, can you find it?

Note that I didn’t pass to ChatGPT the compilation error message: I wanted to see if ChatGPT was able to find the mistake without too much help. But chatGPT did not get the error, and instead replied with something that is totally off:

Yes — it’s in init_rng when no seed is provided.

I had to suggest ChatGPT that the mistake was related to the declaration of the variable t. Only then the AI was able to identify the mistake correctly

4

It is correct that the integer t cannot be declared as a parameter and then used as a variable, but the do loop itself is otherwise allowed. This example is legal:

program do
   integer :: t=4
   do t = 1, t
      write(*,*) t
   enddo
end program do

The capitalization of course does not matter. This is allowed because in fortran the trip count is evaluated before the do index is assigned its values within the loop. This kind of variable usage is not allowed in most other languages.

2 Likes
5

The most common mistakes I see the LLMs do with Fortran is “forgetting” that Fortran is not case sensitive and trying to declare variables as if we were in C/C++, i.e.

do i = 1, N
integer :: loop_count 
loop_count = 0
do j = 1, M
 loop_count = loop_count + 1
end do 
end do

I do like enforcing scope so I usually try to use blockstatements. It usually starts making mistakes if I am talking to it about looking at some code for an extended period of time, since I like to see how far I can push it. Something like trying to make a set of loops faster and playing with the bounds, the ordering, etc. at some point I think I just give it a headache.

Otherwise, I have found that ChatGPT and Claude are quite good at doing Fortran.

1 Like
6

In April I gave a list of eight common errors made by LLMs in their Fortran code, which included the issues mentioned here: forgetting that Fortran is case-insensitive and that declarations must precede executable statements unless block is used.

2 Likes
7

I remember, nice post! I think at some point ChatGPT will learn to overcome these “common” mistakes. Slightly surprised that it still makes them