Changing default shell for "execute_command_line" subroutine

I’m trying to convert some of my old bash scripts to Fortran. To accomplish this, I’m utilizing the execute_command_line subroutine. However, this subroutine uses the sh shell by default on Linux, and I want it to use bash instead. Is there a way to change the default shell used by the execute_command_line subroutine from sh to bash?

Does it work to call bash explicitly? Instead of call execute_command_line("./myscript"), I am thinking call execute_command_line("bash myscript").

Adding some details might get you other solutions. Does your script have a shebang for bash at the top?

Rather call execute_command_line("/bin/bash myscript"), I think

But the sheebang solution looks better

2 Likes

Most shells have a -c option; so you can call “bash -c ‘ls;pwd;date’” for example. In POSIX environments something like GitHub - urbanjost/M_process: read or write to a process from Fortran via a C wrapper might be useful. Note
that on a lot of systems “sh” is a link to “bash” nowadays. Some compilers used to use whatever
the $SHELL environment variable was set to but I think they all now start with essentially “sh”.

It really would be nice if execute_command_line(3f) called the equivalent of popen(c). One advantage of several is that you can change the environment for subsequent commands easily. To do that without creating a file (as already mentioned) ULS systems have the env(1) command, which makes it clear you are setting variables first. This can be a good idea to make sure the commands are using a specific command search path and timezone variable, for example:

  call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c "echo $HOME;ls;pwd")
1 Like

@nshaffer, @PierU, and @urbanjost, thank you for your helpful answers, and I apologize for my delayed reply.

Actually, the exact issue I am facing is in this module: forgnu, specifically in the module_load subroutine. I intend to use Environmental Modules to load the required modules. First, I need to set the module initialization script sourced from the shell configuration startup file:

call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'source /path/to/modules/5.2.0/init/bash'")

In the next step, I want to load the modules:

call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'ml gcc gdb'")

However, it doesn’t work because the initialization is defined in another shell.

If I execute the two commands at once as follows:

call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'source /path/to/modules/5.2.0/init/bash;ml gcc gdb'")

it works, but I want to execute them separately in two steps because my idea is to set the environments and load modules first and then use them.

I’m not sure of your exact objective, but you may create first a bash script file where to put your initializations, and execute this file later. Something like

open(newunit=lu,file="/anotherpath/to/myscript.bash")
write(lu,*) "#!/bin/bash"
write(lu,*) "TZ=0"
write(lu,*) "source /path/to/modules/5.2.0/init/bash"
close(lu)

and later

call execute_command_line("env PATH=/bin:/usr/bin; source /anotherpath/to/myscript.bash; ml gcc gdb'")
1 Like

I am currently in the process of converting bash script files to Fortran in order to execute commands step by step directly within the Fortran code, without the need for generating a separate bash file. To achieve this, I am utilizing the execute_command_line subroutine.

To execute the commands directly in Fortran, I follow these steps. However, it doesn’t work!

  1. Set the environmental modules using the following command:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'source /path/to/modules/5.2.0/init/bash'")
  1. Load the required modules by executing the following command:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'ml gcc gdb'")
  1. As an example, use gcc by calling the following command:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'gcc --version'")

The issue is that every time I want to use a specific command, such as gcc, I need to perform all three steps together at once:

call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'source /path/to/modules/5.2.0/init/bash;ml gcc gdb;gcc --version'")

Each time you call execute_command_line() you fork your current process to create a new child process. So the three commands are executed in three different processes.

The following code:

  ! Show the pid of the processes:
  call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'echo $$'")
  call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'echo $$'")
  call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'echo $$'")
  call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'echo $$'")
  end

displays:

5680
5682
5684
5686
1 Like

I think it is possible to create a daemon shell script and to communicate with it. Maybe it could be a solution to your problem. But I can not help further…

1 Like

This points directly to the problem I am currently facing.

I believe it is safer to use

write(lu,*) "#!/bin/env bash"

instead of

write(lu,*) "#!/bin/bash"

This is because it won’t work otherwise, at least on some GNU/Linux systems, and definitely not in (Free)BSD.

Should be #!/usr/bin/env bash, at least on Unix.

That’s correct. Original post used #!/bin/bash which will work in many GNU/Linux distributions (again, not all,) but won’t work at all in BSDs. I didn’t want to deviate much from the original, but the truth is #!/usr/bin/env bash should work everywhere - except perhaps the ones that don’t follow FHS (Filesystem Hierarchy Standard.)

Here are just some general comments about this issue.

First /bin/sh is now a link to /bin/bash on many systems. The shell looks at how it was invoked and behaves appropriately, but it is the same executable in both cases, it is just that the sh commands are emulated within bash. On MacOS, I think /bin/sh is now emulated within /bin/zsh, and that works the same way.

As you have determined, the problem with your approach is that the different invocations of execute_command_line() all work in different child processes, so you cannot execute a string of commands that all modify or operate within the same environment. This is not a special case for fortran, all shells and languages work the same way on a POSIX or unix-like system.

In POSIX systems, a child process inherits the environment of the parent. So if you modify the environment of the parent process, any subsequent child processes will start with those modifications. Fortran provides a get_environment_variable() intrinsic to query the current environment, but it does not provide an analogous set_environment_variable() to modify it. I don’t know why that function is missing in fortran, but it is.

So the workaround is to use the POSIX setenv() system call. Actually, all unix-like operating systems support this (going back to the 1980s), even if they are not POSIX compliant. You need to set up the appropriate fortran interface to this operating system function using C interop in order to invoke it directly from fortran. You do not need to write a C interface function, you can go directly from fortran to the OS call.

Once the environment has been modified, you can test if the change is correct with the fortran get_environment_variable() intrinsic. When that all works, then you can invoke separate execute_command_line() calls, and they will all begin with that modified environment.

Maybe someone can fill i some details if necessary, or correct me if I’m wrong about any of the above, but that is one possible way forward to solve your problem.

1 Like

That is a work-around for setting the variables. There are other alternatives

If you only need to run on ULS that have popen(3c) you can start a process and write multiple lines to it. There are examples on the Fortran Wiki or see GitHub - urbanjost/M_process: read or write to a process from Fortran via a C wrapper for another example.

You can create a FIFO file either with a system command or via a call to the appropriate C routines and start a shell that reads from it; and the write to it as needed.

You can gather up the lines instead of executing them and write them into a file and then execute all at once if the commands do not need executed at various times; as was mentioned above.

On machines that support it popen(3c) is a very nice way to go. There are equivalents on MSWindows to popen(3c) that you can use the same way, but I have not used them in ages.

2 Likes

FIFO files are highly underutilized in my opinion, and are easy to use from all the
Unix/GNU-Linux systems as far as I know. MSWindows has FIFO files but this
example only works on ULS but shows how a Fortran program (or pretty
much any language that can write to a file) can open a shell and leave
it open and then write to it like it was pretty much any other file:

Fortran Code
program main
   ! example of using FIFO files instead of calling POPEN(3c)
   implicit none
   character(len=*), parameter :: g = '(*(g0))'
   integer                     :: lun, iostat
   !
   ! setup
   !
   ! assumes GNU commands rm, mkfifo, fuser, bash are available
   call run("rm -f __unique_name")
   call run("mkfifo __unique_name")
   ! start shell in background
   call run("bash -v < __unique_name 2>&1", wait=.false.)
   ! open fifo file for writing
   open (newunit=lun, file="__unique_name", action="write")
   !
   ! write commands to bash shell
   !
   call cmd('export PATH=/bin/:/usr/bin TZ=0',iostat)
   call cmd('uname -a;date;whoami',iostat)
   call cmd('source /path/to/modules/5.2.0/init/bash',iostat)
   call cmd('echo $HOME;pwd',iostat)
   call cmd('ml gcc gdb',iostat)
   call cmd('gcc --version',iostat)
   call cmd('cd /tmp',iostat)
   call cmd([character(len=256) :: 'pwd','echo note still in /tmp','printenv TZ;echo $PATH'])
   ! teardown
   close (unit=lun, iostat=iostat)
   call run("fuser -k __unique_name;rm -f __unique_name")

contains
   elemental impure subroutine run(string,wait)
   ! call system command
      character(len=*), intent(in)    :: string
      logical,intent(in),optional     :: wait
      integer                         :: exitstat, cmdstat, iostat
      character(len=256)              :: cmdmsg
      logical                         :: wait_
      if(present(wait))then; wait_=wait; else; wait_=.true.; endif
      call execute_command_line(string, wait=wait_,exitstat=exitstat,cmdstat=cmdstat,cmdmsg=cmdmsg)
      if (exitstat .ne. 0) write (*, g) '< ERROR:RUN: > ', trim(cmdmsg)
   end subroutine run

   elemental impure subroutine cmd(string,iostat)
   ! write command to shell
      character(len=*), intent(in) :: string
      integer,intent(out),optional :: iostat
      integer                      :: iostat_
      character(len=256)           :: iomsg
      write (lun, g, iostat=iostat_, iomsg=iomsg) string
      if (iostat_ .ne. 0) write (*, g) '< ERROR:CMD: > ', trim(iomsg)
      if(present(iostat))iostat=iostat_
   end subroutine cmd

end program main

Just always remember to create your FIFO file in a private directory, as if you open it without correct permission modes other people that can write to the file can run as you; sort of a do-it-yourself su(1).’

As an experiment, this should load the code into the Fortran playground:

playground

3 Likes

@urbanjost , thank you for the great suggestion and the provided example. It works perfectly!
I will test it out and get back to you with the results.