I’m trying to convert some of my old bash scripts to Fortran. To accomplish this, I’m utilizing the execute_command_line
subroutine. However, this subroutine uses the sh
shell by default on Linux, and I want it to use bash
instead. Is there a way to change the default shell used by the execute_command_line
subroutine from sh
to bash
?
Does it work to call bash explicitly? Instead of call execute_command_line("./myscript")
, I am thinking call execute_command_line("bash myscript")
.
Adding some details might get you other solutions. Does your script have a shebang for bash at the top?
Rather call execute_command_line("/bin/bash myscript")
, I think
But the sheebang solution looks better
Most shells have a -c option; so you can call “bash -c ‘ls;pwd;date’” for example. In POSIX environments something like GitHub - urbanjost/M_process: read or write to a process from Fortran via a C wrapper might be useful. Note
that on a lot of systems “sh” is a link to “bash” nowadays. Some compilers used to use whatever
the $SHELL environment variable was set to but I think they all now start with essentially “sh”.
It really would be nice if execute_command_line(3f) called the equivalent of popen(c). One advantage of several is that you can change the environment for subsequent commands easily. To do that without creating a file (as already mentioned) ULS systems have the env(1) command, which makes it clear you are setting variables first. This can be a good idea to make sure the commands are using a specific command search path and timezone variable, for example:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c "echo $HOME;ls;pwd")
@nshaffer, @PierU, and @urbanjost, thank you for your helpful answers, and I apologize for my delayed reply.
Actually, the exact issue I am facing is in this module: forgnu, specifically in the module_load
subroutine. I intend to use Environmental Modules to load the required modules. First, I need to set the module initialization script sourced from the shell configuration startup file:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'source /path/to/modules/5.2.0/init/bash'")
In the next step, I want to load the modules:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'ml gcc gdb'")
However, it doesn’t work because the initialization is defined in another shell.
If I execute the two commands at once as follows:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'source /path/to/modules/5.2.0/init/bash;ml gcc gdb'")
it works, but I want to execute them separately in two steps because my idea is to set the environments and load modules first and then use them.
I’m not sure of your exact objective, but you may create first a bash script file where to put your initializations, and execute this file later. Something like
open(newunit=lu,file="/anotherpath/to/myscript.bash")
write(lu,*) "#!/bin/bash"
write(lu,*) "TZ=0"
write(lu,*) "source /path/to/modules/5.2.0/init/bash"
close(lu)
and later
call execute_command_line("env PATH=/bin:/usr/bin; source /anotherpath/to/myscript.bash; ml gcc gdb'")
I am currently in the process of converting bash script files to Fortran in order to execute commands step by step directly within the Fortran code, without the need for generating a separate bash file. To achieve this, I am utilizing the execute_command_line
subroutine.
To execute the commands directly in Fortran, I follow these steps. However, it doesn’t work!
- Set the environmental modules using the following command:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'source /path/to/modules/5.2.0/init/bash'")
- Load the required modules by executing the following command:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'ml gcc gdb'")
- As an example, use gcc by calling the following command:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'gcc --version'")
The issue is that every time I want to use a specific command, such as gcc, I need to perform all three steps together at once:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'source /path/to/modules/5.2.0/init/bash;ml gcc gdb;gcc --version'")
Each time you call execute_command_line()
you fork your current process to create a new child process. So the three commands are executed in three different processes.
The following code:
! Show the pid of the processes:
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'echo $$'")
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'echo $$'")
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'echo $$'")
call execute_command_line("env PATH=/bin:/usr/bin TZ=0 /bin/bash -c 'echo $$'")
end
displays:
5680
5682
5684
5686
I think it is possible to create a daemon shell script and to communicate with it. Maybe it could be a solution to your problem. But I can not help further…
This points directly to the problem I am currently facing.
I believe it is safer to use
write(lu,*) "#!/bin/env bash"
instead of
write(lu,*) "#!/bin/bash"
This is because it won’t work otherwise, at least on some GNU/Linux systems, and definitely not in (Free)BSD.
Should be #!/usr/bin/env bash
, at least on Unix.
That’s correct. Original post used #!/bin/bash
which will work in many GNU/Linux distributions (again, not all,) but won’t work at all in BSDs. I didn’t want to deviate much from the original, but the truth is #!/usr/bin/env bash
should work everywhere - except perhaps the ones that don’t follow FHS (Filesystem Hierarchy Standard.)
Here are just some general comments about this issue.
First /bin/sh is now a link to /bin/bash on many systems. The shell looks at how it was invoked and behaves appropriately, but it is the same executable in both cases, it is just that the sh commands are emulated within bash. On MacOS, I think /bin/sh is now emulated within /bin/zsh, and that works the same way.
As you have determined, the problem with your approach is that the different invocations of execute_command_line()
all work in different child processes, so you cannot execute a string of commands that all modify or operate within the same environment. This is not a special case for fortran, all shells and languages work the same way on a POSIX or unix-like system.
In POSIX systems, a child process inherits the environment of the parent. So if you modify the environment of the parent process, any subsequent child processes will start with those modifications. Fortran provides a get_environment_variable()
intrinsic to query the current environment, but it does not provide an analogous set_environment_variable()
to modify it. I don’t know why that function is missing in fortran, but it is.
So the workaround is to use the POSIX setenv()
system call. Actually, all unix-like operating systems support this (going back to the 1980s), even if they are not POSIX compliant. You need to set up the appropriate fortran interface to this operating system function using C interop in order to invoke it directly from fortran. You do not need to write a C interface function, you can go directly from fortran to the OS call.
Once the environment has been modified, you can test if the change is correct with the fortran get_environment_variable()
intrinsic. When that all works, then you can invoke separate execute_command_line()
calls, and they will all begin with that modified environment.
Maybe someone can fill i some details if necessary, or correct me if I’m wrong about any of the above, but that is one possible way forward to solve your problem.
That is a work-around for setting the variables. There are other alternatives
If you only need to run on ULS that have popen(3c) you can start a process and write multiple lines to it. There are examples on the Fortran Wiki or see GitHub - urbanjost/M_process: read or write to a process from Fortran via a C wrapper for another example.
You can create a FIFO file either with a system command or via a call to the appropriate C routines and start a shell that reads from it; and the write to it as needed.
You can gather up the lines instead of executing them and write them into a file and then execute all at once if the commands do not need executed at various times; as was mentioned above.
On machines that support it popen(3c) is a very nice way to go. There are equivalents on MSWindows to popen(3c) that you can use the same way, but I have not used them in ages.
FIFO files are highly underutilized in my opinion, and are easy to use from all the
Unix/GNU-Linux systems as far as I know. MSWindows has FIFO files but this
example only works on ULS but shows how a Fortran program (or pretty
much any language that can write to a file) can open a shell and leave
it open and then write to it like it was pretty much any other file:
Fortran Code
program main
! example of using FIFO files instead of calling POPEN(3c)
implicit none
character(len=*), parameter :: g = '(*(g0))'
integer :: lun, iostat
!
! setup
!
! assumes GNU commands rm, mkfifo, fuser, bash are available
call run("rm -f __unique_name")
call run("mkfifo __unique_name")
! start shell in background
call run("bash -v < __unique_name 2>&1", wait=.false.)
! open fifo file for writing
open (newunit=lun, file="__unique_name", action="write")
!
! write commands to bash shell
!
call cmd('export PATH=/bin/:/usr/bin TZ=0',iostat)
call cmd('uname -a;date;whoami',iostat)
call cmd('source /path/to/modules/5.2.0/init/bash',iostat)
call cmd('echo $HOME;pwd',iostat)
call cmd('ml gcc gdb',iostat)
call cmd('gcc --version',iostat)
call cmd('cd /tmp',iostat)
call cmd([character(len=256) :: 'pwd','echo note still in /tmp','printenv TZ;echo $PATH'])
! teardown
close (unit=lun, iostat=iostat)
call run("fuser -k __unique_name;rm -f __unique_name")
contains
elemental impure subroutine run(string,wait)
! call system command
character(len=*), intent(in) :: string
logical,intent(in),optional :: wait
integer :: exitstat, cmdstat, iostat
character(len=256) :: cmdmsg
logical :: wait_
if(present(wait))then; wait_=wait; else; wait_=.true.; endif
call execute_command_line(string, wait=wait_,exitstat=exitstat,cmdstat=cmdstat,cmdmsg=cmdmsg)
if (exitstat .ne. 0) write (*, g) '< ERROR:RUN: > ', trim(cmdmsg)
end subroutine run
elemental impure subroutine cmd(string,iostat)
! write command to shell
character(len=*), intent(in) :: string
integer,intent(out),optional :: iostat
integer :: iostat_
character(len=256) :: iomsg
write (lun, g, iostat=iostat_, iomsg=iomsg) string
if (iostat_ .ne. 0) write (*, g) '< ERROR:CMD: > ', trim(iomsg)
if(present(iostat))iostat=iostat_
end subroutine cmd
end program main
Just always remember to create your FIFO file in a private directory, as if you open it without correct permission modes other people that can write to the file can run as you; sort of a do-it-yourself su(1).’
As an experiment, this should load the code into the Fortran playground:
@urbanjost , thank you for the great suggestion and the provided example. It works perfectly!
I will test it out and get back to you with the results.