Intel Fortran Compiler interfaced in Pycharm

Help
1

Hello, I am trying to wrap some fortran code in the Pycharm IDE using f2py, but I do not know how to link it to the Intel fortran compiler to compile the code in Pycharm. I have tried a lot of different terminal commands through Pycharm, most recently being "f2py -c --fcompiler=intelvem -L “C:
\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\bin\intel64"”, but I keep getting errors about it not knowing how to compile. Any idea how to link the compiler so I can use it through Pycharm?

Thanks!

2

Hi @abm000, welcome to the forum!

I’m not an f2py expert by any means, but your commands looks a bit wrong to me. It should be something like this: f2py -c [<options>, <build_flib options>, <extra options>] <fortran files>. So if you want to compile with Intel, something like…

f2py -c --fcompiler=intelvem /path/to/fortran/file.f90

should work. Can you paste the error message(s) you get here so we can best figure out what the problem is? Is this specific to using the PyCharm terminal or do you get the same error when using other terminals (Command Prompt, PowerShell, etc…)?

3

When running the command you suggested in the terminal within Pycharm, this error was given:

        Wrote C/API module "untitled" to file "C:\Users\user\AppData\Local\Temp\tmp7ivuy_gi\src.win-amd64-3.9\untitledmodule.c"       
        Fortran 90 wrappers are saved to "C:\Users\user\AppData\Local\Temp\tmp7ivuy_gi\src.win-amd64-3.9\untitled-f2pywrappers2.f90"  
  adding 'C:\Users\user\AppData\Local\Temp\tmp7ivuy_gi\src.win-amd64-3.9\fortranobject.c' to sources.
  adding 'C:\Users\user\AppData\Local\Temp\tmp7ivuy_gi\src.win-amd64-3.9' to include_dirs.
copying c:\users\user\project\venv\lib\site-packages\numpy\f2py\src\fortranobject.c -> C:\Users\user\AppData\Local\Temp\tmp7ivuy_gi\src.win-amd64-3.9
copying c:\users\user\project\venv\lib\site-packages\numpy\f2py\src\fortranobject.h -> C:\Users\user\AppData\Local\Temp\tmp7ivuy_gi\src.win-amd64-3.9
  adding 'C:\Users\user\AppData\Local\Temp\tmp7ivuy_gi\src.win-amd64-3.9\untitled-f2pywrappers2.f90' to sources.
build_src: building npy-pkg config files
running build_ext
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
customize MSVCCompiler using build_ext
customize IntelEM64VisualFCompiler
Could not locate executable ifort
Could not locate executable ifl
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\project\venv\Scripts\f2py.exe\__main__.py", line 7, in <module>
  File "c:\users\user\project\venv\lib\site-packages\numpy\f2py\f2py2e.py", line 690, in main
    run_compile()
  File "c:\users\user\project\venv\lib\site-packages\numpy\f2py\f2py2e.py", line 657, in run_compile
    setup(ext_modules=[ext])
  File "c:\users\user\project\venv\lib\site-packages\numpy\distutils\core.py", line 169, in setup       
    return old_setup(**new_attr)
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\distutils\dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "c:\users\user\project\venv\lib\site-packages\numpy\distutils\command\build.py", line 61, in run 
n customize
    self.find_executables()
  File "c:\users\user\project\venv\lib\site-packages\numpy\distutils\fcompiler\__init__.py", line 346, in find_executables
    raise CompilerNotFound('%s: f90 nor f77' % ctype)
numpy.distutils.fcompiler.CompilerNotFound: intelvem: f90 nor f77

I also tried running the same command with --fcompiler=intel which resulted in the same error. If it helps, I have the Intel oneAPI base toolkit and HPC toolkit installed.

When running in the windows command prompt, it tells me that f2py is an unrecognized command.

Let me know what you think, thanks!

4

Looks like f2py can’t find the Intel Fortran compiler. I don’t know enough about f2py to know how it internally searches for compilers (perhaps it just checks your PATH?), but it looks like there is an option to explictly set the path to the compiler, --f90exec=<path>, so I would give that a go. E.g. if the path to the compiler is the path you specified in your first post, then something like this might work:

f2py -c --fcompiler=intelvem --f90exec="C:
\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\bin\intel64" /path/to/fortran/file.f90
5

I know it is not entirely related, but is there any chance you know how to find the path to the compiler? I do not know what file name or type I am looking for. I am also not sure if the intel oneAPI compiler is compatible with f2py since it must be f90 or f77.

6

It might be worth checking out these docs: oneAPI Development Environment Setup. That says that on Windows the compiler is usually installed in a subfolder of C:\Program Files (x86)\Intel\oneAPI\, so I think the path in my previous post is probably quite close.

I haven’t got any oneAPI compilers installed at the moment, so I might be wrong, but… I think when you install it on Windows, you get a Start menu shortcut for a oneAPI command prompt. Open that and run where ifort or where ifx (depending on what compiler you’ve got installed / hope to use), and that should give you the exact location of that compiler.

7

I’m not familiar with the Pycharm environment at all, but this part of the error message you posted from f2py:

...
Could not locate executable ifort
Could not locate executable ifl
...

definitely indicates that f2py was not able to locate the Intel Fortran compilers. The easiest way to ensure that the compilers are in your path, is to run something like C:\Program Files (x86)\Intel\oneAPI\setvars.bat in the active terminal (or ensure that is always done in any session where you want to use ifort). Now, typing e.g. where ifort should give you the path to the executable, and running f2py something like:

f2py -c --fcompiler=intelvem --compiler=msvc /path/to/fortran/file.f90

should work without the need to specify any (compiler) paths explicitly, fingers crossed…

1 Like
8

Ok sorry for the delay, I’ve been messing with this thing. I tried switching my project over to Visual Studio because my Intel oneAPI command prompt is titled “Intel oneAPI command prompt for Intel 64 for Visual Studio 2019”, but this ended up being a hassle so I gave up quite early with that. However, on Pycharm, after running where ifort and where ifx and trying the command

f2py -c --fcompiler=intel --f90exec="C:\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\bin\intel64\ifort.exe" \path\to\fortran\file.f90

with both executable locations, I still received the error

Found executable C:\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\bin\intel64\ifort.exe
Could not locate executable ifort
Could not locate executable ifc

… I just gave it the direct path I do not understand!!! It also still tells me

numpy.distutils.fcompiler.CompilerNotFound: intelvem: f90 nor f77

I’m thinking that it might have something to do with my Windows or Pycharm path variables, but that’s just a hunch and the only reason I can think that this would still be occuring.

Edit: Another though occurred to me, is it possible that me running this with Python 3.9 would affect anything? Maybe I should rollback to a past version?

9

Strange that specifying the path to the compiler didn’t work. Can you try the method @jbdv suggested, of running the setvars.bat file in Pycharm before running f2py? You won’t need to specify the path to the compiler then.

I don’t think this will be related to the Python version at all. You’re right that it’s to do with ifort and ifx not being on your path, but that setvars.bat should fix that for the terminal you are currently in (e.g. Pycharm’s).

10

I just (sort of) tested the setvars.bat approach in PyCharm’s terminal and it worked for me. I say sort of because I don’t have the oneAPI compiler installed, I only have an older ifort installed and so used a different .bat file ifortvars.bat. But the principle is the same.

11

This is honestly so confusing and I really appreciate you guys helping out with it.

This is what I received as output when I ran setvars in the Pycharm terminal:

PS C:\Program Files (x86)\Intel\oneAPI> .\setvars.bat
:: initializing oneAPI environment...
   initializing Visual Studio command-line environment...
   Visual Studio version 16.5.5 environment configured.
   Visual Studio environment initialized for: 'x64'
:  advisor -- latest
:  compiler -- latest
:  dal -- latest
:  debugger -- latest
:  dev-utilities -- latest
:  dnnl -- latest
:  dpcpp-ct -- latest
:  inspector -- latest
:  intelpython -- latest
:  ipp -- latest
:  ippcp -- latest
:  itac -- latest
:  mkl -- latest
:  mpi -- latest
:  tbb -- latest
:  vpl -- latest
:  vtune -- latest
:: oneAPI environment initialized ::

I cd'd back to the directory with my fortran file, and ran the command

 f2py -c -m fortranfile fortranfile.f90

which resulted in

No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
customize MSVCCompiler using build_ext
get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
customize GnuFCompiler
Could not locate executable g77
Could not locate executable f77
customize IntelVisualFCompiler
Could not locate executable ifort
Could not locate executable ifl
customize AbsoftFCompiler
Could not locate executable f90
customize CompaqVisualFCompiler
Could not locate executable DF
customize IntelItaniumVisualFCompiler
Could not locate executable efl
customize Gnu95FCompiler
Could not locate executable gfortran
Could not locate executable f95
customize G95FCompiler
customize IntelEM64VisualFCompiler
customize IntelEM64TFCompiler
Could not locate executable efort
Could not locate executable efc
customize PGroupFlangCompiler
Could not locate executable flang
don't know how to compile Fortran code on platform 'nt'
warning: build_ext: f77_compiler=None is not available.

building 'fortran' extension
error: extension 'fortranfile' has Fortran sources but no Fortran compiler found

The numpy.distutils._msvccompiler module being missing is a recurring error but I am not sure if it would affect this at all. Would I need to move my Fortran files into the folder with ifort.exe or ifc.exe? I really don’t know if this would even be possible but I am grasping at straws…

I actually tried running the command

f2py -c --fcompiler=intel --compiler=Pycharm C:\Users\user\project\fortranfile.f90

but it said that it didn’t know how to compile C/C++ code on platform ‘nt’ with Pycharm compiler.

Another thing, I was reading through f2py help and I noticed two things that I had missed. One of the options is

--include-paths <path1>:<path2>:...   Search include files from the given directories.

I have not completely read through the code in the Fortran files that I am trying to compile (I didn’t write them), so is it possible that there are includes that must be noted? Unfortunately I can’t post the source code, but I can edit it (NDA).

The other thing I noticed was the note at the bottom of the help log that said

  Using the following macros may be required with non-gcc Fortran
  compilers:
    -DPREPEND_FORTRAN -DNO_APPEND_FORTRAN -DUPPERCASE_FORTRAN
    -DUNDERSCORE_G77

Lastly, sorry for the info dump, I am currently working in Pycharm, and I just took another look at the Intel website and it says

"Development Environment Flexibility

Use the compiler in command line or in a supported IDE:

  • Microsoft Visual Studio* (Windows* only)
  • Eclipse* C/C++ Development Tooling (CDT) (Linux* only)
  • Xcode (macOS* and Intel Fortran Compiler Classic only)"

… Maybe I will just have to try to get my project into Visual Studio.

12

Okay I think we’re getting somewhere! For the moment the main issue is that it is still that it’s complaining that it can’t find a compiler. Issues about includes won’t matter until you get to the compilation stage (then we can deal with them :wink: )

Straight after running .\setvars.bat, can you run whereis ifort and whereis ifx to check that the environment variable have indeed been set properly (as the .bat file should be doing)?

If either of those gives a path to the compiler, cd back to where your .f90 file is and try:

f2py -c --fcompiler=intelvem --compiler=msvc fortranfile.f90

You could also try running

f2py --help-fcompiler

That will list all the Fortran compilers f2py finds on your system (and if there are none listed, we know that’s still the problem).

With regards to the IDE you’re using - I believe that all the PyCharm terminal is, is an instance of the command prompt, so anything you’re doing in PyCharm could just be done in the command prompt, or any other terminal you have installed.

1 Like
13

Ok, so sorry about the delay, this project is on the backburner for a bit! I have run the where ifort and where ifx commands which resulted in the same path that we have been working with previously (C:\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\bin\ifx.exe and C:\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\bin\intel64\ifort.exe). Using my Python project path as my cd, one of the .f90 files that I run the command f2py -c --fcompiler=intel --compiler=msvc fortranfile.f90 on results in the error from before numpy.distutils.fcompiler.CompilerNotFound: intel: f90 nor f77. While still cd'd to my Python project path, I run f2py -c --help-compiler which yeilds

Fortran compilers found:
Compilers available for this platform, but not found:
  --fcompiler=absoft    Absoft Corp Fortran Compiler
  --fcompiler=compaqv   DIGITAL or Compaq Visual Fortran Compiler
  --fcompiler=flang     Portland Group Fortran LLVM Compiler
  --fcompiler=g95       G95 Fortran Compiler
  --fcompiler=gnu       GNU Fortran 77 compiler
  --fcompiler=gnu95     GNU Fortran 95 compiler
  --fcompiler=intelem   Intel Fortran Compiler for 64-bit apps
  --fcompiler=intelev   Intel Visual Fortran Compiler for Itanium apps
  --fcompiler=intelv    Intel Visual Fortran Compiler for 32-bit apps
  --fcompiler=intelvem  Intel Visual Fortran Compiler for 64-bit apps
Compilers not available on this platform:
  --fcompiler=compaq   Compaq Fortran Compiler
  --fcompiler=fujitsu  Fujitsu Fortran Compiler
  --fcompiler=hpux     HP Fortran 90 Compiler
  --fcompiler=ibm      IBM XL Fortran Compiler
  --fcompiler=intel    Intel Fortran Compiler for 32-bit apps
  --fcompiler=intele   Intel Fortran Compiler for Itanium apps
  --fcompiler=lahey    Lahey/Fujitsu Fortran 95 Compiler
  --fcompiler=mips     MIPSpro Fortran Compiler
  --fcompiler=nag      NAGWare Fortran 95 Compiler
  --fcompiler=nagfor   NAG Fortran Compiler
  --fcompiler=none     Fake Fortran compiler
  --fcompiler=nv       NVIDIA HPC SDK
  --fcompiler=pathf95  PathScale Fortran Compiler
  --fcompiler=pg       Portland Group Fortran Compiler
  --fcompiler=sun      Sun or Forte Fortran 95 Compiler
  --fcompiler=vast     Pacific-Sierra Research Fortran 90 Compiler

but when running the same command while cd'd to the directory where ifort was found, the log changes to

IntelEM64VisualFCompiler instance properties:
  archiver        = ['C:\\Program Files
                    (x86)\\Intel\\oneAPI\\compiler\\latest\\windows\\bin\\inte
                    l64\\ifort.exe', '/verbose', '/OUT:']
  compile_switch  = '/c '
  compiler_f77    = ['C:\\Program Files
                    (x86)\\Intel\\oneAPI\\compiler\\latest\\windows\\bin\\inte
                    l64\\ifort.exe', 'Files',
                    '(x86)\\Intel\\oneAPI\\compiler\\latest\\windows\\bin\\int
                    el64\\ifort.exe', '/nologo', '/MD', '/nbs',
                    '/names:lowercase', '/assume:underscore', '/O1',
                    '/assume:minus0']
  compiler_f90    = ['C:\\Program Files
                    (x86)\\Intel\\oneAPI\\compiler\\latest\\windows\\bin\\inte
                    l64\\ifort.exe', 'Files',
                    '(x86)\\Intel\\oneAPI\\compiler\\latest\\windows\\bin\\int
                    el64\\ifort.exe', '/nologo', '/MD', '/nbs',
                    '/names:lowercase', '/assume:underscore', '/O1',
                    '/assume:minus0']
  compiler_fix    = ['C:\\Program Files
                    (x86)\\Intel\\oneAPI\\compiler\\latest\\windows\\bin\\inte
                    l64\\ifort.exe', 'Files',
                    '(x86)\\Intel\\oneAPI\\compiler\\latest\\windows\\bin\\int
                    el64\\ifort.exe', '/nologo', '/MD', '/nbs',
                    '/names:lowercase', '/assume:underscore', '/O1',
                    '/assume:minus0']
  libraries       = []
  library_dirs    = []
  linker_exe      = None
  linker_so       = ['C:\\Program Files
                    (x86)\\Intel\\oneAPI\\compiler\\latest\\windows\\bin\\inte
                    l64\\ifort.exe']
  object_switch   = '/Fo'
  ranlib          = None
  version         = LooseVersion ('2021.4.0')
  version_cmd     = ['C:\\Program Files
                    (x86)\\Intel\\oneAPI\\compiler\\latest\\windows\\bin\\inte
                    l64\\ifort.exe', '/FI', '/c',
                    'C:\\Users\\user\\AppData\\Local\\Temp\\tmprbnyu8g0\\j
                    82imlap.f', '/o',
                    'C:\\Users\\user\\AppData\\Local\\Temp\\tmprbnyu8g0\\j
                    82imlap.o']
Fortran compilers found:
  --fcompiler=intelvem  Intel Visual Fortran Compiler for 64-bit apps
                        (2021.4.0)

In both the project directory and the ifort directory, the f2py -c --fcompiler=intel --compiler=msvc fortranfile.f90 command says numpy.distutils.fcompiler.CompilerNotFound: intel: f90 nor f77 for one of my.f90 files, and KeyError: 'void' for another .f90 file that I tried. For whatever reason, the log errors are different for different files. I even tried copying the same files directly in the compiler’s folder, which for the first file gave

fortranfile.f90(7374): error #7002: Error in opening the compiled module file.  Check INCLUDE paths.   [FORTRAN_PACKAGE]
      USE FORTRAN_PACKAGE
----------^
compilation aborted for fortranfile.f90 (code 1)
...
stuff...
failed with exit status 1

And for the other file it gives the key error void again.

Let me know what you think!

14

Have you tried running the setvars.bat file? It should be somewhere like C:\Program Files (x86)\Intel\oneAPI\setvars.bat. Once you’ve done that, it shouldn’t matter what folder you run f2py from, or where your .f90 files are.

That last snippet you posted, when you copied the files directly to the compiler’s folder, implies that the compiler has been found (makes sense, seeing as you were in the compiler’s directory) as we’re now onto Fortran errors, rather than compiler not found errors! That error is telling you it can’t find the TIGER_PACKAGE module, but I can’t help more than that without knowing more about your code I’m afraid.

15

I would venture to say that “the devil is in the details”, and recommend you to try the following steps carefully (copy and paste):

  1. start a new terminal session, and navigate to your project folder
  2. enter: "C:\Program Files (x86)\Intel\oneAPI\setvars.bat"
  3. enter: f2py --fcompiler=intelvem --compiler=msvc -c fortranfile.f90 (note this is slightly different from the command you used)

If you still get errors from Pyhton or f2py after this, then I am also all out of ideas :slight_smile:

16

Even after running the setvars.bat, which I believe we ran before, I still receive the numpy.distutils.fcompiler.CompilerNotFound: intelvem: f90 nor f77 error :smiling_face_with_tear:.

Thank you guys so much for your help with this! I will continue to mess around with it and will update with any developments :unamused:.

Edit: One thing I noticed just now is that when I attempt to run any f2py command (f2py.py --fcompiler=intelvem --compiler=msvc -c fortranfile.f90 for example) within C:\WINDOWS\SYSTEM32\cmd.exe, I receive

'f2py' is not recognized as an internal or external command,
operable program or batch file.

Same thing when I tried with f2py.py instead, as well as python f2py.py.

17

Shame that setvars.bat didn’t work, I was convinced it would :sweat_smile:

The good news is it looks like you’ve got everything you need installed somewhere, it’s just a case of linking it all together. :upside_down_face:

With regards to f2py not being available via cmd.exe - are you using some kind of virtual environment (Conda, pipenv etc) through PyCharm? It might just be installed in there (you can do whereis f2py in PyCharm terminal to figure out where it is installed, if you want to use in cmd.exe).

1 Like
18

I did install Conda yesterday as I saw somewhere that the numpy.distutils.fcompiler.CompilerNotFound error might be solved by using Conda but it didn’t work unfortunately. The reason I am trying to run it in the Windows terminal is because that is what opens through Visual Studio when I click terminal. I do not know if there is a Visual Studio specific terminal, and if there is, how to access it.