Getting fpm working with LFortran

[certik]

If somebody would be able to help getting LFortran working with fpm, that would be awesome. See here for details: Fill in LFortran options into the compilers table by certik · Pull Request #544 · fortran-lang/fpm · GitHub. It can compile the fpm new project if I do it manually. fpm just must be improved to call the lfortran driver with the correct options.

I can help if anything needs to be fixed in LFortran.

Once this works, I want to use fpm to start testing LFortran or bigger projects. And also on Windows, where I am having real trouble getting CMake to work: cmake doesn't work with LFortran on Windows (#536) · Issues · lfortran / lfortran · GitLab

[awvwgk]

Sure, I’ll see what I can do.

I added LFortran support to my compiler refactoring patch:

Add objects for handling compiler and archiver

Currently facing some issues to compile the latest git version of LFortran against conda-forge’s LLVM 11: Compilation fails on Linux (#538) · Issues · lfortran / lfortran · GitLab
Therefore, I currently can’t test it myself. Maybe will I’ll give it another try later.

[awvwgk]

Got it working on my branch (LFortran compiled now with gcc 11 instead of clang 12)

❯ fpm run -- run -C example_packages/hello_world --compiler lfortran
fpm: Entering directory '/home/awvwgk/projects/src/git/fortran-package-manager/example_packages/hello_world'
 + mkdir -p build/dependencies
 + mkdir -p build/lfortran_00000000811C9DC5/hello_world
 + lfortran -c app/main.f90  -J build/lfortran_00000000811C9DC5/hello_world -I build/lfortran_00000000811C9DC5/hello_world  -o build/lfortran_00000000811C9DC5/hello_world/app_main.f90.o
 + mkdir -p build/lfortran_00000000811C9DC5/app/
 + lfortran  -J build/lfortran_00000000811C9DC5/hello_world -I build/lfortran_00000000811C9DC5/hello_world  build/lfortran_00000000811C9DC5/hello_world/app_main.f90.o  -o build/lfortran_00000000811C9DC5/app/hello_world
Hello, World!
fpm: Leaving directory '/home/awvwgk/projects/src/git/fortran-package-manager/example_packages/hello_world'

Still got a failure in the next more complicated example, not sure if this is an issue in LFortran or something still off in fpm here:

❯ fpm run -- run -C example_packages/hello_fpm --compiler lfortran
fpm: Entering directory '/home/awvwgk/projects/src/git/fortran-package-manager/example_packages/hello_fpm'
 + lfortran -c ../hello_complex/source/subdir/constants.f90  -J build/lfortran_00000000811C9DC5/hello_fpm -I build/lfortran_00000000811C9DC5/hello_fpm  -o build/lfortran_00000000811C9DC5/hello_fpm/.._hello_complex_source_subdir_constants.f90.o
 + lfortran -c ../hello_complex/source/farewell_m.f90  -J build/lfortran_00000000811C9DC5/hello_fpm -I build/lfortran_00000000811C9DC5/hello_fpm  -o build/lfortran_00000000811C9DC5/hello_fpm/.._hello_complex_source_farewell_m.f90.o
 + lfortran -c ../hello_complex/source/greet_m.f90  -J build/lfortran_00000000811C9DC5/hello_fpm -I build/lfortran_00000000811C9DC5/hello_fpm  -o build/lfortran_00000000811C9DC5/hello_fpm/.._hello_complex_source_greet_m.f90.o
 + ar -rs build/lfortran_00000000811C9DC5/hello_fpm/libhello_fpm.a build/lfortran_00000000811C9DC5/hello_fpm/.._hello_complex_source_farewell_m.f90.o build/lfortran_00000000811C9DC5/hello_fpm/.._hello_complex_source_greet_m.f90.o build/lfortran_00000000811C9DC5/hello_fpm/.._hello_complex_source_subdir_constants.f90.o
ar: creating build/lfortran_00000000811C9DC5/hello_fpm/libhello_fpm.a
 + lfortran -c app/main.f90  -J build/lfortran_00000000811C9DC5/hello_fpm -I build/lfortran_00000000811C9DC5/hello_fpm  -o build/lfortran_00000000811C9DC5/hello_fpm/app_main.f90.o
<ERROR> Compilation failed for object " app_main.f90.o "
<ERROR>stopping due to failed compilation
STOP 1
 <ERROR> Execution failed for "fpm"
STOP 1

LFortran: Files · 325e56cde7608c1240ff85f233841f043a110a16 · lfortran / lfortran · GitLab
fpm: GitHub - fortran-lang/fpm at 937b87e3a179d8ee969cf157ddd36c1f5cf51698

[certik]

Thank you @awvwgk ! I fixed your compilation issue.

Yes, once it can compile the basic hello world, I was going to test it on the more complicated example and fix any bugs in LFortran.

I plan to have fpm test projects in the LFortran repository itself and test it on all platforms (especially Windows) to ensure everything just works.

[awvwgk]

Just made a quick sweep over all fpm example packages and opened Compiling fpm example packages (#540) · Issues · lfortran / lfortran · GitLab.

Also, I made fpm#527 use cc as C compiler together with LFortran. Hope this is a good default.

[certik]

Thank you @awvwgk, I really appreciate everything you’ve done. I tested:

Add objects for handling compiler and archiver by awvwgk · Pull Request #527 · fortran-lang/fpm · GitHub

and it works perfectly. As part of LFortran’s MVP, I want to compile all the example projects that you reported at lfortran#540, on Linux, macOS (Intel+Arm) and Windows. I looked into the examples and I think it will not be difficult to implement all the missing features in LFortran that those examples need.

Yes cc seems to work on Linux and macOS. I am not sure about Windows, but I think it’s easy to change that by setting the FPM_C_COMPILER variable, so I think this is good enough for now.

[zoziha]

I got up this morning and I was very happy to find that windows-msys2 supports flang, and our windows users have added a convenient Fortran compiler.

fpm is also supported in windows-msys2. I tried to install lfortran in miniconda, but it seems to be very dependent on the conda environment. It is difficult (maybe not) for me to use lfortran in an environment outside of conda, such as cmd.
Is it possible to support lfortran in windows-msys2 in the future? @certik Thank you for your great work!

[certik]

@zoziha

Is it possible to support lfortran in windows-msys2 in the future ? @certik Thank you for your great work!

Yes, it works in cmd.exe without Conda. It just needs to be installed in some way. How do you contribute to windows-msys2? All we need to do is to create a package.

[awvwgk]

@certik

How do you contribute to windows-msys2? All we need to do is to create a package.

To contribute a package you need a PKGBUILD file, which you can submit in a PR to

GitHub - msys2/MINGW-packages: Package scripts for MinGW-w64 targets to build...

I wrote a guide on creating PKGBUILD files with a template for CMake projects here:

Packaging for MSYS2

@zoziha asked me about a MSYS2 version of stdlib when we were preparing the release for version 0.1.0 and while I’m not much of a Windows user, the MSYS2 tool chain is something I found very useful. In particular because its package manager, pacman, is based on Arch Linux’ tool chain, the one I’m using for my own machine. Arch Linux is known for its incredible comprehensive documentation, if you don’t know about it yet, checkout the Arch Linux Wiki. The other nice thing is its very accessible p…

Note that MSYS2 uses LLVM 13, I guess this will cause issues for LFortran since it requires LLVM 11 at the moment.

[awvwgk]

Not quite what you were looking for but here is a PKGBUILD file to build LFortran against the MSYS2 runtime (like Cygwin).

_realname=lfortran
pkgbase=${_realname}
pkgname="${_realname}"
pkgver=0.14.0
pkgrel=1
pkgdesc="Modern interactive LLVM-based Fortran compiler"
arch=('i686' 'x86_64')
url="https://lfortran.org/"
makedepends=("python"
             "cmake"
             "ninja"
             "gcc"
             "zlib-devel")
depends=("llvm"
         "gcc-libs")
options=('strip')
source=("https://lfortran.github.io/tarballs/release/${_realname}-${pkgver}.tar.gz")
sha256sums=('fc3c1d592c56ae2636065ec0228db747f154f65a0867f6311bc8091efd5c13a7')

build() {
  cd "${srcdir}/${_realname}-${pkgver}"
  local _build="build_${CARCH}"

  cmake \
    -B"${_build}" \
    -GNinja \
    -DCMAKE_INSTALL_PREFIX=/usr \
    -DWITH_LLVM=ON

  cmake \
    --build "${_build}"
}

check() {
  cd "${srcdir}/${_realname}-${pkgver}"
  local _build="build_${CARCH}"

  ctest \
    --test-dir "${_build}" \
    --output-on-failure

  python \
    run_tests.py
}

package() {
  cd "${srcdir}/${_realname}-${pkgver}"
  local _build="build_${CARCH}"

  DESTDIR="${pkgdir}" \
  cmake \
    --install "${_build}"
}

I can build it with

$ makepkg --syncdeps --cleanbuild --log --force --nocheck

Note that the checks are disable (--nocheck), because they don’t pass yet. The error log from the failing tests is below:

findSymbol failure in test_llvm

      Start  6: test_llvm
 6/10 Test  #6: test_llvm ........................***Failed    0.31 sec
[doctest] doctest version is "2.3.4"
[doctest] run with "--help" for options
LLVM Version:
===============================================================================
../src/lfortran/tests/test_llvm.cpp:18:
TEST CASE:  llvm 1

../src/lfortran/tests/test_llvm.cpp:29: ERROR: CHECK( e.int64fn("f1") == 4 ) THREW exception: "findSymbol() failed to find the symbol 'f1'"

../src/lfortran/tests/test_llvm.cpp:31: ERROR: CHECK( e.int64fn("f1") == 4 ) THREW exception: "findSymbol() failed to find the symbol 'f1'"

../src/lfortran/tests/test_llvm.cpp:39: ERROR: CHECK( e.int64fn("f1") == 5 ) THREW exception: "findSymbol() failed to find the symbol 'f1'"

../src/lfortran/tests/test_llvm.cpp:41: ERROR: CHECK( e.int64fn("f1") == 5 ) THREW exception: "findSymbol() failed to find the symbol 'f1'"

===============================================================================
../src/lfortran/tests/test_llvm.cpp:63:
TEST CASE:  llvm 2

../src/lfortran/tests/test_llvm.cpp:75: ERROR: CHECK( e.int64fn("f1") == 4 ) THREW exception: "findSymbol() failed to find the symbol 'f1'"

../src/lfortran/tests/test_llvm.cpp:86: ERROR: CHECK( e.int64fn("f2") == 4 ) THREW exception: "findSymbol() failed to find the symbol 'f2'"

===============================================================================
../src/lfortran/tests/test_llvm.cpp:98:
TEST CASE:  llvm 3

../src/lfortran/tests/test_llvm.cpp:121: ERROR: CHECK( e.int64fn("f1") == 5 ) THREW exception: "findSymbol() failed to find the symbol 'f1'"

../src/lfortran/tests/test_llvm.cpp:98: ERROR: test case THREW exception: findSymbol() failed to find the symbol 'inc'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:183:
TEST CASE:  llvm 4

../src/lfortran/tests/test_llvm.cpp:202: ERROR: CHECK( e.int64fn("f1") == 5 ) THREW exception: "findSymbol() failed to find the symbol 'f1'"

../src/lfortran/tests/test_llvm.cpp:183: ERROR: test case THREW exception: findSymbol() failed to find the symbol 'inc'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:237:
TEST CASE:  llvm array 1

../src/lfortran/tests/test_llvm.cpp:276: ERROR: CHECK( e.int64fn("f") == 6 ) THREW exception: "findSymbol() failed to find the symbol 'f'"

===============================================================================
../src/lfortran/tests/test_llvm.cpp:279:
TEST CASE:  llvm array 2

../src/lfortran/tests/test_llvm.cpp:321: ERROR: CHECK( e.int64fn("f") == 6 ) THREW exception: "findSymbol() failed to find the symbol 'f'"

===============================================================================
../src/lfortran/tests/test_llvm.cpp:328:
TEST CASE:  llvm callback 0

../src/lfortran/tests/test_llvm.cpp:345: ERROR: CHECK( e.int64fn("f1") == 5 ) THREW exception: "findSymbol() failed to find the symbol 'f1'"

Module:
; ModuleID = 'LFortran'
source_filename = "LFortran"

define i32 @f() {
.entry:
  %f = alloca i32, align 4
  store i32 5, i32* %f, align 4
  %0 = load i32, i32* %f, align 4
  ret i32 %0
}

===============================================================================
../src/lfortran/tests/test_llvm.cpp:349:
TEST CASE:  ASR -> LLVM 1

../src/lfortran/tests/test_llvm.cpp:374: ERROR: CHECK( e.int32fn("f") == 5 ) THREW exception: "findSymbol() failed to find the symbol 'f'"

Module:
; ModuleID = 'LFortran'
source_filename = "LFortran"

define i32 @f() {
.entry:
  %f = alloca i32, align 4
  store i32 4, i32* %f, align 4
  %0 = load i32, i32* %f, align 4
  ret i32 %0
}

===============================================================================
../src/lfortran/tests/test_llvm.cpp:377:
TEST CASE:  ASR -> LLVM 2

../src/lfortran/tests/test_llvm.cpp:401: ERROR: CHECK( e.int32fn("f") == 4 ) THREW exception: "findSymbol() failed to find the symbol 'f'"

===============================================================================
../src/lfortran/tests/test_llvm.cpp:404:
TEST CASE:  FortranEvaluator 1

../src/lfortran/tests/test_llvm.cpp:404: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_2'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:419:
TEST CASE:  FortranEvaluator 2

../src/lfortran/tests/test_llvm.cpp:419: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_1'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:431:
TEST CASE:  FortranEvaluator 3

../src/lfortran/tests/test_llvm.cpp:431: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_2'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:446:
TEST CASE:  FortranEvaluator 4

../src/lfortran/tests/test_llvm.cpp:446: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_2'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:472:
TEST CASE:  FortranEvaluator 5

../src/lfortran/tests/test_llvm.cpp:472: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_3'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:520:
TEST CASE:  llvm complex type

../src/lfortran/tests/test_llvm.cpp:552: ERROR: CHECK( std::abs(e.floatfn("f") - 8) < 1e-6 ) THREW exception: "findSymbol() failed to find the symbol 'f'"

===============================================================================
../src/lfortran/tests/test_llvm.cpp:556:
TEST CASE:  llvm complex type value

../src/lfortran/tests/test_llvm.cpp:591: ERROR: CHECK( std::abs(e.floatfn("f") - 8) < 1e-6 ) THREW exception: "findSymbol() failed to find the symbol 'f'"

===============================================================================
../src/lfortran/tests/test_llvm.cpp:595:
TEST CASE:  llvm boolean type

../src/lfortran/tests/test_llvm.cpp:621: ERROR: CHECK( e.boolfn("b") == false ) THREW exception: "findSymbol() failed to find the symbol 'b'"

===============================================================================
../src/lfortran/tests/test_llvm.cpp:625:
TEST CASE:  llvm boolean type

../src/lfortran/tests/test_llvm.cpp:651: ERROR: CHECK( e.boolfn("b") == false ) THREW exception: "findSymbol() failed to find the symbol 'b'"

===============================================================================
../src/lfortran/tests/test_llvm.cpp:655:
TEST CASE:  llvm pointers 1

../src/lfortran/tests/test_llvm.cpp:655: ERROR: test case THREW exception: findSymbol() failed to find the symbol 'f'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:678:
TEST CASE:  llvm pointers 2

../src/lfortran/tests/test_llvm.cpp:678: ERROR: test case THREW exception: findSymbol() failed to find the symbol 'f'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:697:
TEST CASE:  llvm pointers 3

../src/lfortran/tests/test_llvm.cpp:697: ERROR: test case THREW exception: findSymbol() failed to find the symbol 'f'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:737:
TEST CASE:  llvm pointers 4

../src/lfortran/tests/test_llvm.cpp:737: ERROR: test case THREW exception: findSymbol() failed to find the symbol 'f'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:774:
TEST CASE:  FortranEvaluator 7

../src/lfortran/tests/test_llvm.cpp:774: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_2'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:786:
TEST CASE:  FortranEvaluator 8

../src/lfortran/tests/test_llvm.cpp:786: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_2'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:798:
TEST CASE:  FortranEvaluator 8 double

../src/lfortran/tests/test_llvm.cpp:798: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_2'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:810:
TEST CASE:  FortranEvaluator 9 single complex

../src/lfortran/tests/test_llvm.cpp:810: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_1'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:823:
TEST CASE:  FortranEvaluator 9 double complex

../src/lfortran/tests/test_llvm.cpp:823: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_1'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:836:
TEST CASE:  FortranEvaluator integer kind 1

../src/lfortran/tests/test_llvm.cpp:836: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_2'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:851:
TEST CASE:  FortranEvaluator integer kind 2

../src/lfortran/tests/test_llvm.cpp:851: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_2'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:866:
TEST CASE:  FortranEvaluator re-declaration 1

../src/lfortran/tests/test_llvm.cpp:866: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_2'

===============================================================================
../src/lfortran/tests/test_llvm.cpp:892:
TEST CASE:  FortranEvaluator re-declaration 2

../src/lfortran/tests/test_llvm.cpp:892: ERROR: test case THREW exception: findSymbol() failed to find the symbol '__lfortran_evaluate_2'

===============================================================================
[doctest] test cases:     33 |      2 passed |     31 failed |      0 skipped
[doctest] assertions:     46 |     29 passed |     17 failed |
[doctest] Status: FAILURE!
LLVM (http://llvm.org/):
  LLVM version 11.0.0
  Optimized build.
  Default target: x86_64-pc-msys
  Host CPU: skylake

This might be related to LFortran mistaking the Cygwin-like runtime for Linux:

$ lfortran --version
LFortran version: 0.14.0
Platform: Linux
Default target: x86_64-pc-msys