Conda + Fortran = Profit

1

New blog post: Degenerate Conic | Conda + Fortran

Now, we just need to get the Flang people (or somebody) to get the Mac version of Flang on conda-forge and we are all set.

10 Likes
2

I can confirm that now it does work with ifx for windows. the 2025.0 version had a bug that impeded me from using conda for a cross-platform experience, I opened a bug report at intel a few months back, now everything rolls smoothly with the latest release ! :slight_smile:

As you, I also recommend strongly Miniforge3.

For Windows users, the other cool thing is that now one can have official installations imposed by your company/organization (say MVS2019+some version of Intel Parallel Studio which is no longer supported by intel) and one can also have a conda env that doesn’t require admin privileges to have the latest compilers without messing up the computer.

Its funny to see your blog as not latter than Thursday I did an internal blog post at my company much like yours to explain how to use Miniforge3 for both windows and linux :laughing:

1 Like
3

I would add Ninja to your list! conda install conda-forge::ninja

1 Like
4

Good point! I will update the post to point out that you don’t need admin privileges.

5

@jacobwilliams nice blog post! I install these compilers using conda also.

I will mention that when we make a new release of LFortran, we then test the conda packages using this repository: GitHub - lfortran/lfortran-conda-testing: Repository for testing conda lfortran package on all platforms and ensure the released binary package passes all tests. So we guarantee that it works.

Overall I am super happy with the conda ecosystem. I use pixi (that you mentioned) and the prefix shell which especially on Windows is a life saver. It has never been easier to install compilers and project dependencies on all platforms in a uniform way.

2 Likes
6

The lfortran Conda package is indeed the easiest way to install Lfortran, but it’s not obvious to me (a Fortran user but not a systems programmer) how to have two or more versions usable at the same time. Installing from source makes that easy but time-consuming. Comparing output from two versions with the same program is useful when checking whether a suspected regression really is one.

1 Like
7

The way to do this is to create different environments with different versions. You just have to activate the environment you want to use (only one can be active at a time). Example:


> conda create --prefix ./env1 lfortran==0.49 
> conda create --prefix ./env2 lfortran==0.50
> conda activate ./env1
> lfortran --version

LFortran version: 0.49.0
Platform: macOS ARM
LLVM: 19.1.1
Default target: arm64-apple-darwin24.4.0

> conda deactivate
> conda activate ./env2
> lfortran --version   

LFortran version: 0.50.0
Platform: macOS ARM
LLVM: 19.1.1
Default target: arm64-apple-darwin24.4.0
1 Like
8

Thanks for this post, it’s a nice summary of how easy getting started with Fortran is becoming! There’s been a real step change on the compiler front over the past couple of years. From personal experience this is making it a lot easier getting non-Fortraners enthused in writing Fortran code.

I also like your way of creating Conda envs with --prefix, I hadn’t though about doing that. I’ve mostly swapped to Pixi these days but one of the big draws to Pixi for me was the default of creating in-directory environments, and the --prefix flag is a nice way of achieving this in Conda. Pixi is just awesome though :slight_smile:

Note that conda itself is and has always been a free and open source project, so don’t let your confused IT department get away with telling you it requires a paid license to use.

That sounds very familiar :laughing:

1 Like
9

Nice blog post! Cool to see Pixi getting a shout out :rocket:

There is occasionally hand-wringing from some folks (e.g., SciPy … that relying on Fortran compilers is somehow a risky proposition. But it has literally never been easier to install an up-to-date modern compiler for all platforms. Using conda …

I think it is fair to say that this isn’t the whole story, as we don’t just need to be able to install a compiler. To quote Is adding new Fortran to SciPy generally allowed? - #11 by rgommers - scipy - Scientific Python :

building wheels for PyPI is the harder and more time-consuming part, and AFAIK no one has even tried to build any redistributable wheels with Flang. It may or may not work at all, it may increase the binary size due to the extra vendored runtime, etc.
…
right now we’re considering dropping support for macOS 12 wheels in our next release, because we’re using gfortran from Homebrew and the deployment target was bumped too high by Homebrew. For C/C++ you can support older macOS versions very simply, by setting the MACOSX_DEPLOYMENT_TARGET environment variable. For Fortran on the other hand it’s intrinsically hard, you have the choice between using a compiler someone else built and lose control over what you can support, or maintaining you own build of a compiler.

It may be that these challenges have nice solutions in waiting, and if so, great! Until that is clear, though, I think the sentiment of ‘risk’ from some in the Scientific Python community is probably slightly more substantiated that it may initially appear.

1 Like
10

Yes, it’s a lot more that you need. For the requirements of SciPy, Fortran needs “friendly users”, i.e., users who are willing and able to work with the Fortran community to find a solution. However, the SciPy developers are what I would call “demanding users”, i.e., users that expect everything to just work, and consequently if it doesn’t, then the technology is labeled not mature enough and they will move to another technology.

Fortran is getting very close for “demanding users” if you are in pure Fortran. But not if you are distributing a Python library on all platforms, for that use case we still need “friendly users”.

11

I give SciPy a little grief I admit. :slight_smile:

I’ve just never really understood why they seem to have so much trouble with gfortran. Or that something there is “intrinsically hard”. I think it’s just a matter of none of the developers being proficient in Fortran, or wanting to take the time to understand or improve the Fortran/Python interface. I fully understand that volunteers work on what they want to work on, and if they have no interest in these things, then we end up where we are now. So it’s not surprising.
But orders of magnitude more time is spent working to convert the Fortran codes to C/C++/Python than it would take to solve all these problems for good.

1 Like
12

well, I can try to be a ‘friendly user’ to the best of my ability, if anyone has suggestions for how to go about solving all these problems for good :slight_smile:

2 Likes
13

Yes, so here are the steps:

  • One has to use a Fortran compiler that can compile and run SciPy correctly, say Flang or GFortran today, later also LFortran.
  • Can you use Conda to install the compilers? If so, get Flang into macOS/arm, then you have all three compilers in conda on all platforms.
  • It seems you are using Homebrew, but they stopped supporting the older macOS apparently, so that doesn’t seem like a good distribution. I would use Conda, if you can.
  • Build the wheel, and see what the issues are. Let’s list the issues, and let’s figure out how to fix them.

I am pretty sure the Flang developers would be happy to do get Flang working for whatever is needed, say in the runtime. I can promise over time we will definitely get whatever is needed into LFortran as well, and maintain it on all platforms.

14

I dropped a :+1: on osx-arm64 Build? ¡ Issue #95 ¡ conda-forge/flang-feedstock ¡ GitHub.

2 Likes
15

A simple wheel build has succeeded with flang-new (llvm-flang 20.1.3) from conda-forge on a GHA ubuntu runner at build a wheel in CI ¡ lucascolley/pixi-dev-scipystack@15f0a8f ¡ GitHub.

I think what remains to be seen is:

I don’t have a linux machine currently, but anyone is welcome to checkout my PR and SciPy following the instructions at pixi-dev-scipystack/README.md at main · lucascolley/pixi-dev-scipystack · GitHub, to investigate. Then you can just run pixi run wheel from the scipy directory, and all of the environment management is handled automatically by Pixi.

(aside: looks like there are plenty of fun warnings from PROPACK, ARPACK, ODEPACK and co. in that log. They may not be due to using flang, or otherwise be pre-existing, though)

1 Like
16

Excellent, looks like the hardest part is already done, now you just need to fix the bullet points.