Why does Lfortran use conda?

1

I guess this is 50% a question and 50% a rant.

The first time I tried to install LFortran I gave up because I couldn’t figure out a way to install it without conda. It was very frustrating. Today I tried again and this time I bit the bullet and installed conda.

I don’t understand why a Fortran compiler needs conda. It’s bad enough that it needs Python, but apparently the Python I already have in my Linux system is not enough and I need to install 9.23 GB to get a whole other separate install of a Python environment, whose only purpose is to allow me to install a compiler for a language that is not Python.

Even provided that LFortran is written in Python, I think I should be able to use it with the Python I already have in my Ubuntu desktop.

Seems bonkers to me. Just saying.

3 Likes
2

There is a lot to unpack in your comment, but I will try and be brief.

It doesn’t, LFortran Downloads, they also offer tarballs of the latest release, amongst other options.

If you try and build that on a clean system, you will realise that there are multiple massive dependencies like LLVM, which is why installing binaries through conda is offered as an easy alternative.

I cannot emphasise this enough, but this is not a unique thing to LFortran. I invite you to try and build flang-new or gfortran from source. My personal opinion, is that LFortran is a lot easier and less intrusive, when building it, to the global system than the other two.

Well Miniconda, a bare bones version of Anaconda, is only a few tens/hundreds? Megabytes so that seems like the reasonable choice for your case.

The reason why your system Python is not a adequate for LFortran, is because LFortran uses the conda package to control, compile and install its miriad of non-Python dependencies across multiple OS and architectures. Conda and conda-forge are truly great tools for that. This is also a very standard practice when it comes to packaging and distribution of a complex codebase.

I will only mention that the interactive component is based on Jupyter, so for that to be enabled you probably need a Python runtime, but I might be wrong on this (I don’t remember how it works exactly).

I think this Discourse is a good place to ask for help in these matters. We are happy to help and share our knowledge.

9 Likes
3

I think it does not requires it, but it uses it as a distribution channel. I think it was a good decision. Why? While anaconda can be seen mainly around Python, it is a Package Manager that goes beyond the Python language itself. Not only Lfortran uses it to distribute, but many other compilers and packages around, just to give you examples:

> conda config --add channels conda-forge
> conda install -c conda-forge make
> conda install -c conda-forge cmake
> conda install -c conda-forge cxx-compiler
> conda install -c conda-forge gcc
> conda install -c conda-forge gfortran
> conda install -c conda-forge gxx
> conda install -c conda-forge lfortran
> conda install -c nvidia cuda-toolkit !! https://anaconda.org/nvidia/cuda-toolkit
> conda install -c intel <package_name> !! https://www.intel.com/content/www/us/en/docs/oneapi/installation-guide-windows/2023-2/conda.html

You can creat a stable dev stack using conda as a “meta”-package manager if you will. And it will save you a lot of time once you have it installed. For instance, in my WSL Ubuntu 22.04 I was able to get gcc/gfortran 13.2.0 through conda which at the time the build essentials would ship I think it was gnu compilers 11.

Give it a little more time and you might just find it very resourceful!

2 Likes
4

Perhaps I was being unfair because LFortran is such a young project. I wasn’t comparing it to building gfortran from source. I was comparing it to installing Julia or nvfortran, which are binary downloads in a tar.gz file.

Perhaps things will improve as LFortran matures.

Even though you’re not responsible for LFortran? Thanks for letting me know. Next time I have a similar problem I’ll ask here.

5

Wow. I had not realized that conda could do all of that.

Yeah, I thought Anaconda was just a distribution of Python. But what you describe is very different. Almost like a BSD-style “ports” system for Linux. Thanks for explaining that to me.

2 Likes
6

Fortran-lang and LFortran are 2 very friendly orgs with each other, many times one complements the other. @certik is admin in both and there’s a bunch of us that have spent some time using LFortran, that being said there’s a dedicated Zulip chat for LFortran.

2 Likes
7

Even more than that. Conda also works on Windows and MacOS. As a multi-system package manager, Conda is especially useful for operating systems that don’t have a package manager. It competes with system-specific package management solutions (e.g. those using DEB files (Debian, Ubuntu, …) and RPM files (SUSE, Fedora, …)) and language specific package managers which are traditionally used for scripting languages (Python, Julia, Ruby, Javascript, …) but are now also popular for compiled languages (fmp, Conan).

The only alternative with a similar broad scope I’m aware off is spack.

As a developer who wants to distribute software, Conda and spack are nice because distributing for native package managers is a lot of work. As a user, I prefer native package managers because they are less complex and don’t duplicate all libraries.

3 Likes
8

I have to admit I didn’t like the idea of another compiler starting from scratch, and I didn’t like its connection with python. Now, don’t bite me here, yes, I dislike python, I believe it is a hugely overestimated interpreted language and all the hype about it is not justified in any way. But don’t bite, if you like python then good for you, keep using it for all you like, and with all my blessings. Just don’t take what I am saying personally, and grant me the right to dislike what you like. Peace out.

Now, deespite the above, I started testing LFortran quite some time ago, and let me tell you first and foremost I like LFortran a lot. It is the mega-project in the Fortran community nowadays, it has a lot of potential and its progress is impressive. The community is also friendly and listening. It’s really a great project, and so far I have only good words to say about it. I look forward for the day it will be a “daily-compiler”.

That being said, I also have to mention I really dislike conda as well. Now, don’t get me wrong here, conda is a great tool for what it is. I just don’t want another environment in my system, with its own includes, libs, etc. Yes, it is isolated, it doesn’t really interfere with the rest of the system it is “installed” - but I still don’t like it. People do stuff like that in… that other popular “operating system” (they use MinGW, TDM-GCC, MSYS2, etc,) simply because there is no other sane way to do some programming there. But in GNU/Linux, FreeBSD etc, well those systems are tailor-made for programmers. In theory, we should not need yet another /usr. In practice, however, conda is needed - for now. LFortran is in development, I can perfecty understand why conda is there, and I’m not blaming anyone for that.

Compiling LFortran from source is not always an option. Sometime ago, I managed to do that, right now building fails, presumably because of a dependency (which is installed on my system but doesn’t have the required version.) I could probably solve the problem by creating custom packages for the problematic dependencies, and install those on top of the ones the distro provides. But testing this workaround would be a pain, none of my computers is particularly “modern” and the system is literally unusable during compiling, with the fans turning at maximum speed. Compiling such a project from source again and again until the workaround works is not easy, especially since the problem comes midways, and I would have to do it several times until dependencies are ok. So conda is helping here. I can see it’s there for a reason.

To sum up: Don’t blame the project if you don’t like some things that are there for a reason. If you like what you get so far (as I do,) just be thankful to the developers for the insane amount of time and effort they put on the project. And if you want, help in any way you can. Testing and reporting problems is a great (and not particularly hard) way to help.

3 Likes
9

If the goal is to encourage more use of Fortran in appropriate applications every additional dependency will cut out a few more users. Does a compiler really need make AND cmake to compile modern Fortran? Or is that just for developers of the compiler? Is Nvidia CUDA necessary for users who might not have an Nvidia GPU? To me it appears that the distributions are intended for compiler deelopers, not Fortran programmers. A distribution for Fortran users might have far fewer dependencies and attract more users to Fortran itself. Many compilers can be installed with just a single executable and a couple of libraries. That promotes widespread adoption.

Advanced developers who already know and use many tools, may assume that such knowledge is widespread, and that others would benefit from learning them. They act as evangelists for an advanced developement environment that users in other fields might find to be overkill.

Note that Anaconda requires a (costly) license in commercial applications, so that is another barrier to widespread use.

2 Likes
10

@danielc, @Pap I am the lead developer of LFortran. I agree with you, installing Conda from scratch is a pain. However once it is installed, using conda is not too bad. LFortran does not depend on Conda. If you don’t like Conda, you don’t have to use it, just install from source, either from the release tarball (just a few dependencies), or from git (then you need even more dependencies).

The reason we use Conda as our “blessed” binary distribution is because it’s the easiest for us to support on all platforms (Linux, MacOS Intel/ARM, Windows), and guarantee that the Conda binary package that we provide actually works. We test every release of the actual Conda package here: https://github.com/lfortran/lfortran-conda-testing.

I think what you are asking is to create binary tarballs, in addition to Conda. We used to do that, but nobody used them, so we just did Conda until now. What platforms are you interested in?

@danielc if we add an official binary tarball for your platform, will you test it out and provide feedack? :slight_smile: If so, I’ll be happy to work on it. The only way it will work is if we test the tarball automatically at our CI, so we need to setup the infrastructure for that. The way we used to do it is to build a statically built binary on Linux, so that you just unpack the tarball and run it. We might be able to just repackage the Conda package, because we actually build the binary statically in it. @danielc or @Pap if you were interested in helping us set this infrastructure up, that would be awesome. The way we do releases is that I just tag a release at GitHub and everything happens automatically. So our CI must build these binaries automatically. Then we have a testing repository (see above) where the (currently only Conda) binary package is tested to ensure it passes all the integration tests.

2 Likes
11

I know LFortran does not depend on conda, and Ilike I said I understand why the project favors conda installations. My projects are not that big and GNU Make is enough for any platform I install them (even Haiku worked like a charm.) But for such a big project, I would probably use something like conda during development. I’m not complaining.
And indeed conda is not bad. I just wrote a short script that sets paths then launches Emacs - and then I have everything I need to use LFortran. The executables it produces actually work without setting paths. And, to my surprise, even the compiler itself works even if I don’t set paths.

Yes, that’s what I wanted, not because it’s needed right now, but from a distro packager’s perspective. In fact, I don’t think it’s your job to release binary tarballs - that’s packager’s job. That’s what I do in two distributions, and I’m planning to introduce LFortran there. Those are based on Slackware and Slackware-Current (the rolling version of Slackware, updates daily.) The packages are essentially xz-ipped tarballs, compatible with any distribution based on Slackware or Slackware-Current. For the distro based on Slackware-Current, I could also add LFortran in the official repositories (they have strict rules but I’m guessing it will pass the screening process without issues.) The Slackware-based distro is a “stable” distribution, so nothing is added or upgraded in the official repos (except security updates). But even there, LFortran can be in my own “extra” repository on the servers, so that anyone using Slackware could download the package and install easily.

However, with LFortran changing quickly, I just use conda binaries for now. I’m building from source when I can because I’m used to that. Essentially I do it to have the packaging scripts for the distributions I am involved ready and updated.

2 Likes
12

It likes to mess up with your shell prompt and it will edit your PATH. That kind of rubbed the wrong way.

Like you, I’m really not a fan of having a whole other environment installed. But now that I’ve learned what conda brings, I’ve made my peace with it.

1 Like
13

Package managers are probably useful, but I found Anaconda something awful when I had to use it once…

Multiple package managers together, and/or package managers nested in other package managers, which all interfere with your default environment, can be really messy :frowning:

1 Like
14

I also avoid Anaconda. I only ever install mini-conda. Since I always forget the conda commands needed to switch between environments, more often than not I deactivate conda and just use the system provided Python and pip…

If we reach a similar state of affairs with fpm in a decade or two, I’m not sure if I’d call that a success or not :laughing:

1 Like
15

Don’t discount miniconda as mentioned by @gnikit above (and @ivanpribec i see now). It’s only about 65 MB last time i checked and very easy to get set up. I think some of the dislike for conda comes from the association with Anaconda that a belive is a massive install and not just a bare bones package manager. (I don’t know why a package manager needs to be even 65 MB but it’s cooperatively lightweight)

2 Likes
16

There’s another important point here, and that is that Conda is pretty much the de facto package manager in the (Python) data science world. If Fortran ever wants to start attracting new users from those communities, installation via Conda is a must!

I use Miniconda (not Anaconda) extensively and genuinely struggle to think how working without it would be possible. But that’s probably because a lot of my development is in Python (don’t shoot me!), for which dependency management can be a complicated mess…

2 Likes
17

And if 65 MB is too much for you, you could always try Micromamba - I think that’s only around 5 MB!

2 Likes
18

Or Pixi: GitHub - prefix-dev/pixi: Package management made easy, which uses Conda packages, but does not modify your environment.

I usually use Miniforge: GitHub - conda-forge/miniforge: A conda-forge distribution. to install Conda. It works great. I set it so that it does not modify my environment by default. Only if I do “conda activate some_env”.

I agree that a conda package is necessary (although not sufficient).