The counter-intuitive rise of Python in scientific computing

My (extremely limited) experience with Anaconda is that it is a giant mess…

1 Like

For many years students in my department would come asking for help with their broken Python dependencies. Most of the times I helped them fix their problems by wiping their conda environment and starting clean with virtualenv and pip. For non-Python dependencies, many would be available via the OS package manager and we’d do that. In rarer cases, I taught them how to build from source. Now that I have my own group, conda is not allowed. They can still use it, but they won’t get help from me. :slight_smile: Not a popular opinion, I know…

1 Like

Conda is great! If you are deploying cross-platform apps to non-programmer types on multiple platforms that have numerous non-Python dependencies, it is a life saver. Telling users they have to compile third party packages from source, or get their admin to install a bunch of packages system wide, and having to write up instructions for all that for multiple platforms is just a waste of everybody’s time. I don’t want to compile other people’s code, I want the people who know what they are doing to do that for me. :slight_smile: I can ship a conda environment file, so users can run one command and then they have everything they need in a folder in their user directory ready to go that doesn’t interfere with anything else.

One of the problems people get into with conda is that they install things in their base environment. You shouldn’t do that. You should make separate environments for whatever it is you are doing.

It’s also great for developer tools. I can get python, gfortran, cmake, fpm, etc. for Mac, windows, and Linux all from conda-forge. Easy to set up and try different versions in different environments, with the same instructions for all platforms.

4 Likes

For that scenario, of course, it’s a life saver. Not exactly my environment, nor a typical one in research.

1 Like

It’s interesting that people have so widely opposite experiences.

My experience is just like @jacobwilliams that indeed Conda is the only tool that actually works, on all platforms. Don’t install anything in your base environment and don’t activate it by default. Then Conda just works. We also use it for LFortran where we create conda packages, and then we test them on all platforms at GitHub - lfortran/lfortran-conda-testing: Repository for testing conda lfortran package, to ensure they actually work. After that passes all our tests, we can guarantee that the conda binary packages work. So we use it as a “reference build” in this sense. All other platforms can then always reference the conda builds to see how to build it correctly.

3 Likes

I don’t find it interesting. It’s expected. You guys clearly need to ship cross-platform. I only need Linux. I think the trouble with Conda in my surroundings is that beginners are told to just type conda everything everywhere and not to worry what it means. For most things here it’s not needed.

2 Likes

Yes I think you are right! It’s the same problem as with pip install X, ideally prepending sudo in front to make it work… (Don’t do either of it.)

Answer: one has to always load an environment (either python env, or conda env) and only then one can do pip install X, or conda install X.

3 Likes

my experience is also similar to @jacobwilliams. I have two environments for Fortran (one for LFortran and another gfortran). For python I use different envs so as to be consistent. It is really cool.

1 Like

Matlab has a similar user base as python (maybe 1/2)

Julia is new and has relatively few features though it is incorporating modules from python as fast as it can. Obv better for performance but there is a bit of a learning curve there. Julia also sits in an awkward space between byte compiled tools like python that have toolkits and installed bases and are ‘fast enough’ for many problems and compiled languages with parallel frameworks more suitable for very large (>> 100 nodes).

Why python? Large user base, many many performant packages pulled in from compiled libraries, copied from others (see previous pt, matplotlib from matlab, etc), free, older and more established than julia. Came along at the right time when computers could delivery good performance for byte compiled languages. Compare to java and the mid 90s.

1 Like

Dont agree with most of this. The term “syntactic sugar” does not have the broad interpretation being cited. A concrete example would also be more helpful. Compiler implementations, language complexity, learning curve for new users are not one time trivial costs. There is a reason the fortran standard is more than 2x shorter than the C++ one.

@jcwright,

You’re entirely missing the forest for the trees here.

You fail to understand “syntactic sugar” ain’t some deeply researched technical term in “computer science” and applied to programming languages and their development that there shall be some “rigor” around the interpretation of this term.

TL;DR of my post is “syntactic sugar” is wielded rather questionably by some “vendor reps” in the context of Fortran language advancement, often with the explicit purpose of dismissing certain ideas and suggestions from consideration, treating posts at sites such as GitHub Fortran proposals as “DOA”. That’s absolutely nasty, a disservice to many, many practitioners of Fortran.

Separately, if you buy into the arguments around cost and effort, then first learn to be absolutely honest and accurate about both costs and benefits, and present unbiased benefits-to-costs analysis on language changes that go beyond your “gut feel”.

Or go with the “gut feel” re: benefits only from the practicing Community, who actually code “in anger” using Fortran; forget about cost, which is rarely if ever quantifiable accurately and honestly. And forget about vendor reps, they simply are in position to share any info re: actual.costs. Moreover all the biases, implicit or conscious, are in place with the vendors to whine and moan about any change that their “big customer” does not demand.

By and large, that’s what occurs with Python PEP workflow and Microsoft .NET.
By focusing on the benefits case rather than cost, both these language paradigms are serving their user base much better than Fortran language committee does for its practitioners.

agree to disagree here. If you are trying effect any change here, this will not accomplish anything.