Compile and distribute binary generated with CI

I have a package that is built with cmake and gfortran (or any other compiler). It is easy to compile, but I only have Linux machines, and from time to time some (particularly Windows) user requires it for a different platform.

Is there a way to generate binaries in a CI run for all three major platforms and make them available for distribution?

I mean, this compilation is supposed to happen already for testing, isn’t it?

1 Like

It is possible. But I have not had the time to set it up for my own use cases so far, so I cannot comment on the pragmatic aspect of it. The search keyword for it is likely “automatic deployment to GitHub releases”. Here is some help with Travis CI. I am sure CircleCI and others also have (perhaps easier) ways to achieve the goal. Both Intel ifort and gfortran could be likely installed on the fly in a CI environment. But the ifort installation is likely going to take longer, so you may need to cache the installations to avoid waste of time.

1 Like

Continuous delivery (CD) is quite possible via GitHub actions, fpm is using this approach to generate binary artifacts for releases as well as for a rolling release (workflow file). There is more than one way to achieve this goal, fpm’s approach is one way, for another project of mine I’m using a bit different approach (workflow file).

CD aside, if you releasing your software on a regular basis, a good way to make it accessible is to package it for one or more package ecosystems. Conda-forge has a pretty decent infrastructure for build Linux (x86_64, aarch64, ppc64le), MacOS (x86_64, aarch64) and Windows (x86_64).

Usually my main problem with packaging is MacOS, because you can’t link statically there and they change and break things frequently.

3 Likes

For LFortran I have also settled on just using conda-forge. It works really well.

However, it’s good to have just simple binary tarballs, and @awvwgk described how to do that. Regarding macOS, my understanding is that you have to link libc dynamically (against the macOS system libraries), but everything else you can link statically and it should work. For Windows I think you have to follow a similar approach.

1 Like

The way we usually build in the CI tends to be problematic for most CD approaches. I’m usually building with code coverage, full debug flags and maybe with runtime checks enabled, this is not something I want to ship to my users. The question is whether you want to also build in release mode (doubles CI cost) or build differently on push / pull-request events (risk of breakage with full optimization). The alternative is to build in a debug+optimization setup (-DCMAKE_BUILD_TYPE=RelWithDebInfo) and ship the binaries you are testing in the CD. Fully optimized binaries can still be built on releases, i.e. when packaging.

1 Like

Thanks, I’m using GitHub actions, and I will try that approach.

Concerning conda, the package is already available through it, but I have nothing to do with that - nor I use conda, so I’m not quite familiar with it - and, apparently, it is not available for Windows: Packmol :: Anaconda.org

I’m unsure if the people that asks me for binaries for Windows will be familiar with Conda. I need to generate statically-linked binaries, so they don’t need to install any dependency.

If you plan to release for Linux, using https://openbuildservice.org/ could be an option. It has support for CI (SCM/CI Workflow Runs and More - Open Build Service) but I have never tried it.

I found the learning curve for writing recipes to build rpm and deb packages very steep. Even with a working setup, regular maintenance work is required: Every time a new version of an supported OS is released you need at least to check whether it still works. Often, minor adjustments are also needed.

1 Like

I have not tried it, but it sounds like it would be interesting to try
fpm - packaging made simple — fpm - packaging made simple 1.9.0 documentation
which has a CLI interface from a CI script to create various packages.

1 Like