Maintenance programming

There are new projects being started in Fortran, but since the language is old, a large fraction of Fortran-related work can be classified as maintenance. A few forum members work at the Archaeologic consultancy, and other members represent companies that sell tools for maintaining and modernizing Fortran codes. Members have modernized many of the classic FORTRAN 77 codes from Netlib. A recent Hacker News thread discusses a 2011 blog post The joys of maintenance programming by Greg Jorgensen.

I’ve done my share of new development, but today I do maintenance programming almost exclusively. After working on way too many failed ground-up “green fields” projects I prefer to debug, fix, and enhance production software. I like working with real users rather than making up “use cases.” I prefer building confidence and making my clients happy with my work rather than arguing with them about features, specs, budget, and schedule. I like having a prioritized list of well-defined tasks that refer to existing code rather than trying to blue-sky specifications.

A positive attitude toward maintenance programming (which can include optimizing a program for current hardware) may make someone more willing to learn a language with a long history, such as Fortran. The emphasis of the standards committee and compiler developers on backwards compatibility eases maintenance while allowing incremental improvement.

4 Likes

There are severe issues with “maintenance programming” in the case of legacy codebases:

  1. the code remains a mixed bag of FORTRAN 77 and nonstandard dialects with something that introduces language features only up to the Fortran 95 standard revision, even as source may have been translated to free-form,
  2. inadequate attention to concurrent execution and parallel programming possibilities,
  3. the APIs themselves remain unexpanded, the consumers still feel they are consuming something older than the dinosaurs. LAPACK dgemm is simply an illustrative case here of an API a legacy codebase might include; you “lose” the consumers at the first argument, TRANSA!

Last year I worked for a few months on the maintenance of a large-ish (~1M LOC) codebase, mostly poorly written in FORTRAN 77 + non-standard extensions. Although all of the issues listed by @FortranFan were present, working with code directly was challenging but enjoyable. On the other hand, what I found less pleasant (nightmarish, actually) were the specific requirements about the customer’s and customer’s customers’ environments: Must run on Windows, with specific (old) versions of Visual Studio, with specific (old) versions of Intel Fortran compiler. And not only that it must run, but understand and explain why the code (which simulates and predicts future states of certain markets, so, non-linear and thus chaotic) doesn’t produce the same results when changing compiler versions (!!). I suspect that many such legacy projects come encumbered by unreasonable environment requirements.

5 Likes

I knew a small group of programmers in the 1980s that worked on nuclear reactor simulations. These codes ran on IBM 360 and 370 architectures. There was some kind of official benchmark suite that any code modifications was required to pass, and passing that gauntlet required essentially bitwise reproductions. That sounds like a good idea for something as critical as nuclear reactors. However, it presented all kinds of problems relating to numerical algorithm choices and eventually underlying hardware. For example, it was impossible to replace a numerical algorithm embedded within the code with a superior algorithm (more robust, more stable, better accuracy, faster, etc.) if it did not reproduce the benchmark results exactly. It was essentially requiring that the known errors be reproduced exactly. And this was a time when IBM mainframes were being replaced with RISC workstations and parallel machines that were not only faster than the old mainframes but two to three orders of magnitude cheaper to purchase and to operate. But those workstations used IEEE 32-bit and 64-bit binary floating point arithmetic, not the hexadecimal IBM style arithmetic, so bitwise reproduction of the benchmarks was not possible, even with the same algorithms. These conditions resulted in not only keeping outdated IBM hardware in service well beyond its time, but in the purchases of new IBM hardware that for all other purposes was outdated upon arrival, and sucking up limited financial resources that could have been better spent in other ways. I do not know if and how this was eventually resolved. That group of programmers eventually evaporated over time and they were never replaced.

4 Likes

I’ve never liked this type of “golden master file” testing. It goes both ways, in some cases people will simply relax the tolerance and move on without determining if the cause was a bug, or if it was just due to a difference in the sequence of operations.

Slowly, some tools are appearing for verification, validation, and uncertainty quantification, such as EasyVVUQ (EasyVVUQ: Uncertainty intervals for everyone! — EasyVVUQ 0.8 documentation). The idea is to determine uncertainty intervals of the output parameters by simulating an ensemble of models selected to describe the probability distribution (uncertainy) of the input parameters.

For example if one has uncertainty in one input parameter that follows a normal distribution, you can propagate a set of models with the inputs selected as the knots of the Hermite-Gauss quadrature. From the independent simulations at the quadrature knots, you then have a way of calculating the mean and variance of the model outputs.

The beauty about Fortran is that you could use collectives for this purpose. Each knot would become one parallel image or one model invocation, and at any point of the code where you save outputs you would use collectives to obtain the mean and variance. Of course you need to have sufficient resources to allocate the images.

I’m familiar with UQ at the basic level and it’s a powerful tool to study the model sensitivity to its inputs. But what I described above has to do with changes in outputs due to compiler and/or environment version changes, not changes in the code or input data. Across ~10 years of compiler upgrades, bugs get fixed, new bugs get introduced, optimization techniques change, etc.

Every few years or so, I see a conference talk at AMS and/or AGU showing ensembles of hurricane simulations (e.g. with WRF but any model works), each ensemble member using a different compiler version, and otherwise using the same code and input data. What you get is a spread of hurricane tracks that make landfall anywhere from Texas to Florida, just based on different compiler version.

4 Likes

I am reminded of the numerical experiments of meteorologist Edward Lorenz that lead to the discovery of chaos: Edward Lorenz, father of chaos theory and butterfly effect, dies at 90 | MIT News | Massachusetts Institute of Technology. I guess the compiler version and the hardware used are also “initial conditions”.

1 Like