A patch was sent to Voyager 2 yesterday

@vmagnin no keep posting here, this thread is one of the most interesting threads here!

2 Likes

Thanks @certik for encouragement! Passion can easily lead us off-topic and feeling the limit can be sometimes difficult… So I will continue my archeological work.

I have found that interesting book:

There are a few “Fortran” occurrences inside, especially about the Gemini program:

An interesting fact to be mentioned aside is that for the purpose of OBSW tests on the ground IBM wrote a counterpart for the OBSW in FORTRAN which in fact simulated the GDCs input as presumed to occur during flight and allowed logging of computed maneuver data. Looking at today’s simulation approaches during entire spacecraft development this system could be interpreted as the first instance of what today is called a “Software Verification Facility”, (SVF), see also chapter 10.5.2 and [98].

The abbreviations are:

  • OBC = Onboard computer
  • OBSW = Onboard software
  • GDC = Gemini Digital Computer

Also:

For the MIL-STD-1750 processor generation numerous high level language compilers were developed. Among them C, FORTRAN, JOVIAL (cf. [42] and [43]). The most widespread language however used for space applications in this era was Ada (cf. [44] and [45]), a language derived from PASCAL. Ada was standardized 1983 by the U.S. Air Force and ANSI under ANSI / MIL-STD-1815 (“Ada 83”).

(MIL-STD-1750 is a 16 bit microprocessor architecture)

“Hierarchic Object-Oriented Design”, (HOOD), is a software design method based on hierarchical decomposition of software architectures (cf. [89] to [92]). It comprises textual and graphical representations of the design – see also an example diagram provided in figure 10.8. HOOD is principally targeted towards SW implementation in procedural languages, namely Ada83, FORTRAN and C.

See HOOD method - Wikipedia for more information
and ESA - Software engineering and standardisation - HOOD

There are three pages about the Voyager missions in the section “3.2.2 Transistor based OBCs with CMOS Memory” with some interesting facts:

The implementation technology of the computers was low / medium scale integrated circuits – partly based on TTL circuits (AACS) and partly on integrated circuits. While the CCS still used magnetic core memory (4 kWords of 18 bit each) the FDS due to required camera image processing performance already applied CMOS RAM (8 kWords) and provided direct memory access (also for the FDS computer). The CCS provides sequencing of time tagged commands and control functions. It contains fixed routines such as

  • command decoding, fault detection and corrective routines, antenna pointing information handling, and
  • spacecraft sequencing information.

For the first time in a space probe the CCS was equipped with a built-in self test software.

I don’t know why writing on the tape is 2x faster than reading:

The tape inside the Voyager recorders is 328m long and can store 536 MBit of data. Write speed is 115,2 kBit/s. Playback speed is 57,6 kBit/s.

The OBSW was written in assembler and especially for the AACS was updated multiple times between planetary encounters for trajectory control and also during target fly-by maneuvers.

Finally, I am sure you would like to have such a PC tower: :grin:

Due to mission targeting to reach the heliopause and to encounter high energetic ion impacts the OBC was encapsulated in a housing made of tantalum and titanium.

I will try to find 70’s papers about these missions. Nowadays web pages are all copying the same statements, it is better to go back to source.

1 Like

On some hard disk devices, it is sometimes the opposite: the data is written, then read, and then verified on each write operation. If this verification fails, then the sector is marked as invalid so that future write operations can skip over it. In contrast, a read (with correct checksum) is just a single read.

I’m surprised that there is an actual physical tape device on the spacecraft, and that it is still working after all this time.

2 Likes

Indeed, in the “Space Flight Operations Schedule (SFOS)” documents, we can see that it is still used nowadays (search for “PLAYBACK” or “TAPPOS”):

We can see the duration of a PLAYBACK is typically 7 hours: 7*3600 s * 160 bits/s = 403,2 kbits (although it could be 10x more if the five antennas are used).

See also that discussion :

And:

This last one says the tape recorder was shut down in Voyager 1, in 2007, although it seems to be contradictory to what I see in the “Space Flight Operations Schedule” documents… (on StackExchange they say S/C 31 is Voyager 1)

1 Like

The Pioneer and then especially the Voyager missions are a pinnacle of engineering and humankind achievement. And they should be studied and learned from and then repeated!

Lately there are many very impressive similar missions, such as the New Horizons probe.

2 Likes

Yes, it is definitely an Odyssey (with a big O!), the greatest space mission of all times with the lunar landings. A lifetime mission… JPL engineers are for sure among the best of the world… Not only for designing spacecrafts, but also to make them function for decades and to repair failures billions km away…

I hope New Horizons will last for so long (household appliances are lasting less and less, but this one is not a fridge :grin:, although in the cold outer solar system now).

1 Like

I learn something every day. Today I learned C is more “modern”. I also learned people send space probes to the outer Space with software written, even partially, in a language where the easiest thing in the world is to cause a memory leak which can be hard to debug. :stuck_out_tongue:
But then again if the original software was written in Fortran 77 (or worse,) the code is probably a spaghetti hell.

Same here, and I think of it as if it was yesterday… @vmagnin, could you please stop remind me I am getting older? :person_with_white_cane:

Easy, we did it for decades, and with the extra bonus of not having to wait for templates compilation, which is remarkably slow. Also, and there is life without OOP too! And yes, I know OOP is very convenient in some cases - but I’m not so sure that is the case for space probes’ software, which is probably full of Numerical Analysis.

Nice, that’s actually better than my typical upload/download rates, even though my Internet provider’s prices are exorbitant. I guess I am in the Deep Space and I don’t know it. :laughing:

1 Like

@Pap, I was being sarcastic in my comments about OOP and templates. The more I try to use OOP the more I realise just how much of it is a bunch of hype. Also, I see generics (templates etc.) as being of value to people who write libraries where you need to support multiple KINDS and ranks. I don’t see a lot of value in the kinds of codes I write (CFD and FEM) ie standalone applications. For these types of codes you normally know what precision you need and the largest rank arrays you need beforehand. The one place where generics might be of value is with defining generic containers for things like lists, maps/dicts, sets etc. But as I’ve stated several times in the past, its my opinion those should be an integral part of the language on par with arrays (and should have been for several decades now).

2 Likes

F77 (and earlier) can (and often was) implemented so that it cannot have memory leaks. There was no dynamic memory allocation, everything could be statically allocated. Run time memory checks, e.g. array bounds checking, involves extra code, which consumes memory. If you only have 3k words of memory to work with, there probably is little or no run time memory checking going on.

I had a friend in the Air Force who worked on the space program in the 1970s and early 1980s. He told me at that time that all the hardware they used in satellites and spacecraft was a decade or two out of date. There were several reasons for that, but one of them is that the old hardware had been tested and its vulnerabilities to temperature extremes, cosmic rays, and EM pulses were known. So instead of having MBs of “modern” DRAM memory (like in their earthbound office computers), they were still using computers with kBs of static memory or even magnetic core memory.

2 Likes

An illustration might be the use of notebooks by Thinkpad in space missions. Here I found the linked paper IBM Thinkpad radiation testing and recovery during EUROMIR missions by Martignano et al. in 1995 (the article’s landing page by IEEE, and Martignano’s author copy on researchgate) to monitor MS-DOS, MS Windows 3.1/3.11 and 95. Though exposed to much lesser intensity of radiation, radiation hardening plausibly interferes with e.g. intercontinental flights (ceiling heights around 10 km) and computerized planes, too.

Happily it was only a few thousand lines. I hope to find more information in old papers… Stay tuned!

If you are still wondered by Miranda’s cliffs, Triton’s geysers and the great dark spot on the deep blue Neptune, then something inside you is forever young!

@pap, are you using carrier pigeons? :laughing: Well, organic low techs may be the future, who knows?

2 Likes

I have found an interesting column on Voyager software:

  • G. Booch, “The Incredible Lightness of Software” in IEEE Software, vol. 31, no. 03, pp. 88-88, 2014, doi: 10.1109/MS.2014.76, CSDL | IEEE Computer Society

No microprocessor in the Voyagers, just integrated circuits. As said by Wikipedia, the first commercially available microprocessor was the Intel 4004, designed by Federico Faggin and introduced in 1971. Too novel to be put in a space mission!

each of these three systems was architected with two processors and built out of approximately 150 standard transistor-transistor logic integrated circuits.

Great, we now have two names that could help find some NASA reports or interviews with more information:

Voyager’s flight software was largely the product of just two men, Richard Rice and Edgar Blizzard. As a software engineer, one thing that Rice said about his experience stuck out to me: “We didn’t worry about top-down or structured,” he said. “We just defined functions.” Indeed. Testable, measurable, executable software that works is always the most important artifact of any software-­intensive project.

Fun detail: the title of the paper deals with software weight in the spacecraft… We learn that software weights nothing, because it’s all just holes… :grin:

(in punched cards of course!)
True, software is just information. Although it needs a material support for storage or energy for transmission, it is immaterial. Just a work of the mind.

A link to dig: https://history.nasa.gov/computers/Part2.html

1 Like

And I forgot that great citation:

Software: it’s in the holes. The trick, of course, is putting the holes in exactly the right place.

1 Like

I think that’s still the case. For example the Perseverance rover launched in 2020 and the JSWT launched in 2021 uses hardware from 2001.

1 Like

Wow! $325,402 per cpu chip.

3 Likes

So given the current inflation about what the Zen 6 and 15th Gen Something Lake processors will cost :slightly_smiling_face:

An interesting video about the Voyagers computers and memory (you will see the hardware inside the probes), and the problems encountered:

FORTRAN is cited two times. No code is shown except assembly language, but just for illustration: it seems to be modern x86 code! Nothing to do with the machine language of the probes… Well, that is just a popularizing science video, not a scientific report.

You will be surprised by the advertising at 2’53"! Just jump to 4’00" to continue immediately the video.

1 Like

Another name that could be used for software archeology concerning the Voyagers: Larry Zottarelli. Found in that 2015 news:

A few delightful quotes:

“It’s like flying an Apple II computer,” said Suzy Dodd, Voyager’s project manager. “It should be in a museum.”

[…] with Voyager in its 38th year, it’s hard to find the documentation that goes with it."

Virtually every document about Voyager was printed or written down on paper. Each time the Voyager team moved locations, some of the papers would be lost during the packing process.

Dodd has a secretary whose full-time job is scanning paper about Voyager into a document cloud system to make the manuals easily searchable. But engineers don’t always write stuff down. Some Voyager engineers have passed away, taking the spacecraft’s secrets with them.

For example, the Voyager team realized last decade that part of the spacecraft’s flight software was going to turn off in 2010. Dodd called as many retired engineers on the Voyager team as she could, but no one remembered why that routine was programmed into Voyager.

That news is in fact quoting the document “Voyager - The Right Staff” (April 2016, Compiled by Joachim J. Kehr, Editor SpaceOps News, Journal of Space Operations & Communicator (http://opsjournal.org)), in which you will learn how engineers are devoted to their beloved probes, often postponing their retirement…

Most of the engineers here have dedicated their career to this project. They have turned down opportunities for promotions and other things because they like Voyager so much they want to stay with it.”
Medina’s devotion to the Voyager is clear to see. “This has been part of my life for so long, and they pay us to do it, so how can you stop doing something you love? I even talk about the spacecraft like it’s a person, especially if it’s my sub-system.”

Suzanne Dodd: The operations team is a total of 12 people, 9 of which are engineers. 10 of the people are dedicated to Voyager and we have several others who are part time. We have 6 workstations in our mission support area and another 6 workstations used for the science data processing.

1 Like

This week (December 12, 2023), the JPL engineers are wrestling to fix an issue: Engineers Working to Resolve Issue With Voyager 1 Computer – The Sun Spot

[…]
This week, JPL engineers have some trouble with a computer inside Voyager 1:
Recently, the TMU began transmitting a repeating pattern of ones and zeros as if it were “stuck.” After ruling out other possibilities, the Voyager team determined that the source of the issue is the FDS. This past weekend the team tried to restart the FDS and return it to the state it was in before the issue began, but the spacecraft still isn’t returning useable data.

It could take several weeks for engineers to develop a new plan to remedy the issue. Launched in 1977, the spacecraft and its twin, Voyager 2, are the two longest-operating spacecraft in history. Finding solutions to challenges the probes encounter often entails consulting original, decades-old documents written by engineers who didn’t anticipate the issues that are arising today. As a result, it takes time for the team to understand how a new command will affect the spacecraft’s operations in order to avoid unintended consequences.
[…]

Fixing a bug with a 45 hours round trip is challenging…
Voyager 1 distance to Earth: 163 AU = 24e9 km = 15e9 miles

Planned obslescence :sweat_smile: !