Where there is a will

Where there’s a will…

Grace Hopper and Margaret Hamilton were recently named among 21 recipients of the Presidential Medal of Freedom in honor of their contributions to the advancement of computer technology. Their names are well known among software professionals, even if not as familiar to the general public.

Hopper was fascinated by gadgets, and when she encountered the Mark I computer in the Navy she was hooked. She became only the third person to try and program the machine, and ultimately was awarded the Naval Ordnance Development Award for her work.

Later she developed the first compiler. Called A-0, or Arithmetic Language version zero, it comprised a set of pre-built subroutines that could take immediate arguments. It functioned as a loader or linker, and did not have text parsing functionality; but it was far more than had been done previously.

Her interest in making computers usable by humans led her to drive a series of improvements in compiler technology, from A-0, A-1, A-2 (ARITH-MATIC) to AT-3 (MATH-MATIC) to B-0 (FLOW-MATIC) to COBOL (Common Business Oriented Language). At its height in the late 20th century, COBOL is thought to have represented about 97% of all production business software worldwide. Not a bad legacy.

Hopper was a high-energy person well known for perseverance. One of her favorite catch-phrases was, “It’s better to ask for forgiveness than permission.” She was determined to figure out a way to get things done.

Years later, Margaret Hamilton would demonstrate the same personal traits and would drive further advancement in the field of software. Contrary to many of the popular accounts that name her as the “head” or “lead” or even the sole author of the software used on the Apollo flights, Hamilton joined the NASA team as the junior-most programmer.

She came to the job with a background in mathematics rather than engineering, and with no more idea what a “programmer” does for a living than anyone else on the team. In those days, there was no such job description. As Hamilton put it in an interview years later, “Because software was a mystery, a black box, upper management gave us total freedom and trust. We had to find a way and we did.”

It was hammered into everyone that this had to be a zero-defect system. If an error occurred a quarter million miles from Earth, the astronauts would die. So, Hamilton inquired as to how the team was ensuring defect-free software. The answer: Augekugel.

She pressed the question and learned this meant “eye-balling.” There was one guy who was pretty adept at spotting potential integration errors by examining the source code visually. Hamilton thought this was a bit risky. She sat with him and learned his method. Eventually she saw that it was a pattern recognition process, and she perceived it could be done programmatically. She invented Higher Order Software; software that operated on other software rather than the domain problem. Today we call it static code analysis.

In the Lunar descent phase of the Apollo 11 mission, another of Hamilton’s innovations came into play. Although they were top-tier test pilots and had completed a massive amount of training, Neil Armstrong and Buzz Aldrin were doing something no one had done before, and it was hardly routine. There are detailed reports of what happened, but in a nutshell the astronauts basically turned everything on and loaded every program, just in case.

Turning everything on was a mistake, as there was a known issue with RF interference between the guidance computer and the rendezvous radar, which Aldrin switched on even though it was not needed in the descent phase. Loading the ascent program was a mistake, as the limited memory in the guidance computer didn’t allow for all the programs to be loaded at once, and that program was not needed in the descent phase.

Between sporadic hardware errors and the scary 1201 and 1202 messages on the little numeric console, the astronauts decided to land manually. Those error codes pertained to another innovation of Hamilton’s – the system could dump programs it didn’t need in order to free up scarce memory resources for the programs it did need. When the astronauts loaded too many programs, the system dutifully dumped the unnecessary ones, based on the mission phase they were in. Despite the scary-looking error codes, there was nothing wrong with the computer or the software.

By that time, Hamilton’s value to the team had been well proven time and again. But initially, she was the junior member of the team. They gave the junior team member the least important program to write. They called it the Forget It program. It would be executed in case of some catastrophic failure. They figured there would be no catastrophic failures. Fast forward to Apollo 13, and Hamilton is in the forefront of the recovery effort.

After Apollo, Hamilton and the person who actually was the team lead, Dan Lickly, formed a company they called Higher Order Software. They were ahead of their time, it seems. Companies weren’t interested in static code analysis…or else the mathematician/programmer and the engineer/professor just weren’t good at sales. Today, it’s almost unthinkable to run a professional software delivery organization without static code analysis. Not a bad legacy.

…there’s a way

When I read that Hopper and Hamilton had been honored by the President, it put me in mind of the attitude we encounter so often in large companies today. People are just one or two steps away from automating something, and they stop short. They continue to repeat the same tasks manually, dozens or hundreds of times per year.

A development team may have a well-vetted and clearly-documented procedure to run tests, integrate their code, and deploy. But they never convert their documentation into an executable script. It would save them so much effort and time!

People are pretty skilled at finding excuses not to do things. At one company, teams were hesitant to set up a continuous integration server because their architecture team was still mulling over the choice of product. I still don’t understand why the team couldn’t set up their own CI server in the meantime. It would have made their lives easier, and they would have worked out any kinks in the general CI process, making the transition to the “official” product that much smoother. Oh, well.

An infrastructure team may have a well-vetted and clearly-documented procedure to provision virtual servers of various kinds. But they never convert their documentation into executable scripts. It would save them so much effort and time!

At one company, the infrastructure team maintained a few hundred servers, mostly VMs, and mostly standard build-outs. One of the guys told me they didn’t script any of the builds because some of the systems were one-off configurations that supported legacy applications with built-in dependencies on various obsolete things. Okay, that’s understandable, but what about the 80-90% of configurations that were standard? Why not go ahead and script those? An individual engineer could at least script his/her own repeated configuration tasks, even without telling anyone about it. Wouldn’t that make for an easier work day? Oh, well.

Maybe I’m the one who’s wrong, here. Maybe I’m just too lazy to repeat the same manual tasks over and over again every day. Maybe I should be more diligent. I’ll work on that one of these days. Promise. Maybe.

If I’m not wrong, then I’m not sure what’s missing, unless it’s the preference to ask for forgiveness rather than permission, or the determination to find a way. As the recent Presidential awards suggest, there are role models available for that sort of thing.