After Columbia

LAST DECEMBER, THE EXTERNAL TANK THAT WILL SOON THUNDER OFF THE LAUNCH PAD bolted to the space shuttle Discovery lay on its side in NASA's Michoud Assembly Facility, near New Orleans. It was a pale, pretty yellow. The insulating foam applied to its metal skin would later darken with exposure to ultraviolet light, eventually turning the familiar reddish orange color of the tanks we see on the launch pad.

A dozen technicians were working at various levels of a multi-story scaffolding erected around the tank, which they refer to as ET-120-the 120th external tank built at Michoud. Off to the side of the broad scaffolding, several clear plastic garbage bags held pieces of foam that had been cut away. I reached into one and pulled out a hand-size piece. I was shocked at how little it weighed; I could have been holding a Kleenex. But this is the stuff that cracked a hole in the leading edge of the space shuttle Columbia's wing, resulting in the vehicle's disintegration and the death of all seven of its crew on February 1, 2003. When the culprit was discovered, the people at NASA and tank builder Lockheed Martin Space Systems-people who had worked with the foam for twenty-odd years-were just as shocked as I was.

Neil Otte [pronounced "OTT-ee"] is NASA's chief engineer for the External Tank Project at the Marshall Space Flight Center in Huntsville, Alabama, though he has spent much of his time at Michoud over the past two years. "The morning of the Columbia accident," Otte said, "I know personally-when I got called and went into Marshall because I was on the investigation team-[the foam] truly didn't cross my mind as being a potential candidate. When [the investigation] led to the left wing, when it led to the foam strike, yeah, it hit us hard. Because we didn't know that our foam could do that type of damage. And we felt responsible. So, yeah. It hit us hard."

At the direction of the Columbia Accident Investigation Board (CAIB), technicians at Michoud cut away foam from other tanks, the first time in shuttle history that the inside of the material had been examined after being applied. What they found, said Otte, "is that the structure of the foam that we were putting on in certain areas was not the structure that we thought we had in there."

Meanwhile, at the Johnson Space Center in Houston, engineers testing the large, curved panels of insulation that form the leading edge of the shuttle's wing were also finding surprises. This reinforced carbon-carbon (RCC) material turned out to be more vulnerable than anyone knew, and the problem of repairing it in orbit more challenging than anyone thought. The early hope that the next mission, STS-114, could launch in the spring of 2004 quickly faded. The date was postponed to fall, then to March 2005, then May.

At Michoud, Marshall, Johnson, and other NASA centers, and at government labs and contractor facilities around the country, thousands of people have been trying for two years, at an estimated cost of $2 billion, to better understand a machine NASA has been flying for nearly a quarter of a century. The work has been guided by 29 specific recommendations from the CAIB, 15 of which need to be addressed before the shuttle flies again. One requirement, though, has stood out from the rest. As NASA head of space operations Bill Readdy told a group of reporters in January 2004, about six months after the CAIB report was released, "Return to Flight has always been driven by fixing the tank."

The reason the surface of the bullet-shaped container-all 12,620 square feet of it-is covered with insulating foam is to keep the supercold propellants within from boiling away in the Florida sun. To stay liquid, hydrogen must remain at -423 degrees Fahrenheit; oxygen at -298 degrees. The insulation also helps to keep ice from forming on the tank.

Computer-controlled machines spray the smooth areas of the tank with a polyurethane foam, while technicians use spray guns to apply another foam called BX-265 to irregularly shaped areas-around propellant feedlines, for example. BX-265 is similar to foam used in refrigerators and roof insulation, but has more exotic thermal and cryogenic properties, and was formulated especially for NASA.

"You spray it down and it goes on about like water," said Otte. "Then you build it up in layers as you spray it on. When you spray it around [protuberances], the foam can lap over itself and create voids. It can cause pressure as it's expanding. If the geometry is right, it can pull on the foam that you just sprayed underneath it. Those are the types of features that we saw inside the foam structure at the bipod ramp."

Until 2003, each tank had two such ramps-wedges of foam covering the feet of a V-shape (hence bipod) fitting that connects the orbiter to the tank. Oddly, the right one had never been known to shed foam; only the left was troublesome, and it was the instrument of Columbia's destruction. The irony is that the foam ramps were applied to prevent ice from forming on the bipod attachment post; NASA feared that during launch, ice on the post could break away and damage the orbiter.

While the workers at Michoud were autopsying bipod ramps and other areas identified as tough sprays, engineers in the laboratory at Marshall were placing mockups of the tough-spray areas in thermal-vacuum chambers, which mimic the temperature and pressure extremes of launch. The engineers now have a failure curve: For a given void size at a given depth, they can predict the size and weight of a divot of foam that would be released during launch. Despite many attempts, they were unable to duplicate the failure at the bipod ramp.

If NASA had ever considered finding a new insulating material, it was only the briefest thought. "Could we develop a foam that could perform better structurally? Yeah, we probably could," said Otte. "But it typically takes us four to five years to develop a totally new foam system. And then of course we have to develop the processes to put it on and all the controls that we'd have to put in place."

Otte is an engineer in his 40s with a compact, athletic build and the air of a man proceeding methodically through a long list of action items. As he explained how the program he helps manage had gone wrong and what is being done to correct it, his tone was at times chastened, at other times confident in the investigation and NASA's ability to find and fix problems. Armed with an immense knowledge of his program-Otte has worked with the external tank since 1987-he seemed ever so slightly defensive of the agency "culture," a word the CAIB used to describe an organization-wide weakness, which the investigators partly blamed for the accident. "We've got 2,000 people, counting Lockheed Martin and NASA employees, who do this because they believe that this is what the country needs to do," Otte said. "Everybody has to have a paycheck, don't get me wrong, but these guys are involved in this because they believe in what they're doing. And they're proud of what they do. But they also understand that this is a risky business and they accept that. [Risk] is something that anybody that works shuttle understands. It's something the American people need to understand. If you're going to work shuttle and if the American people are going to fly shuttle, we have to be prepared to accept risks. If we're not, we shouldn't be in this business."

ET-120 has no bipod ramps. Unable to duplicate that fatal failure in the laboratory, the project office eliminated the foam covers altogether. Instead, ice formation will be prevented by four rod-shaped heaters at the feet of the bipod attachment post. The radical changes in the external tank program are not in structure, however. The real change is in the methods by which foam is applied to sections of the tank where foam had broken off...or could.

The Vehicle Assembly Building at Michoud is so vast it doesn't feel like a building; it feels like space, which happens to be enclosed. On the frequent steamy days in Louisiana, Michoud workers who walk for exercise do their miles in the air-conditioned comfort of the VAB.

Before entering the areas where workers were applying foam to ET-121, I was asked to remove my earrings and watch, seal them in a Ziploc bag, and carry them in a small waist pack with a Velcro closure. (A manager helped me wrap tape around my wedding and engagement rings.) Foreign object debris, or FOD, is something NASA, Lockheed Martin, and the United Space Alliance, the NASA contractor that prepares and helps launch the shuttles, have always taken seriously, though not seriously enough, according to the CAIB. Because FOD was the subject of one of the CAIB return-to-flight recommendations, new requirements and procedures have been established. At the beginning of every shift at Michoud, workers and supervisors spend at least 15 minutes discussing safety precautions, including how to search for and dispose of FOD.

Leon Richard (pronounced ri-SHARD, in the style of New Orleans, where he grew up) is in charge of "large acreage applications" for the external tank thermal protection system. He is a personable man who, even after 28 years with Lockheed Martin, clearly relishes the operations in the VAB. At one point in our tour, Richard, who worked his way up to senior manager from installation mechanic, swept his arm out over the great space and said, "This is my American dream."

Besides managing the machines that spray most of the tank with foam of a uniform thickness (after some machines spray it on, other machines shave away excesses), his crew applies foam to the intertank flanges, where big sections of the tank are joined. The greatest amount of foam loss, historically, has been from this area. One reason: The surface is covered by complex ridges and bumps.

In the bay where ET-121 was being outfitted, we rode an elevator 101 feet up to the intertank. There Richard showed me the crevices that had to be filled and the protrusions that had to be carefully covered. "We can't have any voids," he said, "which is what got us in this pickle in the first place."

The intertank has 108 hollow stringers and 52 solid ribs, stiffening structures that help the huge tank stand up to the stress imposed by seven million pounds of thrust. Around each flange, 178 bolts must be carefully covered, first with a sealant, then with foam. "It's 60 feet all the way around," said Richard. "Sixty tedious feet." Watching the painstaking work reminded me of scenes I'd witnessed at the Kennedy Space Center, where technicians removed individually numbered shuttle tiles, inspected and reapplied them, while others sewed insulation blankets beneath the orbiter's skin. For all its sophistication, the space shuttle has many parts that are essentially handmade.

Five sprayers were trained to apply foam to the intertank, and Richard hopes to certify five more. One crew is trained for "closing out," or finishing, the bipod area, another for protuberance-air-load (PAL) ramps, and another for the longeron, a structural support for the tank's aft orbiter attachment struts. Before Columbia, a close-out sprayer could have done the applications on any of those areas; today the workforce is divided into specialties.

Ron McQueen, a production supervisor of foam applications who has been with Lockheed Martin for 23 years, watches a video screen while two sprayers apply foam. A third sprayer is on hand only to watch what the other two are doing. Immediately after the application is complete, the video is run again, and the sprayers, the quality control people, the production supervisor, and several engineers watch the replay, looking, says McQueen, to see if anything has contaminated the foam, or if a void has developed, FOD has been introduced, or more than 45 seconds has elapsed between the first spray and the second. If they spot a defect over a large enough area, the foam will have to be removed and replaced.

"These new [procedures] were developed over a long period of time," says McQueen. "On the bipod, we probably worked for a year and a half coming up with a new [process], which we were all involved in-the supervisors, the hourly employees. [NASA and Lockheed Martin managers] took our suggestions right along with theirs."

Application is now a six-part process, not counting the video review and other quality control steps. Before the foam is applied to a real tank, it is applied to a high-fidelity mockup of the tank area to be sprayed. The sprayer applies foam to a "lead-in" test panel, then to the mockup, then to a "lead-out" test panel. All three are later dissected to see if any voids or weak areas developed in the foam. The steps are repeated for the tank that will eventually be sent into space, but only the lead-in and lead-out panels are dissected, unless the technicians see something unusual.

And yet, despite all the safeguards, when Discovery is launched this spring, some foam will almost certainly shed from the tank.

Wayne Hale, the deputy space shuttle program manager, explained last December that NASA managers interpreted the CAIB's recommendation as "eliminate all debris that could cause damage." After hundreds of tests, the engineers determined that a piece of foam weighing as little as .023 pound, if it came off the top of the tank, could damage the wing leading edge so severely that safe reentry would be questionable. (The piece that struck Columbia's wing weighed an estimated 1.67 pound.) NASA believes that no debris larger than .008 pound will come off; that leaves a safety margin of only .015 pound. "We were very clear from day one...that if in fact the requirement becomes 'No debris,' we are not going to be able to make it-not with this foam system," Neil Otte said.

When the CAIB required "an aggressive program to eliminate all External Tank Thermal Protection System debris-shedding," was it envisioning zero debris? That's the type of question debated by a group of 26 experts appointed by NASA Administrator Sean O'Keefe in June 2003, as NASA began responding to the CAIB's preliminary recommendations. The Return to Flight Task Group, co-chaired by Apollo astronaut Thomas Stafford and shuttle astronaut Richard Covey, who piloted Discovery in 1988 on the shuttle's first flight after the 1986 Challenger accident, shadowed NASA's employees and contractors at every step of their return to flight, questioning their analyses and decisions and compiling its own report, to be delivered to the administrator about six weeks before the next shuttle is launched. (The task group reports to a different administrator from the one who chartered it; O'Keefe left the agency in February to return to academia.)

The task group's job is to make an independent assessment of NASA's response to the CAIB's 15 return-to-flight recommendations. It is a self-described "umpire calling balls and strikes in a zone defined by the CAIB." Last December, the umpire seemed disposed to approve NASA's solution to external tank debris-shedding. Dan Crippen heads the group's panel evaluating actions taken to improve NASA management. A former director of the Congressional Budget Office, Crippen holds a doctorate in public finance and is the only one of the group's leaders who had never worked in the space program.

"The CAIB clearly understood some limitations," he said at a December press conference. "Their intention was to eliminate all debris. Well, if they thought that was possible, they wouldn't have gone on to say, 'Oh, by the way, you ought to be able to inspect for damage, you ought to be able to repair damage.' Because if you had eliminated all debris, then you wouldn't need that. So they clearly understood in their own discourse that their recommendations were subject to imperfection."

Reading the first four CAIB recommendations, the ones that directly address the physical cause of the Columbia tragedy, you can hear What if ? whispered after each one, as if the committee were trying to lock the shuttle's survivability inside a strong box, then inside a combination safe, then inside a bank vault. In the CAIB plan, foam will not shed from the tank. But what if it does? Then the RCC panels on the orbiter's leading edge will be more impact-resistant than they were. But what if they're not? Then the astronauts will be able to inspect and repair them. But what if they can't?

There's no doubt that the next shuttle flight will be the most carefully watched in history. To the 14 tracking cameras active at the Florida launch site, NASA is adding nine more. There are new cameras on the external tank and solid rocket boosters, and more are planned. Department of Defense telescopes, which could have seen the damage done to Columbia's wing had NASA requested photographs of the craft while it was in orbit, will view every shuttle from now on as a matter of course. (The CAIB censured the STS-107 mission controllers for not availing themselves of these assets after seeing foam fall from the tank.)

If a strike should occur during launch, its impact will be detected by 88 accelerometers and temperature sensors newly installed inside the wing. Even if the sensors register no impact, the astronauts, once in orbit, will deploy a new piece of hardware, the Orbiter Boom Sensor System. Attached to the shuttle's robot arm, the 50-foot boom carries two laser imaging systems and a low-light black-and-white television camera that will scan the shuttle's nose cone and the 44 RCC panels on the wings' leading edges to provide three-dimensional maps of those areas.

In addition, on flight day three, before Discovery docks with the space station, STS-114 Commander Eileen Collins will perform a pitch-around maneuver 600 feet from the station, exposing the craft's tiled underside to photography by the station crew. The two astronauts on the station will have about 100 seconds, as the shuttle turns a slow somersault at a rate of .75 degree per second, to scan the shuttle's underside for damaged tiles and take pictures of specific areas, like the seals over the main landing gear, that experience high heating. "They will almost immediately downlink those digital photos to mission control," says Collins, "and once we've docked, we'll be able to go into the space station and see the pictures."

Collins is pure astronaut. A thoughtful, exuberant pilot, with the no-problem-too-big attitude the astronauts are famous for, she is the only woman to have commanded a space shuttle mission. STS-114 will be her second command. (She is also a mother of two, and there is a remarkable interview on NASA's Web site in which she describes explaining the Columbia accident to her eight-year-old daughter.) The pitch-around maneuver has never been done, and Collins admits, "Initially it was a big concern to shuttle management." In fact, it violated one of the program's long-standing safety rules: When a shuttle is in close proximity to another spacecraft, the target must be in sight at all times. For this maneuver, the space station will be out of the astronauts' view for six or seven minutes.

Over time, shuttle managers have satisfied themselves that the risk is acceptable. "I've been flying this [in the simulator] for over a year and a half," Collins says. "We've made some good changes to make it safer." Even if the astronauts spot trouble during the inflight inspections, they won't be able to repair the kind of damage that brought down Columbia. With just months left before the STS-114 launch, NASA and its contractors were still struggling to develop materials and techniques to repair tiles and RCC.

"The space agency gave up on tile repair in the 1980s as an impossible task," says NASA's Wayne Hale. Although some progress has been made since, in an emergency the STS-114 astronauts will have only a minimal repair capability-applying a patch or using a kind of caulk gun to fill holes were the two leading repair methods proposed as of early this year. This CAIB mandate has proven among the hardest to meet, and NASA may only be able to satisfy the letter-of-the-law requirement for a "capability to effect emergency repairs." Foreseeing the difficulties with repair, NASA added another safety measure for STS-114: the rescue shuttle. If the astronauts make it to orbit but can't return to Earth, they will dock with the International Space Station and wait for another shuttle to arrive and take them home. The station has enough oxygen, food, and other supplies to host seven refugees for 45 days.

Early last year, in a self-imposed directive, shuttle program managers decided to make rescue capability a requirement for the first few flights. Dick Covey's Return to Flight Task Group has endorsed the idea. Yet when Covey piloted Discovery after the Challenger accident, there was no space station to offer shelter and no second shuttle ready to rescue him and his crew. His commander on that flight was Rick Hauck, a veteran of three shuttle flights and now the president and CEO of AXA Space in Bethesda, Maryland, which insures commercial satellites. Hauck, who has led National Research Council studies on space program risks, calls himself an interested but distant observer of the return-to-flight process. I asked him why he thought a safe haven was required now, when it hadn't been for his flights.

"After Challenger we were dealing with an accident that destroyed the shuttle going uphill," he says. "So the question of being safely in orbit and then being able to be rescued was not deemed to be the highest likelihood. Columbia changed that thinking."

What has not changed, despite Columbia, is the judgment that the ascent to orbit is the most dangerous phase of any shuttle flight. According to NASA's risk assessments, the solid rocket motors and space shuttle main engines are as likely to cause a catastrophic accident as all other shuttle systems combined.

Launching a second shuttle is an action that would be taken only in the direst emergency and, as NASA has admitted in its implementation plan, would put the second crew at risk. "I'm sure the astronauts would take that risk in a moment," says Henry McDonald, former director of NASA's Ames Research Center in California and now a professor of engineering at the University of Tennessee. "The question is whether we ought to let them."

In 1999 McDonald headed a panel to scrutinize shuttle operations after two close calls on the STS-93 mission grounded the shuttle fleet. At takeoff, a pin broke loose and ruptured cooling tubes in one of the three main engines, affecting its performance. Separately, during the same launch, two of the engine controllers unexpectedly shut down. (The commander of the mission was Eileen Collins.) McDonald's panel identified problems with the shuttle program very similar to the ones described by the CAIB more than three years later. He is still "dreadfully disappointed" that more of his panel's advice wasn't followed. More important than a rescue shuttle, he believes, is that the program be constantly subjected to neutral outside review. "Many times [the] people who are candidates for external reviewers may be retired or former employees that share many of the views," says McDonald. "They have the expertise but they don't have the critical view of the vehicle that is necessary to introduce what some people have called 'fresh eyes.'

"Some of the reviews are led by former astronauts," he adds. "I happen to have a great deal of admiration for astronauts. I think they're terrific people and extremely brave. But their acceptance of risk levels is way beyond mine." In what was probably its toughest recommendation, the CAIB called for NASA to do a thorough recertification of the shuttle "at the material, component, subsystem, and system levels" if it planned to keep operating the vehicle beyond 2010. Early last year, the Bush administration decided to retire it instead. When the space station is complete, in another 28 missions, that will be the end of the space shuttle.

Although NASA would like to reach that milestone as soon as possible, in part to free money for Bush's new moon-Mars exploration program, there's nothing sacred about 2010, says shuttle program manager Bill Parsons. "The guidance I received is to look at this as a 28-flight profile," he says. "Don't get caught up in 2010. It's not the driving factor by any means." Parsons is well aware that the CAIB criticized NASA management for allowing schedule pressure rather than safety to guide decision-making in the months before the Columbia accident. That won't happen again, he vows.

Because NASA plans to phase out the shuttle, last December it canceled several upgrades including an "advanced health management system" for the main engines. This suite of sensors and computers would have monitored the engines so that if something started to go wrong, the onboard computers would react instantly. Testifying before a Senate subcommittee in September 2001, NASA head of space operations Bill Readdy said the proposed update would reduce the risk of catastrophic engine failure by up to 40 percent. But leaving real-time decisions about engine throttling to an automated system entails its own risk, says Parsons. Much more testing would have to be done, and given the shuttle's limited life expectancy, he says the money would be better spent on additional ground tests to improve engineers' understanding of the engines.

When I asked Eileen Collins if she spent time worrying about main engine failures, she said that "worrying" is not the word she would use. "I spend time training for failures," she said, "because we have to be ready for that."

Approximately two weeks before the launch of Discovery, NASA's senior managers will hold a Flight Readiness Review, a final meeting to consider any questions left over from the hundreds of program reviews leading up to launch. The managers will have guidance from Covey's Return to Flight Task Group, which will already have reported its findings. And at some point they will agree that everything possible has been done to mitigate the dangers of launching a vehicle into space.

But what if it hasn't?

There are seven astronauts-and thousands of people working every day to ensure their safety-who will still be willing to take the risk.

Originally published in Air & Space/Smithsonian, April/May 2005 . All rights reserved.