Tuesday, June 20, 2006
The Matrix
As usual, the science-hungry blogging community has turned as one to Andromeda for a clear, concise explanation of NASA's "risk matrix" and the decision to fly STS-121. Well, if there's one thing we at Andromeda cannot stand, it's clamoring -- ceaseless, pitiful clamoring for our wisdom. If there's two things we can't stand, they're clamoring, and people who say "literally" when they mean "figuratively." And if there's three things we can't stand, they're clamoring, the "literally" thing, and genocide. And people who talk loud on their cell phones.
Flight Readiness Review, under the "new" new NASA, begins by gathering the entire team together and asking each to fill out a form, saying "GO" or "NO GO." Now, by defining the "team" broadly, NASA can create the impression that a "GO" decision is widely supported, because there were only two "NO GO" votes out of several dozen votes cast. Now, your carping critics like to say that this could be misleading, giving the impression of consensus if, for example, the laptop wrangler, the "Fuzzy Dice" tiger team, and the NASA Lunch Lady, okay flight, while the "Make Sure Nothing Explodes" team expresses reservations. Be that as it may, it is important that the two "NO" votes here were from teams in charge of relatively unimportant systems: the safety chief and NASA's chief engineer. Those were the only two votes against!
Bryan O'Connor, wearing the highly inflated title of "Chief of Safety and Mission Assurance," and Chris Scolese, who goes around calling himself, "NASA's Chief Engineer," rated the risk posed by the current configuration of the ice frost ramp as a "PROBABLE/CATASTROPHIC" threat. Yes, those words sound scary, but remember, you don't have to fly in the thing.
Now, look at the chart provided by this web site. It shows how easily the words "probable" and "catastrophic" can be misunderstood. "Probable" simply means "expected in the life of the program." Well, foam loss around the ice ramp has already happened a few times, so you gotta START at "probable" on this crazy matrix. And then the word "catastrophic" also has a kooky definition: "Hazard could result in a mishap causing fatal injury to personnel and/or loss of one or more major elements of the flight vehicle or ground facility." That's COULD result in a mishap. And the loss of just ONE major element of the flight vehicle OR the ground facility triggers this confusing little word. So, if you translate these words into layman's language, "probable catastrophic" just means "something bad might happen." And keep in mind, only two members of the Flight Readiness Review said that!
The important thing to remember is NASA's lost only two vehicles in over 100 flights. To put that in real terms, imagine you drive to work 250 times a year. Would you have any reluctance at all about getting into your car each morning if you knew it was only going to blow up killing everyone inside once every fifty or sixty drives? Of course not. Well, what if you were only going to drive that car 16 more times? And what if you planned to fix the problem before you've driven it even three or four more times? Obviously, that lowers the odds considerably, any statistician will tell you that.
As Wayne Hale explained (pdf), the important thing is not whether it was a good decision, or whether wise counsel was actually followed, but whether lots and lots of people got to ask questions:
NASA Administrator Mike Griffin said the same thing at that press conference:
Everyone talked, and since others were nearby it's reasonable to assume everyone was listened to.
But the question the blogging community keeps pestering us about, strangely, is not, "Was it a good discussion?" but "Is there crew risk?" Mike Griffin answered that question categorically, "No, there is only programmatic risk." But can the crew be killed by programmatic risk? Absolutely not! Not anymore. With the new new NASA.
To begin with, the foam projectile problem is not an ascent risk. That foam may blow huge holes in the shuttle thermal protection system, but come on, this is a freaking huge rocket and ascent is not the problem.
So what if the TPS is shot to hell on ascent and the shuttle has no chance of re-entry? (a) the crew has practiced tile repair techniques; (2) there is a rescue option in the "Launch On Need" shuttle; and (3) the crew can hold out in the International Space Station for quite a while.
But wait, you say, Griffin himself later in the same press conference described those repair capabilities as "rudimentary," saying merely that "we are beginning to converge on some rudimentary repair methods that may be useful." Well, it's true the repair option isn't really much of an option at all. But luckily, there's old reliable "Launch on Need":
So with no real repair capability, and the LON an equally mythical option, there's still the option of holing up in the ISS. In other words, the principal strategy for the next mission is to cross our fingers. (It worked for the ECO sensor problem on the last flight.)
Flight Readiness Review, under the "new" new NASA, begins by gathering the entire team together and asking each to fill out a form, saying "GO" or "NO GO." Now, by defining the "team" broadly, NASA can create the impression that a "GO" decision is widely supported, because there were only two "NO GO" votes out of several dozen votes cast. Now, your carping critics like to say that this could be misleading, giving the impression of consensus if, for example, the laptop wrangler, the "Fuzzy Dice" tiger team, and the NASA Lunch Lady, okay flight, while the "Make Sure Nothing Explodes" team expresses reservations. Be that as it may, it is important that the two "NO" votes here were from teams in charge of relatively unimportant systems: the safety chief and NASA's chief engineer. Those were the only two votes against!
Bryan O'Connor, wearing the highly inflated title of "Chief of Safety and Mission Assurance," and Chris Scolese, who goes around calling himself, "NASA's Chief Engineer," rated the risk posed by the current configuration of the ice frost ramp as a "PROBABLE/CATASTROPHIC" threat. Yes, those words sound scary, but remember, you don't have to fly in the thing.
Now, look at the chart provided by this web site. It shows how easily the words "probable" and "catastrophic" can be misunderstood. "Probable" simply means "expected in the life of the program." Well, foam loss around the ice ramp has already happened a few times, so you gotta START at "probable" on this crazy matrix. And then the word "catastrophic" also has a kooky definition: "Hazard could result in a mishap causing fatal injury to personnel and/or loss of one or more major elements of the flight vehicle or ground facility." That's COULD result in a mishap. And the loss of just ONE major element of the flight vehicle OR the ground facility triggers this confusing little word. So, if you translate these words into layman's language, "probable catastrophic" just means "something bad might happen." And keep in mind, only two members of the Flight Readiness Review said that!
The important thing to remember is NASA's lost only two vehicles in over 100 flights. To put that in real terms, imagine you drive to work 250 times a year. Would you have any reluctance at all about getting into your car each morning if you knew it was only going to blow up killing everyone inside once every fifty or sixty drives? Of course not. Well, what if you were only going to drive that car 16 more times? And what if you planned to fix the problem before you've driven it even three or four more times? Obviously, that lowers the odds considerably, any statistician will tell you that.
As Wayne Hale explained (pdf), the important thing is not whether it was a good decision, or whether wise counsel was actually followed, but whether lots and lots of people got to ask questions:
"We had everyone engaged. It was a huge room full of people. I think that room holds about 250 people. It was full. There was overflow rooms provided, and I think we answered questions from all comers.
NASA Administrator Mike Griffin said the same thing at that press conference:
The discussion that you heard earlier about that since the day I walked into this position, I have said that we need to regain a culture of openness and a culture of willingness to engage in technical discussion on the merits without it having an adversarial component to it. We did that. If there was anybody in that room who didn't get to have their say, you are going to have to search them out because it was a great two days.
Everyone talked, and since others were nearby it's reasonable to assume everyone was listened to.
But the question the blogging community keeps pestering us about, strangely, is not, "Was it a good discussion?" but "Is there crew risk?" Mike Griffin answered that question categorically, "No, there is only programmatic risk." But can the crew be killed by programmatic risk? Absolutely not! Not anymore. With the new new NASA.
To begin with, the foam projectile problem is not an ascent risk. That foam may blow huge holes in the shuttle thermal protection system, but come on, this is a freaking huge rocket and ascent is not the problem.
So what if the TPS is shot to hell on ascent and the shuttle has no chance of re-entry? (a) the crew has practiced tile repair techniques; (2) there is a rescue option in the "Launch On Need" shuttle; and (3) the crew can hold out in the International Space Station for quite a while.
But wait, you say, Griffin himself later in the same press conference described those repair capabilities as "rudimentary," saying merely that "we are beginning to converge on some rudimentary repair methods that may be useful." Well, it's true the repair option isn't really much of an option at all. But luckily, there's old reliable "Launch on Need":
"OK, Atlantis, it looks like we're gonna have to use you as launch on need."
"Huh. Why is that?"
"Well, Discovery's TPS literally got shredded on ascent."
"Huh. Didn't you say that wouldn't happen?"
"Yep, but it did anyway, how about that?"
"Well, why won't that happen to us?"
"Our analysis shows it won't happen. Only the chief engineer and the safety dude disagree. Everyone else is on board. Literally."
So with no real repair capability, and the LON an equally mythical option, there's still the option of holing up in the ISS. In other words, the principal strategy for the next mission is to cross our fingers. (It worked for the ECO sensor problem on the last flight.)
