The decision to launch the Space Shuttle Challenger
02/07/2016In the second chapter of the classic "Visual & Statistical Thinking" Edward Tufte wrote extensively about his critique of the failure of analysis that was present in the moments leading up to the 1986 Challenger disaster, in which seven astronauts tragically died aboard a space shuttle which broke apart 73 seconds into flight. The explosion was traced to the failure of a rubberized "O-ring" on one of the shuttle's booster rockets for the flight, a failure which was in turn traced to the exceptionally low temperature (29 °F, just below freezing—a rare cold snap in Florida) at the time of the launch.
Engineers at Thiokol, the manufacturer of the rockets, had reason to be concerned, before the flight took place, that the temperatures were too low for a safe flight. Their initial recommendation was to hold off the launch, and in the ensuing conferences between the engineers and managers at Thiokol on the one hand and at NASA on the other, they provided thirteen hand-written charts arguing against the launch.
Ultimately, however, go fever won out: after much wrangling those in the room that were favor of the flight were able to convince the dissenting engineers that there was no association between the previous history of O-ring damage and temperature, and that the data they had collected was not convincing enough to stay the launch. In this manner, Tufte believes, the ensuing disaster hinges on the inability of those suspicious engineers to adequately present and argue the data that they had collected before their assembled colleagues. Had the data been presented better, he argues, then the true narrative ought to have been convincing enough to move the launch to another day.
Tufte's criticism has its own criticism, including this article by Roger Boisjoly. Nevertheless, borrowing one of the visual ideas that Tufte hypothesizes for constructing a more complete and convincing argument, here is a chart of the past history of temperature at launch set against the amount of damage the O-rings suffered, gradated in red. The leftmost launch in particular, STS-51-C, suffered nearly fatal damage, with a complete burn-through of the primary O-ring and bad charring on the backup secondary O-ring.
Visualizations of this sort have a way of establishing the conversation about what they depict. A good visualization (like, I hope, this one) provides its user with a fast, visual way of summarizing the data that it presents, making the questions that it is attempting to address and the connections that it draws from them obvious and palatable. Compare this to the kind of charts that Thiokol brought to the meeting:
Damage data is not actually presented here: it was provided in more handwritten notes on the side of the page.
And compare further with the chart that was brought before the subsequent federal commission:
One further data point is missing from our chart, as it has nothing to say about the exceptionality of the low temperature at the time of the Challenger's launch, fully 24° F colder than any previous launch temperature. If we throw that data in, and shrink the rest of the visualization to accompany it, this is the result:
I rest my case.