When the chick hits the fan: Representativeness and reproducibility in technological tests

Before a new turbojet engine design is approved, the Federal Aviation Administration (FAA) must assure themselves that, among many other things, the engine can safely ingest birds. They do this by mandating a series of well-defined - if somewhat Pythonesque -'birdstrike tests' through which the manufacturers can demonstrate the integrity of their engines. In principle, the tests are straightforward: engineers run an engine at high speed, launch birds into it, and watch to see if it explodes. In practice, the tests rest on a complex and contentious logic. In this paper I explore the debate that surrounds these tests, using it to illustrate the now-familiar idea that technological tests like scientific experiments - unavoidably contain irreducible ambiguities that require judgments to bridge, and to show that these judgments can have real consequences. Having established this, I then explore how the FAA reconciles the unavoidable ambiguities with its need to determine, with a high degree of certainty, that the engines will be as safe as Congress requires. I argue that this reconciliation requires a careful balance between the opposing virtues of reproducibility and representativeness - and that this balance differs significantly from that in most scientific experiments, and from the common perception of what it ought to be.