Sampling design begets conclusions: the statistical basis for detection of injury to and recovery of shoreline communities after the 'Exxon Valdez' oil spill

The joint effect of multiple initial decisions made about sampling design in evaluation of environmental impacts using observational field assessments influences the ability to detect and accurately estimate responses. The design can dictate in advance whether the study can identify even large impacts that truly exist. Following the 'Exxon Valdez' oil spill in 1989, 4 separate studies of effects of the spill on the intertidal biota were conducted. Studies overlapped sufficiently in geographic area, shoreline habitat, and biological response variables to permit contrasts showing how the aggregate of multiple design decisions led to differences in conclusions. The SEP (ShoreLine Ecology Program) supported by Exxon and the CHIA (Coastal Habitat Injury Assessment) funded by the Exxon Valdez Oil Spill Trustee Council shared a core approach of establishing a stratified random design of site selection. The Exxon-supported GOA (Gulf of Alaska) study and the NOAA (National Oceanographic and Atmospheric Administration) Hazmat (Hazardous Materials) study both chose to employ subjective choices of fixed sites. Despite many common goals, these 4 studies differed greatly in: (1) sampling effort (area covered per sample quadrat, sample replication within sites, numbers of study sites per category, numbers of samplings, and total areas sampled) and sampling design (philosophy of targeting sampling effort, complete randomization versus matched pair designs, sampling frame, treatment of habitat heterogeneity within sites, interspersion of sites, and control of shoreline treatment and oiling intensity); (2) analytical methodology (analysis of covariance versus paired designs, treatment of subsamples as replicates in F-ratios, logic of inferring recovery, and power calculations); and (3) choice of biological response variables (taxonomic level of analysis. aggregating versus spitting separate communities, and scope of communities and habitats examined). The CHIA and NOAA Hazmat studies of epibiotic responses in sheltered rocky shores of Prince William Sound made several decisions to enhance detection power and produced similar conclusions about large reductions in total biotic cover of intertidal space, Fucus cover, mussel abundance, abundance of the Limpet Tectura persona and a balanoid barnacle, and increases in open space and abundance of an opportunistic barnacle, Chthamalus dalli. In contrast, the SEP study of this same habitat and geographic region adopted design choices resulting in lower power of detection in 12 (vs CHIA and vs NOAA Hazmat) of 15 separate decisions (with one tie in each contrast). Accordingly, the SEP study was able to detect declines only in Fucus cover and occasionally in total Limpet abundance but not in total epifaunal or mussel or balanoid barnacle abundance and, unlike the results of the other 2 studies, most of the taxa analyzed showed apparent increases rather than decreases from oiling and shoreline treatment. The more powerful GOA and CHIA studies of impacts of oiling in the Gulf of Alaska, where oil grounded 1 to 8 wk later and in more weathered condition than in Prince William Sound, showed more consistent and larger reductions in intertidal biota in the sheltered rocky habitat than did the SEP study of Prince William Sound. Thus, the combined effects of many design decisions that reduced power to detect impacts in the SEP study led to failure to demonstrate large impacts of the spill documented by other studies of the same habitat in the same and the more remote region.