On the robustness of analysis techniques for molecular detections using high-resolution exoplanet spectroscopy

High-resolution Doppler spectroscopy provides a powerful means for chemical detections in exoplanetary atmospheres. This approach involves monitoring hundreds of molecular lines in the planetary spectrum Doppler shifted by the orbital motion of the planet. The molecules are detected by cross-correlating the observed spectrum of the system with a model planetary spectrum. The method has led to molecular detections of H2O, CO, and TiO in hot Jupiters using large ground-based telescopes. Critical to this method, however, is the accurate removal of the stellar and telluric features from the observed spectrum, also known as detrending. Previous molecular detections have relied on specific choices of detrending methods and parameters. However, the robustness of molecular detections across the different choices has not been investigated in detail. We conduct a systematic investigation of the effect of detrending algorithms, parameters, and optimizations on chemical detections using high-resolution spectroscopy. As a case study, we consider the hot Jupiter HD 189733b. Using multiple methods, we confirm high-significance detections of H2O (4.8 sigma) and CO (4.7 sigma). Additionally, we report evidence for HCN at high significance (5.0 sigma). On the other hand, our results highlight the need for improved metrics and extended observations for robust confirmations of such detections. In particular, we show that detection significances of greater than or similar to 4 sigma can be obtained by optimizing detrending at incorrect locations in the planetary velocity space; such false-positives can occur in nearly 30 per cent of cases. We discuss approaches to help distinguish molecular detections from spurious noise.