CEDAR Electrodynamics Thermosphere Ionosphere (ETI) Challenge for systematic assessment of ionosphere/thermosphere models: Electron density, neutral density, NmF2, and hmF2 using space based observations

In an effort to quantitatively assess the current capabilities of Ionosphere/Thermosphere (IT) models, an IT model validation study using metrics was performed. This study is a main part of the CEDAR Electrodynamics Thermosphere Ionosphere (ETI) Challenge, which was initiated at the CEDAR workshop in 2009 to better comprehend strengths and weaknesses of models in predicting the IT system, and to trace improvements in ionospheric/thermospheric specification and forecast. For the challenge, two strong geomagnetic storms, four moderate storms, and three quiet time intervals were selected. For the selected events, we obtained four scores (i.e., RMS error, prediction efficiency, ratio of the maximum change in amplitudes, and ratio of the maximum amplitudes) to compare the performance of models in reproducing the selected physical parameters such as vertical drifts, electron and neutral densities, NmF2, and hmF2. In this paper, we present the results from comparing modeled values against space-based measurements including NmF2 and hmF2 from the CHAMP and COSMIC satellites, and electron and neutral densities at the CHAMP satellite locations. It is found that the accuracy of models varies with the metrics used, latitude and geomagnetic activity level.