The physics of space weather/solar-terrestrial physics (STP): what we know now and what the current and future challenges are

Major geomagnetic storms are caused by unusually intense solar wind southward magnetic fields that impinge upon the Earth's magnetosphere (Dungey, 1961). How can we predict the occurrence of future interplanetary events? Do we currently know enough of the underlying physics and do we have sufficient observations of solar wind phenomena that will impinge upon the Earth's magnetosphere? We view this as the most important challenge in space weather. We discuss the case for magnetic clouds (MCs), interplanetary sheaths upstream of interplanetary coronal mass ejections (ICMEs), corotating interaction regions (CIRs) and solar wind high-speed streams (HS Ss). The sheath- and CIR-related magnetic storms will be difficult to predict and will require better knowledge of the slow solar wind and modeling to solve. For interplanetary space weather, there are challenges for understanding the fluences and spectra of solar energetic particles (SEPs). This will require better knowledge of interplanetary shock properties as they propagate and evolve going from the Sun to 1 AU (and beyond), the upstream slow solar wind and energetic seed particles. Dayside aurora, triggering of nightside substorms, and formation of new radiation belts can all be caused by shock and interplanetary ram pressure impingements onto the Earth's magnetosphere. The acceleration and loss of relativistic magnetospheric killer electrons and prompt penetrating electric fields in terms of causing positive and negative ionospheric storms are reasonably well understood, but refinements are still needed. The forecasting of extreme events (extreme shocks, extreme solar energetic particle events, and extreme geomagnetic storms (Carrington events or greater)) are also discussed. Energetic particle precipitation into the atmosphere and ozone destruction are briefly discussed. For many of the studies, the Parker Solar Probe, Solar Orbiter, Magnetospheric Multiscale Mission (MMS), Arase, and SWARM data will be useful.