Emerging Trends: SOTA-Chasing

Many papers are chasing state-of-the-art (SOTA) numbers, and more will do so in the future. SOTA-chasing comes with many costs. SOTA-chasing squeezes out more promising opportunities such as coopetition and interdisciplinary collaboration. In addition, there is a risk that too much SOTA-chasing could lead to claims of superhuman performance, unrealistic expectations, and the next AI winter. Two root causes for SOTA-chasing will be discussed: (1) lack of leadership and (2) iffy reviewing processes. SOTA-chasing may be similar to the replication crisis in the scientific literature. The replication crisis is yet another example, like evaluation, of over-confidence in accepted practices and the scientific method, even when such practices lead to absurd consequences.

Automated summary

Pursuing state-of-the-art (SOTA) numbers in research papers can have costs, such as missing out on more promising opportunities and potentially leading to unrealistic expectations. Lack of leadership and uncertain reviewing processes are identified as the root causes of SOTA-chasing. This phenomenon is compared to the replication crisis in scientific literature.