A critical problem in benchmarking and analysis of evolutionary computation methods

Benchmarking is a cornerstone in the analysis and development of computational methods, especially in the field of evolutionary computation, where theoretical analysis of the algorithms is almost impossible. In this Article, we show that some of the frequently used benchmark functions have their respective optima in the centre of the feasible set and that this poses a critical problem for the analysis of evolutionary computation methods. We carry out an analysis of seven recently published methods and find that these contain a centre-bias operator that lets them find optima in the centre of the benchmark set with ease. However, this mechanism makes their comparison with other methods (that do not have a centre-bias) meaningless. We compare the computational performance of these seven new methods to two long-standing ones in evolutionary computation ('differential evolution' and 'particle swarm optimization') on shifted problems and on more advanced benchmark problems. Only one of the seven methods performed consistently better than the pair of old methods, three performed on par, two performed very badly and the worst one performed barely better than a random search. We provide several suggestions that could help to improve analysis and benchmarking in evolutionary computation.

Automated summary

The article highlights the critical issue in evolutionary computation where some frequently used benchmark functions have their optima in the center of the feasible set, posing challenges in algorithm analysis. Through analyzing seven recently published methods, it was found that the presence of a center-bias operator enables easy identification of optima in the center of the benchmark set, rendering comparisons with methods lacking this bias meaningless. The computational performance comparison of these methods with established ones like 'differential evolution' and 'particle swarm optimization' showed varied results with only one new method consistently outperforming the old ones.