Eventually, all you need is a simple evolutionary algorithm (for neuroevolution of continuous control policies)

Artificial neural networks (ANNs) are a popular choice for tackling continuous control tasks due to their approximation abilities. When the ANN architecture is fixed, finding optimal weights becomes a numerical optimization problem, suitable for evolutionary algorithms (EAs), i.e., a form of neuroevolution. Here, we compare the performance of well-established EAs in solving neuroevolution problems, focusing on continuous control. We evaluate them on a set of navigation problems and a set of control problems based on modular soft robots. As a reference, we compare the same EAs on regression problems and classic numerical optimization benchmarks. Our findings suggest that simple EAs like genetic algorithm (GA) and differential evolution (DE) achieve good performance on control problems, even if they are surpassed by more sophisticated algorithms on benchmark problems. We hypothesize that the effectiveness of these simpler EAs stems from their use of crossover, which can be advantageous in the rugged fitness landscapes encountered in complex control tasks.