EvoDNN - An Evolutionary Deep Neural Network with Heterogeneous Activation Functions

Many problems in Computational Biology and Bioinformatics involve classification, such as the classification of cell samples into malignant (cancer) or benign (normal). For such tasks, we propose EvoDNN, an evolutionary deep neural network that employs an evolutionary algorithm to evolve deep heterogeneous feed-forward neural networks. While the majority of current feed-forward neural networks employ user defined homogeneous activation functions, EvoDNN creates heterogeneous multi-layer networks where each neuron's activation function is not statically defined by the user, but dynamically optimized during evolution. The main advantage offered by EvoDNN lies in that the activation functions do not need to be differentiable. This feature gives users a great degree of flexibility over which activation functions EvoDNN can utilize. This paper demonstrates how EvoDNN can simultaneously optimize each neuron's weight, bias, and activation function, and empirically shows a superior performance compared to a backpropagation-trained feed-forward neural network at the cost of additional training time. In addition, advantages of the deep architecture of EvoDNN over our earlier approach, EvoNN, which employed a single hidden layer are discussed.