On the approximation by neural networks with bounded number of neurons in hidden layers

In the current note, we show that a two hidden layer neural network with d inputs, d neurons in the first hidden layer, 2d+2 neurons in the second hidden layer and with a specifically constructed sigmoidal and infinitely differentiable activation function can approximate any continuous multivariate function with arbitrary accuracy. (C) 2014 Elsevier Inc. All rights reserved.