4.7 Article

Geometrical Interpretation and Design of Multilayer Perceptrons

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNNLS.2022.3190364

Keywords

Feedforward design; interpretable machine learning; interpretable multilayer perceptron (MLP); MLP; neural networks

Funding

  1. U.S. Army Research Laboratory (ARL) [W911NF2020157]

Ask authors/readers for more resources

This work interprets the multilayer perceptron (MLP) neural network from a geometrical viewpoint and proposes a new three-layer feedforward MLP (FF-MLP) architecture for its implementation. Experiments show that the FF-MLP outperforms the traditional backpropagation-based MLP (BP-MLP) in terms of design time, training time, and classification performance.
The multilayer perceptron (MLP) neural network is interpreted from the geometrical viewpoint in this work, that is, an MLP partition an input feature space into multiple nonoverlapping subspaces using a set of hyperplanes, where the great majority of samples in a subspace belongs to one object class. Based on this high-level idea, we propose a three-layer feedforward MLP (FF-MLP) architecture for its implementation. In the first layer, the input feature space is split into multiple subspaces by a set of partitioning hyperplanes and rectified linear unit (ReLU) activation, which is implemented by the classical two-class linear discriminant analysis (LDA). In the second layer, each neuron activates one of the subspaces formed by the partitioning hyperplanes with specially designed weights. In the third layer, all subspaces of the same class are connected to an output node that represents the object class. The proposed design determines all MLP parameters in a feedforward one pass fashion analytically without backpropagation. Experiments are conducted to compare the performance of the traditional backpropagation-based MLP (BP-MLP) and the new FF-MLP. It is observed that the FF-MLP outperforms the BP-MLP in terms of design time, training time, and classification performance in several benchmarking datasets.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available