Fishers Linear Discriminant Analysis With Space-Folding Operations

Fisher's linear discriminant analysis (LDA) is an easy-to-use supervised dimensionality reduction method. However, LDA may be ineffective against complicated class distributions. It is well-known that deep feedforward neural networks with rectified linear units as activation functions can map many input neighborhoods to similar outputs by a succession of space-folding operations. This short paper shows that the space-folding operation can reveal to LDA classification information in the subspace where LDA cannot find any. A composition of LDA with the space-folding operation can find classification information more than LDA can do. End-to-end fine-tuning can improve that composition further. Experimental results on artificial and open data sets have shown the feasibility of the proposed approach.

Automated summary

Fisher's linear discriminant analysis (LDA) is a supervised dimensionality reduction method that may be ineffective against complicated class distributions. This paper demonstrates that deep neural networks with rectified linear units can reveal classification information in a subspace where LDA cannot find any. Combining LDA with space-folding operations and fine-tuning can enhance the classification performance. Experimental results on artificial and open data sets validate the feasibility of the proposed approach.