Topological Regularization for Representation Learning via Persistent Homology

Generalization is challenging in small-sample-size regimes with over-parameterized deep neural networks, and a better representation is generally beneficial for generalization. In this paper, we present a novel method for controlling the internal representation of deep neural networks from a topological perspective. Leveraging the power of topology data analysis (TDA), we study the push-forward probability measure induced by the feature extractor, and we formulate a notion of separation to characterize a property of this measure in terms of persistent homology for the first time. Moreover, we perform a theoretical analysis of this property and prove that enforcing this property leads to better generalization. To impose this property, we propose a novel weight function to extract topological information, and we introduce a new regularizer including three items to guide the representation learning in a topology-aware manner. Experimental results in the point cloud optimization task show that our method is effective and powerful. Furthermore, results in the image classification task show that our method outperforms the previous methods by a significant margin.

Automated summary

This paper presents a novel method for controlling the internal representation of deep neural networks from a topological perspective, providing a theoretical framework for better generalization. By studying the push-forward probability measure induced by the feature extractor and quantifying the notion of separation using persistent homology, the authors propose a new weight function and a topology-aware regularizer to enforce this property. Experimental results demonstrate the effectiveness and superiority of the proposed method in point cloud optimization and image classification tasks.