Eigen-guided deep metric learning

This paper proposes a new approach to improve the optimization performance of gradient-based optimizers for training neural networks in metric learning. In metric learning, the neural networks use a linear activation in their output layer to generate real-valued data embeddings. Hence, the optimal model parameters can be deterministically calculated with respect to the output layer on the basis of the eigenvalue decomposition. These optimal model parameters that were computed by the eigenvalue decomposition were referred to eigenguidance. The training performance of a gradient-based optimizer is boosted by replacing the current model parameters with the eigen-guidance at each pre-defined number of epochs. In addition, the optimal number of embedding dimensions is determined directly while computing the eigen-guidance without additional complexities. In our experiments, gradient-based optimizers with the eigen-guidance converged significantly faster than the gradient-based optimizers alone in metric learning tasks.

Automated summary

This paper proposes a new approach to improve the optimization performance of gradient-based optimizers in metric learning by using eigenguidance, which is calculated based on eigenvalue decomposition. The experiments showed that the gradient-based optimizers with eigenguidance converged significantly faster than those without in metric learning tasks.