Distributed Networked Learning with Correlated Data

This paper considers a learning problem with heteroscedastic and correlated data that is distributed across nodes. We propose a distributed learning scheme where each node asynchronously implements stochastic gradient descent updates and exchanges their current models with neighbors. We ensure the similarity among the local models and the ensemble average by having a network regularization penalty to the least squares problem. This penalty is associated with weights that are proportional to the relative accuracy of local models. We further provide finite time characterization of the disparity between local models and the ensemble average model based on the penalty constants and network connectivity. We compare the proposed method with generalized least squares and logistic regression in the prediction of activities of individuals based on head movement data.