4.7 Article

Distributed networked learning with correlated data

Journal

AUTOMATICA
Volume 137, Issue -, Pages -

Publisher

PERGAMON-ELSEVIER SCIENCE LTD
DOI: 10.1016/j.automatica.2021.110134

Keywords

Large scale optimization problems and methods; Network-based computing systems; Learning theory; Statistical analysis; Parameter and state estimation; Multi-agent systems

Ask authors/readers for more resources

This paper considers a distributed estimation method for heterogeneous streams of correlated data distributed across nodes in a network. In this method, linear models are locally estimated with a network regularization term that penalizes differences between local and neighboring models. The paper analyzes computation dynamics and information exchange and provides a finite-time characterization of convergence for the weighted ensemble average estimate, comparing it to federated learning.
We consider a distributed estimation method in a setting with heterogeneous streams of correlated data distributed across nodes in a network. In the considered approach, linear models are estimated locally (i.e., with only local data) subject to a network regularization term that penalizes a local model that differs from neighboring models. We analyze computation dynamics (associated with stochastic gradient updates) and information exchange (associated with exchanging current models with neighboring nodes). We provide a finite-time characterization of convergence of the weighted ensemble average estimate and compare this result to federated learning, an alternative approach to estimation wherein a single model is updated by locally generated gradient updates. This comparison highlights the trade-off between speed vs precision: while model updates take place at a faster rate in federated learning, the proposed networked approach to estimation enables the identification of models with higher precision. We illustrate the method's general applicability in two examples: estimating a Markov random field using wireless sensor networks and modeling prey escape behavior of flocking birds based on a publicly available dataset. (C) 2021 Elsevier Ltd. All rights reserved.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available