期刊
IEEE TRANSACTIONS ON SIGNAL PROCESSING
卷 66, 期 12, 页码 3240-3255出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TSP.2018.2830299
关键词
Distributed algorithms; machine learning algorithms; adaptive algorithms; nonlinear filters; artificial intelligence; statistical learning
资金
- NSF [CCF-1017454, CCF-0952867]
- ONR [N00014-12-1-0997]
- ARL MAST CTA
- ASEE SMART
We consider multiagent stochastic optimization problems over reproducing kernel Hilbert spaces. In this setting, a network of interconnected agents aims to learn decision functions, i.e., nonlinear statistical models, that are optimal in terms of a global convex functional that aggregates data across the network, with only access to locally and sequentially observed samples. We propose solving this problem by allowing each agent to learn a local regression function while enforcing consensus constraints. We use a penalized variant of functional stochastic gradient descent operating simultaneously with low-dimensional subspace projections. These subspaces are constructed greedily by applying orthogonal matching pursuit to the sequence of kernel dictionaries and weights. By tuning the projection-induced bias, we propose an algorithm that allows each individual agent to learn, based on its locally observed data stream and message passing with its neighbors only, a regression function that is close to the globally optimal regression function. That is, we establish that with constant stepsize selections agents' functions converge to a neighborhood of the globally optimal one while satisfying the consensus constraints as the penalty parameter is increased. Moreover, the complexity of the learned regression functions is guaranteed to remain finite. On both multiclass kernel logistic regression and multiclass kernel support vector classification with data generated from class-dependent Gaussian mixture models, we observe an stable function estimation and the state-of-the-art performance for distributed online multiclass classification. Experiments on the Brodatz textures further substantiate the empirical validity of this approach.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据