Academy of Mathematics and Systems Science, CAS Colloquia & Seminars
Speaker:
Senior Researcher Lin XIAO, Microsoft Research
Inviter:
Yuhong Dai
Title:
Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss
Time & Venue:
2015.6.23 15:30-16:30 Z311
Abstract:
We consider distributed convex optimization problems originated from sample average approximation of stochastic optimization, or empirical risk minimization in machine learning. We propose a communication-efficient distributed algorithm, which requires a small number of communication rounds to reach a specified optimization precision. The algorithm is based on an inexact damped Newton method, where the inexact Newton steps are computed by a distributed preconditioned conjugate gradient method. We analyze its iteration complexity and communication efficiency for minimizing self-concordant empirical loss functions, and discuss the results for several popular machine learning tasks. In a standard setting for supervised learning where the problem condition number grows with the total sample size, the required number of communication rounds of our algorithm does not increase with the sample size, but only grows slowly with the number of machines in the distributed system. This is joint work with Yuchen Zhang.
Appendix：
Copyright@2008, All Rights Reserved, Academy of Mathematics and Systems Science, CAS
Tel: 86-10-82541777 Fax: 86-10-82541972 E-mail: contact@amss.ac.cn
