2018年4月9日
Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis
21st International Conference on Artificial Intelligence and Statistics (AISTATS2018)
- ,
- ,
- 巻
- PMLR 84
- 号
- 開始ページ
- 269
- 終了ページ
- 278
- 記述言語
- 英語
- 掲載種別
- 研究論文(国際会議プロシーディングス)
Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions. The present paper proposes a Riemannian stochastic quasi-Newton algorithm with variance reduction (R-SQN-VR). The key challenges of averaging, adding, and subtracting multiple gradients are addressed with notions of retraction and vector transport. We present convergence analyses of R-SQN-VR on both non-convex and retraction-convex functions under retraction and vector transport operators. The proposed algorithm is evaluated on the Karcher mean computation on the symmetric positive-definite manifold and the low-rank matrix completion on the Grassmann manifold. In all cases, the proposed algorithm outperforms the state-of-the-art Riemannian batch and stochastic gradient algorithms.