论文标题
联邦平均水平的收敛理论：超越平滑度
A Convergence Theory for Federated Average: Beyond Smoothness
论文作者
论文摘要
联合学习使大量的边缘计算设备能够学习模型而无需共享数据共享。作为在这种情况下的领先算法，联合的平均FedAvg（在本地设备上并行运行随机梯度下降（SGD）），并且由于其简单性和低通信成本而被广泛使用。然而，尽管最近的研究工作，但在超出平稳性的假设下仍缺乏理论分析。在本文中，我们分析了FedAvg的收敛性。与现有工作不同，我们放松了坚固的平稳性的假设。更具体地说，我们假设损耗函数的半平滑度和半lipschitz属性，在假设定义中具有额外的一阶项。此外，我们还假设绑定在梯度上，该梯度比收敛分析方案中常用的有限梯度假设弱。作为解决方案，本文提供了有关联合学习的理论融合研究。
Federated learning enables a large amount of edge computing devices to learn a model without data sharing jointly. As a leading algorithm in this setting, Federated Average FedAvg, which runs Stochastic Gradient Descent (SGD) in parallel on local devices and averages the sequences only once in a while, have been widely used due to their simplicity and low communication cost. However, despite recent research efforts, it lacks theoretical analysis under assumptions beyond smoothness. In this paper, we analyze the convergence of FedAvg. Different from the existing work, we relax the assumption of strong smoothness. More specifically, we assume the semi-smoothness and semi-Lipschitz properties for the loss function, which have an additional first-order term in assumption definitions. In addition, we also assume bound on the gradient, which is weaker than the commonly used bounded gradient assumption in the convergence analysis scheme. As a solution, this paper provides a theoretical convergence study on Federated Learning.
