Distributing control of deep learning training delivers 10x performance improvement
My IBM Research AI team and I recently completed the first formal theoretical study of the convergence rate and communications complexity associated with a decentralized distributed approach in a deep learning training setting. ...