We implement the paper Communication-Efficient Learning of Deep Networks from Decentralized Data. Its blog is here.
We wish to explore several possible approaches:
- server pulling gradient update from clients;
- clients eagerly train and push updates to server;
- extend to multi-server; servers gossip to sync;
- client-only p2p gradient updates