SILO



The weekly SILO Seminar Series is made possible through the generous support of the 3M Company and its Advanced Technology Group

3M

with additional support from the Analytics Group of the Northwestern Mutual Life Insurance Company

Northwestern Mutual

Speeding up Machine Learning using Graphs and Codes

Dimitris Papailiopoulos, Assistant Professor, Department of Electrical and Computer Engineering, University of Wisconsin-Madison

Date and Time: Sep 28, 2016 (12:30 PM)
Location: Orchard room (3280) at the Wisconsin Institute for Discovery Building

Abstract:

Video: https://vimeo.com/185052120

I will talk about three simple combinatorial ideas to speed up parallel and distributed learning algorithms. We will start off with serial equivalence in asynchronous parallel ML, its significance, and how we can guarantee it using a recent phase transition result on graphs. We continue on the issue of stragglers (i.e., slow nodes) in distributed systems, where we will use erasure codes to robustify gradient based algorithms against delays. In our third example, we will reduce the high communication cost of data-shuffling in distributed learning, using the seemingly unrelated notion of coded caching. For all the above, we will see real world experiments that indicate how these simple ideas can significantly improve the speed and accuracy of large-scale learning.