PCTS Seminar Series: Deep Learning for Physics

Please Note: The seminars are not open to the general public, but only to active researchers. Register here for this event: https://docs.google.com/forms/d/e/1FAIpQLScJ-BUVgJod6NGrreI26pedg8wGEyP… Abstract for talk #1: The affinity between statistical physics and machine learning has long history, this is reflected even in the machine learning terminology that is in part adopted from physics. I will describe the main lines of this long-lasting friendship in the context of current theoretical challenges and open questions about deep learning. Theoretical physics often proceeds in terms of solvable synthetic models, I will describe the related line of work on solvable models of simple feed-forward neural networks. I will highlight a path forward to capture the subtle interplay between the structure of the data, the architecture of the network, and the learning algorithm. Abstract for talk #2: What interplay of dynamics, architecture, and data make good generalization possible in overparameterized neural networks? Approaches from statistical physics have shed light on this question by considering a variety of simple limiting cases. I will describe results emerging from two simple models: deep linear neural networks and nonlinear student-teacher networks. In these models, good generalization from limited data arises from aspects of training dynamics and initialization. Finally, I will briefly tour open problems facing practitioners that seem amenable to analysis with similar methods.