Workshop on Theory of Deep Learning: Where next?
The event was live-streamed.
Organizers:
The workshop was organized by Sanjeev Arora (IAS/Princeton University), Joan Bruna (IAS/NYU), Rong Ge (IAS/Duke), Suriya Gunasekar(IAS/Toyota Technical Institute), Jason Lee (IAS/USC), Bin Yu (IAS/UC Berkeley)
This workshop sought to bring together deep learning practitioners and theorists to discuss progress that has been made on deep learning theory, and to identify promising avenues where theory is possible and useful. There were several invited talks each day and also spotlight talks by young researchers.
The workshop was free of charge thanks to the support from the Institute for Advanced Study and the Schwab Charitable Fund made possible by the generosity of Eric and Wendy Schmidt.
Invited Speakers who confirmed participation:
Anima Anandkumar, Raman Arora, Sanjeev Arora, Mikhail Belkin, Léon Bottou, Joan Bruna, Michael Collins, Simon Du, Gintare Karolina Dziugaite, Surya Ganguli, Rong Ge, Suriya Gunasekar, Stefanie Jegelka, Chi Jin, Sham Kakade, Yann LeCun, Jason Lee, Ke Li, Tengyu Ma, Aleksander Madry, Chris Manning, Behnam Neyshabur, Dan Roy, Nathan Sbrero, Rachel Ward, Bin Yu
Agenda:(all talks were in Wolfensohn Hall)
October 15, 2019
REGISTRATION: Opening at 8:30 am - 9:30 am in Wolfensohn Hall and will remain open until 3 pm
| Workshop on Theory of Deep Learning: Where next? | |
| Topic: | Emergent linguistic structure in deep contextual neural word representations slides video |
| Speaker: | Chris Manning, Stanford University |
| Time/Room: | 9:30am - 10:10am/Wolfensohn Hall |
| Topic: | Explaining landscape connectivity of low-cost solutions for multilayer nets video |
| Speaker: | Rong Ge, Duke University; Member, School of Mathematics |
| Time/Room: | 10:10am - 10:40am/Wolfensohn Hall |
| Topic: | Fixing GAN optimization through competitive gradient descent video |
| Speaker: | Anima Anandkumar, Caltech |
| Time/Room: | 11:10am - 11:50am/Wolfensohn Hall |
| Topic: | Tightening information-theoretic generalization bounds with data-dependent estimates with an application to SGLD video |
| Speaker: | Daniel Roy, University of Toronto |
| Time/Room: | 11:50am - 12:20pm/Wolfensohn Hall |
| Topic: | Spotlight Talks: Yuanzhi Li, Soham De, Mahyar Fazlyab, Maithra Raghu, Valentin Thomas video |
| Speaker: | Various |
| Time/Room: | 12:20pm - 1:00 pm |
| Topic: | Is optimization the right language to understand deep learning? video |
| Speaker: | Sanjeev Arora, Princeton University; Distinguished Visiting Professor, School of Mathematics |
| Time/Room: | 2:30pm - 3:10pm/Wolfensohn Hall |
| Topic: | Spotlight Talks: Amir Asadi, Dimitris Kalimeris video |
| Speaker: | Various |
| Time/Room: | 3:10pm - 3:40pm/Wolfensohn Hall |
| Topic: | PAC-Bayesian approaches to understanding generalization in deep learning video slides |
| Speaker: | Gintare Karolina Dziugaite, Simons Institute for the Theory of Computing |
| Time/Room: | 4:00pm - 4:30pm/Wolfensohn Hall |
| Topic: | Overcoming the Curse of Dimensionality and Mode Collapse video slides |
| Speaker: | Ke Li, University of California, Berkeley |
| Time/Room: | 4:30pm - 5:00pm/Wolfensohn Hall |
| Topic: | Are All Features Created Equal? video slides |
| Speaker: | Aleksander Madry, Massachusetts Institute of Technology |
| Time/Room: | 5:00pm - 5:40pm/Wolfensohn Hall |
October 16, 2019
| Topic: | Energy-based Approaches to Representation Learning slides video |
| Speaker: | Yann LeCun, NYU and Facebook AI |
| Time/Room: | 9:30am - 10:10am/Wolfensohn Hall |
| Topic: | On Large Deviation Principles for Large Neural Networks video slides |
| Speaker: | Joan Bruna, New York University |
| Time/Room: | 10:10am - 10:40am/Wolfensohn Hall |
| Topic: | Neural Models for Speech and Language: Successes, Challenges, and the Relationship to Computational Models of the Brain video |
| Speaker: | Michael Collins, Columbia University |
| Time/Room: | 11:10am - 11:50am/Wolfensohn Hall |
| Topic: | On the Connection between Neural Networks and Kernels: a Modern Perspective video |
| Speaker: | Simon Du, Member, School of Mathematics |
| Time/Room: | 11:50am - 12:20pm/Wolfensohn Hall |
| Topic: | Hike in Institute Woods |
| 12:20 pm-1:00 pm | |
| Topic: | From Classical Statistics to Modern ML: the Lessons of Deep Learning video slides |
| Speaker: | Mikhail Belkin, Ohio State University |
| Time/Room: | 2:30pm - 3:10pm/Wolfensohn Hall |
| Topic: | Spotlight Talks: Vaishnavh Nagarajan, Preetum Nakkiran, Xiaowu Dai, Weijie Su video |
| Speaker: | Various |
| Time/Room: | 3:10pm-3:40pm/Wolfensohn Hall |
| Topic: | Towards a theoretical foundation of neural networks video |
| Speaker: | Jason Lee, Princeton University; Member, School of Mathematics |
| Time/Room: | 4:00pm - 4:30pm/Wolfensohn Hall |
| Topic: | Panel Session |
| Time/Room: | 4:30pm - 5:30pm/Wolfensohn Hall |
October 17, 2019
| Topic: | Learning Representations Using Causal Invariance video |
| Speaker: | Leon Bottou, Facebook AI Research |
| Time/Room: | 9:30am - 10:10am/Wolfensohn Hall |
| Topic: | Understanding the inductive bias due to dropout video |
| Speaker: | Raman Arora, Johns Hopkins University; Member, School of Mathematics |
| Time/Room: | 10:10am - 10:40am/Wolfensohn Hall |
| Topic: | Interpreting Deep Neural Networks video slides |
| Speaker: | Bin Yu, University of California, Berkeley |
| Time/Room: | 11:10am - 11:50am/Wolfensohn Hall |
| Topic: | Designing explicit regularizers for deep models video slides |
| Speaker: | Tengyu Ma |
| Time/Room: | 11:50am - 12:20pm/Wolfensohn Hall |
| Topic: | Spotlight Talks: Arjun Nitin Bhagoji, Jiaoyang Huang, Rosemary Ke, Or Sharir, Omar Shehab |
| Speaker: | Various video |
| Time/Room: | 12:20pm - 1:00pm/Wolfensohn Hall |
| Topic: | Kernel and Rich Regimes in Deep Learning video |
| Speaker: | Nati Srebro, TTIC |
| Time/Room: | 2:30pm - 3:10pm/Wolfensohn Hall |
| Topic: | Spotlight Talks: Sebastian Goldt video |
| Speaker: | Various |
| Time/Room: | 3:10pm - 3:40pm/Wolfensohn Hall |
| Topic: | Provably Efficient Reinforcement Learning with Linear Function Approximation video |
| Speaker: | Chi Jin, Member, School of Mathematics |
| Time/Room: | 4:00pm - 4:30pm/Wolfensohn Hall |
| Topic: | Poster Session |
| Speaker: | Various |
| Time/Room: | 4:30pm - 5:30pm/Wolfensohn Hall |
October 18, 2019
| Topic: | Reinforcement Learning, Deep Learning, and the Role of Policy Gradient Methods video |
| Speaker: | Sham Kakade, University of Washington |
| Time/Room: | 9:30am - 10:10 am/Wolfensohn Hall |
| Topic: | Statistical Mechanics of Machine Learning video |
| Speaker: | Surya Ganguli, Stanford University |
| Time/Room: | 10:10am - 10:40am/Wolfensohn Hall |
| Topic: | Concentration inequalities for random matrix products |
| Speaker: | Rachel Ward, The University of Texas at Austin; von Neumann Fellow, School of Mathematics |
| Time/Room: | 11:10am - 11:50am/Wolfensohn Hall |
| Topic: | Representational Power of Graph Neural Networks video slides |
| Speaker: | Stefanie Jegelka, Massachusetts Institute of Technology |
| Time/Room: | 11:50am - 12:20pm/Wolfensohn Hall |
| Topic: | Spotlight Talks: Zhiyuan Li, John Zarka, Stanislav Fort video |
| Speaker: | Various |
| Time/Room: | 12:20pm - 1:00pm/Wolfensohn Hall |
| Topic: | Toward a Causal Analysis of Generalization in Deep Learning video |
| Speaker: | Behnam Neyshabur, Google |
| Time/Room: | 2:30pm - 3:00pm/Wolfensohn Hall |
| Topic: | Spotlight Talks: Zhifeng Kong, Daniel Paul Kunin, Omar Montasser video |
| Speaker: | Various |
| Time/Room: | 3:10pm - 3:40pm/Wolfensohn Hall |
| Topic: | Informal discussion sessions |
| Time/Room: | 3:40pm - 5:30pm/Wolfensohn Hall |
~end
Contributed talks: Deadline was Sept 2
A few shorter slots were given to showcase late-breaking results and work by young researchers (grads and postdocs).
Selected papers were invited as either talks or posters. Notification deadline was Sept 10.
Date & Time
October 15, 2019 | 9:00am – October 18, 2019 | 6:00am