2022 Program for Women and Mathematics: The Mathematics of Machine Learning

Melanie Weber, University of Oxford
4:30 pm - 4:50 pm
Title: Geometric Methods for Machine Learning and Optimization
Abstract: A key challenge in machine learning and optimization is the identification of geometric structure in high-dimensional data. Such structural understanding is of great value for the design of efficient algorithms and for developing fundamental guarantees for their performance. This talk will discuss research on utilizing Riemannian geometry in machine learning and data science, motivated by the observation that many applications involve non-Euclidean data, such as graphs, strings, or matrices. First, we consider the task of learning a classifier in hyperbolic space. Such spaces have received a surge of interest for representing large-scale, hierarchical data, since they achieve better representation accuracy with fewer dimensions. Secondly, we consider the problem of optimizing a function on a Riemannian manifold. Specifically, we will consider classes of optimization problems where exploiting Riemannian geometry can deliver algorithms that are computationally superior to standard (Euclidean) approaches.

Anna Ma, University of California, Irvine
4:50 pm - 5:10 pm
Title: Approaches for working with Large-Scale Data
Abstract: Data is available at larger scales than ever before seen, making it an important task to be able to implement and understand the tools and techniques that utilize this data. For example, if data is too large to fit into a computer's memory, how can one use it to train a machine learning algorithm? What if you have the ability to pick and choose which data you want to utilize? How can you pick the best data for your problem? Also, what if your data goes beyond the standard data-as-a-matrix setting and you have multidimensional data? What can be done then? In this brief talk, we will speak broadly to these questions and discuss motivations, approaches, and open directions for working with large-scale data.

Courtney Paquette, McGill University
5:10 pm - 5:30 pm
Title: Optimization Algorithms in the Large
Abstract: In this talk, I will present a framework, inspired by random matrix theory, for analyzing the dynamics of optimization algorithms (e.g., 1^st-order methods, stochastic gradient descent (SGD), and momentum) when both the number of samples and dimensions are large. Using this new framework, we show that the dynamics of optimization algorithms on a least squares problem with random data become deterministic in the large sample and dimensional limit. In particular, the limiting dynamics for stochastic algorithms are governed by a Volterra integral equation. This model predicts that SGD undergoes a phase transition at an explicitly given critical stepsize that ultimately affects its convergence rate, which we also verify experimentally. Finally, when input data is isotropic, we provide explicit expressions for the dynamics and average-case convergence rates. These rates show significant improvement over the worst-case complexities.