Codes and Expansions (CodEx) Seminar

Alex Sietsema (UCLA)
A Geometric Approach to Overfitting

Some machine learning models exhibit benign overfitting, where a model is capable of both perfectly interpolating a training dataset as well as generalizing well to unseen data. Recent work in overparameterized machine learning has studied benign overfitting in a variety of settings. We study overfitting in a general, fixed dimensional setting, and show that mild assumptions on the structure of data and our models produce the opposite result: models which interpolate the data and are almost as smooth as possible must have generalization error that does not vanish, no matter how much data we use. We discuss the background surrounding this area, the result and its implications, and give an overview of the novel geometric proof methodology.