Codes and Expansions (CodEx) Seminar

Demetrio Labate (University of Houston):
Provable approximations of multivariate functions on smooth manifolds using deep ReLU neural networks

The expressive power of deep neural networks is manifested by their remarkable ability to approximate multivariate functions in a way that appears to overcome the curse of dimensionality. This ability is exemplified by their success in solving high-dimensional problems where traditional numerical solvers fail due to their limitations in accurately representing high-dimensional structures. To provide a theoretical framework for explaining this phenomenon, we analyze the approximation of Hoelder functions defined on a d-dimensional smooth manifold M embedded in \(\mathbb{R}^D\), with \(d \gg D\), using deep neural networks. We prove that the uniform convergence estimates of the approximation and generalization errors by deep neural networks with ReLU activation functions do not depend on the ambient dimension \(D\) of the function but only on its lower manifold dimension \(d\), in a precise sense. This result improves existing results from the literature where approximation and generalization errors were shown to depend weakly on \(D\).

This is a joint work with Ji Shi.