Department of Applied Mathematics and Statistics, Colorado School of Mines
Title
Efficient Training of Infinite-Depth Neural Networks via Jacobian-Free Backpropagation
Abstract
A promising trend in deep learning replaces fixed depth models by approximations of the limit as network depth approaches infinity. This approach uses a portion of network weights to prescribe behavior by defining a limit condition. This makes network depth implicit, varying based on the provided data and an error tolerance. Moreover, existing implicit models can be implemented and trained with fixed memory costs in exchange for additional computational costs. In particular, backpropagation through implicit networks requires solving a Jacobian-based equation arising from the implicit function theorem. We propose a new Jacobian-free backpropagation (JFB) scheme that circumvents the need to solve Jacobian-based equations while maintaining fixed memory costs. This makes implicit depth models much cheaper to train and easy to implement. Numerical experiments on classification are provided.
Department of Mathematics, University of California, Los Angeles
Title
Robust Optimization-based Solvers and Smooth Reformulations for 3D Contact
Abstract
Contact is ubiquitous and often unavoidable, and yet
modeling contacting systems continues to stretch the limits of
available computational tools. In part, this is due to the unique
hurdles posed by contact problems. Several intricately intertwined
physical and geometric factors make contact computations hard,
especially in the presence of friction and nonlinear elasticity. In
this talk, I will discuss our recent work on a new optimization-based
finite element solver, which is constructed for mesh-based
discretizations of nonlinear elastodynamic problems supporting large
nonlinear deformations, implicit time-stepping with contact and
friction. Built on top of a smooth barrier reformulation and a custom
Newton-type optimization, it is a first-of-its-kind "plug-and-play"
contact simulation framework that provides convergent and
unconditionally feasible intersection-free trajectories. The method is
greatly useful for applications in 3D animations, movie visual
effects, and video games. The scheme also enables future studies of
differentiable simulations of nonsmooth physics-constrained inverse
problems in design, control, and robotics.
