Clayton Shonkwiler

Math 676

Instructor: Dr. Clayton Shonkwiler
Time: Monday, Wednesday, Friday 2:00–2:50
Location: Eddy 11
Office: Weber 206C
Office Hours: By appointment
Email Address: clayton.shonkwiler@colostate.edu

Syllabus


Overview

Information geometry has (slightly poetically) been described as “the geometry of decision making”; it is the story of a natural differential geometric structure possessed by families of probability distributions, and provides a coherent conceptual framework for parameter estimation, including performance bounds like the classical Cramér–Rao bound.

The first main goal of this course is to understand the Fisher information metric, which is a Riemannian metric on the manifold parametrizing a family of probability distributions induced by the Fisher information matrix. In turn, this metric induces an intrinsic distance, sometimes called the Fisher–Rao distance, which in general has desirable features (like symmetry) that other notions of distance between probability distributions (like the Kullback–Leibler divergence) lack.

From there, we will go deeper on the special structure of information manifolds, particularly conjugate connections, divergences, and the Legendre transformation. This will set us up for a differential geometric approach to statistical inference (e.g., trying to determine which distribution in a family your observed data came from).

The main running example throughout all of this is the family of multivariate Gaussian distributions, which is simple and concrete enough to allow for explicit calculations while still being of practical relevance. In this case one can see that the corresponding manifold is hyperbolic space, and as distance goes to zero the (symmetrized) Kullback–Leibler divergence limits to the intrinsic distance.

In general, the manifolds coming from exponential families (which are the focus of most of the literature) have global coordinates, though this is not true for general manifolds. So an optimistic potential outcome of the class is some insight into the situation when parameters satisfy manifold constraints, including orthogonality constraints.

There is no official text for the course, but the following books and papers will be useful resources: