EE 680	OPTIMIZATION METHODS FOR CONTROL AND COMMUNICATION

Instructor: Prof. Edwin K. P. Chong

Text: E. K. P. Chong and S. H. Zak, An Introduction to Optimization,
Second Edition, New York, NY: John Wiley & Sons, Inc.,
(Wiley-Interscience Series), 2001.  ISBN 0-471-39126-3.

Grading: Homeworks 15%, mid-term exams 50%, final exam 35%.

Prerequisites by Topic:  (1) Elements of linear algebra (matrix
manipulations, vector spaces, bases, eigenvalues); (2) Calculus of
several variables (differentiating functions of several variables,
chain rule, gradients, Taylor series, limits).

Course Description:  Introduction to optimization theory and methods,
with applications in systems, control, and communication.  Nonlinear
unconstrained optimization, linear programming, nonlinear constrained
optimization, various algorithms and search methods for optimization,
and their analysis.  Examples from various engineering applications are
given.

Course Outcomes:  A student who successfully fulfills the course
requirements will have demonstrated:
(i) an ability to formulate optimization problems and identify possible
solutions to such problems;
(ii) an ability to apply and analyze basic linear and nonlinear
optimization algorithms;
(iii) a background needed to understand more advanced techniques for
solving challenging optimization problems;
(iv) an ability to make formal and rigorous arguments in analyzing
optimization problems and solution techniques.

Course Outline:

1. Introduction (1 week)
 1.1 Motivating examples
 1.2 Mathematical preliminaries

2. Unconstrained optimization (4 weeks)
 2.1 First and second order conditions
 2.2 Algorithms for unconstrained optimization
     One dimensional search methods; gradient methods; Newton methods; 
     conjugate direction methods; quasi-Newton methods

3. Least squares analysis (1 week)
 3.1 Examples and basic properties
 3.2 Recursive least squares algorithm

4. Random search algorithms (1 week)
 4.1 Simulated annealing
 4.2 Genetic algorithms

5. Linear programming (2.5 weeks)
 5.1 Examples and basic properties
 5.2 Simplex method
 5.3 Duality

6. Nonlinear constrained optimization (4 weeks)
 6.1 Lagrange and second order conditions
 6.2 Karush-Kuhn-Tucker and second order conditions

7. Convex optimization (1.5 weeks)
 7.1 Convexity
 7.2 Optimality conditions