Thu, 24.11.2022 11:00

Solution Concepts for Optimal Feedback Control of Nonlinear Differential Equations: How to win against the curse of dimensionality

Group Seminar: Optimization and Optimal Control Group

Karl Kunisch, RICAM and University of Graz

Optimal feedback controls for nonlinear systems are characterized by the solutions to a Hamilton Jacobi Bellmann (HJB)  equation. In the deterministic case, this is a first order hyperbolic equation.
Its dimension is that of the statespace of the nonlinear system. Thus solving the HJB equation is a formidable task and one is confronted with a curse of dimensionality.

In practice, optimal feedback controls are frequently  based on linearisation and subsequent treatment by efficient Riccati solvers. This can be effective, but it is  local procedure, and it may fail or lead to erroneous results.

In this talk, I  give a  brief survey of current  solution strategies to partially cope with this challenge. Subsequently I describe three  approaches in some detail. The first one  is a data driven technique, which approximates  the solution to the  HJB equation and its gradient from an ensemble of open loop solves.
The second one is based on Newton steps applied to the HJB equation. Combined with tensor calculus which allows to approximately solve HJB equations up to dimension 100. Results are shown for the control of discretized Fokker Planck equations. The third technique circumvents the direct solution of the HJB equation. Rather a neural network is trained by means of a succinctly chosen ansatz. It is proven that it approximates the optimal feedback gains  as the dimension of the network is increased.

This work relies on collaborations with B.Azmi, S.Dolgov, D.Kalise, D.Vasquez, and D.Walter.