Inverse Optimal Control with Regular Language Specifications

I. Papusha, M. Wen, and U. Topcu

Abstract

Given a description of system dynamics, input constraints, and a cost function, the problem of optimal control is to find a sequence of inputs and a state trajectory that minimizes the total cost. A related problem posed by Kalman in the 1960s, called the inverse problem of optimal control, is to determine the objective function given optimal inputs and state trajectories.

In this work, we pose the inverse problem of optimal control under temporal behavior specifications. In our setting, we are given demonstrations of optimal trajectories of a finite transition system, which may come from an expert, simulation, or some other process. In addition to these demonstrations, we are also given extra side information that optimal trajectories must satisfy temporal behavior constraints, expressed as an automaton over the state labels. We explore the value of this temporal side information in imputing an approximately optimal policy from finitely many demonstrations, and give a gridworld example with an eye toward extending the framework to hybrid systems with continuous states.

Citation

I. Papusha, M. Wen, and U. Topcu. “Inverse Optimal Control with Regular Language Specifications,” American Control Conference (ACC), pp. 770–777, Milwaukee, WI, June 27–29, 2018.