New patent allows real-time learning based on previous decisions

UT Arlington electrical engineers have patented an innovative method that improves a controller's ability to make real-time decisions.

Frank Lewis, a professor in the Electrical Engineering Department and at the UT Arlington Research Institute, along with Draguna Vrabie, a senior scientist at United Technologies, and Kyriakos Vamvoudakis, a faculty project research scientist at the University of California, Santa Barbara, were recently issued a patent entitled, "Control Methodology for Online Adaptation to Optimal Feedback Controller Using Integral Reinforcement Learning."

Lewis and his collaborators developed Integral Reinforcement Learning, which is a process by which a device learns and makes control decisions in reaction to a set of variables that changes based on each previous decision. IRL learns by a batch process that requires taking of a set of data prior to updating the control law. This patent develops a new technology for IRL by which decisions can be made continuously in real time, online, allowing greater autonomy and faster response.

"Optimal feedback controllers allow a device to use the minimum energy necessary while saving time and fuel," Lewis said. "They are often seen in aircraft autopilots, emission controls in vehicles, and other similar technologies. The advantage of using adaptive integral reinforcement learning is that a device now can have optimal controls by looking within the system and calculating changes in real time, rather than offline where changes cannot be made until the device is no longer in use."

Vamvoudakis added that adaptive integral reinforcement may eventually allow machines to work in increasingly difficult situations.

"Efficiency will be defined by the potential to adapt autonomously in decentralized, unknown and complex environments to enable capabilities beyond human limits," Vamvoudakis said.

Khosrow Behbehani, dean of the UT Arlington College of Engineering, said the patent could have many financial and environmental benefits.

"By increasing the efficiency of various modes of transportation with the IRL smart controller, we make it possible to move goods and deliver services less expensively and with less negative impact on our environment," he said.