Selective decentralization to improve reinforcement learning in unknown linear noisy systems

Abstract

In this paper, we answer the question of to what extend selective decentralization could enhance the learning and control performance when the system is noisy and unknown. Compared to the previous works in selective decentralization, in this paper, we add the system noise as another complexity in the learning and control problem. Thus, we only perform analysis for some simple toy examples of noisy linear system. In linear system, the Halminton-Jaccobi-Bellman (HJB) equation becomes Riccati equation with closed-form solution. Our previous framework in learning and control unknown system is based on the following principle: approximating the system using identification in order to apply model-based solution. Therefore, this paper would explore the learning and control performance on two aspects: system identification error and system stabilization. Our results show that selective decentralization show better learning performance than the centralization when the noise level is low.