Distributed Q-Learning-Based Online Optimization Algorithm for Unit Commitment and Dispatch in Smart Grid

Economic dispatch (ED) and unit commitment (UC) problems need to be revisited in order to make a transition from a traditional power system to a smart grid. In this paper, we formulate the ED and UC problems into a unified form, which is also capable of characterizing the infinite horizon UC problem. Based on the formulation, a centralized Q-learning-based optimization algorithm is proposed. The proposed algorithm runs in an online manner and requires no prior information on the mathematical formulation of the actual cost functions, thus being capable of dealing with situations for which such cost functions are too difficult to obtain. Then, the distributed counterpart of the centralized algorithm is developed by relaxing the demand for global information and balancing exploration and exploitation cooperatively in a distributed way. Theoretical analysis of the proposed algorithms is also provided. Finally, several case studies are presented to demonstrate the effectiveness of the proposed algorithms.