Journal
APPLIED ENERGY
Volume 304, Issue -, Pages -Publisher
ELSEVIER SCI LTD
DOI: 10.1016/j.apenergy.2021.117733
Keywords
Deep reinforcement learning; Deep deterministic policy gradient algorithm; Smart buildings; Control systems; Distributed energy resources; Load flexibility; Energy efficiency
Categories
Funding
- Building Technologies Office, of the U.S. Department of Energy [DE-AC02-05CH11231]
- Office of Science, Office of Basic Energy Sciences, of the U.S. Department of Energy [DE-AC02-05CH11231]
Ask authors/readers for more resources
The study highlights the effectiveness of distributed energy resources in supporting carbon reduction goals and increasing grid reliability. Traditional control algorithms are inadequate for optimizing across systems in commercial buildings, while deep reinforcement learning offers a potentially effective solution. The research applied a deep reinforcement learning-based control method in practice, resulting in significant cost savings.
Behind-the-meter distributed energy resources (DERs), including building solar photovoltaic (PV) technology and electric battery storage, are increasingly being considered as solutions to support carbon reduction goals and increase grid reliability and resiliency. However, dynamic control of these resources in concert with traditional building loads, to effect efficiency and demand flexibility, is not yet commonplace in commercial control products. Traditional rule-based control algorithms do not offer integrated closed-loop control to optimize across systems, and most often, PV and battery systems are operated for energy arbitrage and demand charge management, and not for the provision of grid services. More advanced control approaches, such as MPC control have not been widely adopted in industry because they require significant expertise to develop and deploy. Recent advances in deep reinforcement learning (DRL) offer a promising option to optimize the operation of DER systems and building loads with reduced setup effort. However, there are limited studies that evaluate the efficacy of these methods to control multiple building subsystems simultaneously. Additionally, most of the research has been conducted in simulated environments as opposed to real buildings. This paper proposes a DRL approach that uses a deep deterministic policy gradient algorithm for integrated control of HVAC and electric battery storage systems in the presence of on-site PV generation. The DRL algorithm, trained on synthetic data, was deployed in a physical test building and evaluated against a baseline that uses the current best-in-class rule-based control strategies. Performance in delivering energy efficiency, load shift, and load shed was tested using price-based signals. The results showed that the DRL-based controller can produce cost savings of up to 39.6% as compared to the baseline controller, while maintaining similar thermal comfort in the building. The project team has also integrated the simulation components developed during this work as an OpenAIGym environment and made it publicly available so that prospective DRL researchers can leverage this environment to evaluate alternate DRL algorithms.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available