期刊
PETROLEUM SCIENCE
卷 20, 期 1, 页码 261-276出版社
KEAI PUBLISHING LTD
DOI: 10.1016/j.petsci.2022.08.016
关键词
Production optimization; Deep reinforcement learning; Evolutionary algorithm; Real-time optimization; Optimization under uncertainty
Production optimization is crucial in the smart oilfield community for maximizing economic benefits and oil recovery. This study proposes an efficient and robust method, evolutionary-assisted reinforcement learning (EARL), to achieve real-time production optimization under uncertainty. The approach models the optimization problem as a Markov decision process and uses a deep convolutional neural network to adaptively adjust well controls based on reservoir states. Simulation results demonstrate that EARL outperforms prior methods in terms of optimization efficiency, robustness, and real-time decision-making capability.
Production optimization has gained increasing attention from the smart oilfield community because it can increase economic benefits and oil recovery substantially. While existing methods could produce high-optimality results, they cannot be applied to real-time optimization for large-scale reservoirs due to high computational demands. In addition, most methods generally assume that the reservoir model is deterministic and ignore the uncertainty of the subsurface environment, making the obtained scheme unreliable for practical deployment. In this work, an efficient and robust method, namely evolutionary -assisted reinforcement learning (EARL), is proposed to achieve real-time production optimization under uncertainty. Specifically, the production optimization problem is modeled as a Markov decision process in which a reinforcement learning agent interacts with the reservoir simulator to train a control policy that maximizes the specified goals. To deal with the problems of brittle convergence properties and lack of efficient exploration strategies of reinforcement learning approaches, a population-based evolutionary algorithm is introduced to assist the training of agents, which provides diverse exploration experiences and promotes stability and robustness due to its inherent redundancy. Compared with prior methods that only optimize a solution for a particular scenario, the proposed approach trains a policy that can adapt to uncertain environments and make real-time decisions to cope with unknown changes. The trained policy, represented by a deep convolutional neural network, can adaptively adjust the well controls based on different reservoir states. Simulation results on two reservoir models show that the proposed approach not only outperforms the RL and EA methods in terms of optimization efficiency but also has strong robustness and real-time decision capacity.(c) 2022 The Authors. Publishing services by Elsevier B.V. on behalf of KeAi Communications Co. Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/ 4.0/).
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据