Journal
JOURNAL OF PETROLEUM SCIENCE AND ENGINEERING
Volume 217, Issue -, Pages -Publisher
ELSEVIER
DOI: 10.1016/j.petrol.2022.110868
Keywords
Well control optimization; Deep reinforcement learning; Policy transfer; Generalizable optimization
Categories
Funding
- National Natural Science Foundation of China [52074340, 51874335]
- Shandong Provincial Natural Science Foundation [JQ201808]
- Fundamental Research Funds for the Central Universities [18CX02097A]
- Major Scientific and Technological Projects of CNPC [ZD 2019-183-008]
- Science and Technology Support Plan for Youth Innovation of University in Shandong Province [2019KJH002]
- 111 Project [B08028]
Ask authors/readers for more resources
This study proposes an adaptive and robust deep learning-based framework for well control optimization, which addresses the generalization problem. Compared to previous methods, the framework trains a policy that is robust to environmental variations and can adapt to unseen but similar environments.
Well control optimization is a challenging task but plays a critical role in reservoir management. Traditional methods independently solve each task from scratch and the obtained scheme is only applicable to the environment where the optimization process is run. In stark contrast, human experts are adept at learning and building generalizable skills and using them to efficiently draw inferences and make decisions for similar scenarios. Inspired by the recently proposed generalizable field development optimization approach, this work presents an adaptive and robust deep learning-based Representation-Decision-Transfer (DLRDT) framework to deal with the generalization problem in well control optimization. Specifically, DLRDT uses a three-stage workflow to train an artificial agent. First, the agent develops its vision and understands its surroundings by learning a latent state representation with domain adaptation techniques. Second, the agent is tasked with using high-performance deep reinforcement learning algorithms to train the optimal control policy in the latent state space. Finally, the agent is transferred and evaluated in several environments that were not seen during the training. Compared with previous methods that optimize a solution for a specific scenario, our approach trains a policy that is not only robust to variations in their environments but can adapt to unseen (but similar) environments without additional training. For a demonstration, we validate the proposed framework on water-flooding well control optimization problems. Experimental evaluations on two three-dimensional reservoir models demonstrate the trained agent has excellent optimization efficiency and generalization performance. Our approach is particularly favorable when considering the deployment of schemes in the real world as it can handle unforeseen situations.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available