Efficient reinforcement learning with partial observables for fluid flow control

Even if the trajectory in a viscous flow system stays within a low dimensional subspace in the state space, reinforcement learning (RL) requires many observables in the active control problem. This is because the observables are assumed to follow a policy-independent Markov decision process in the usual RL framework and full observation of the system is required to satisfy this assumption. Although RL with a partially observable condition is generally a difficult task, we construct a consistent algorithm with the condition using the low dimensional property of viscous flow. Using typical examples of active flow control, we show that our algorithm is more stable and efficient than the existing RL algorithms, even under a small number of observables.

Automated summary

This study presents a consistent RL algorithm based on the low-dimensional property of viscous flow, which is shown to be more stable and efficient than existing algorithms, even with a small number of observables.