Journal
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
Volume 25, Issue 1, Pages 288-298Publisher
IEEE COMPUTER SOC
DOI: 10.1109/TVCG.2018.2864504
Keywords
Deep Q-Network (DON); reinforcement learning; model interpretation; visual analytics
Categories
Funding
- US Department of Energy Los Alamos National Laboratory [47145]
- UT-Battelle LLC [4000159447]
- NSF [IIS-1250752, IIS-1065025]
- US Department of Energy [DE-SC0007444, DE-DC0012495]
Ask authors/readers for more resources
Deep Q-Network (DON). as one type of deep reinforcement learning model. targets to train an intelligent agent that acquires optimal actions while interacting with an environment. The model is well known for its ability to surpass professional human players across many Atari 2600 games. Despite the superhuman performance, in-depth understanding of the model and interpreting the sophisticated behaviors of the DQN agent remain to be challenging tasks, due to the long-time model training process and the large number of experiences dynamically generated by the agent. In this work, we propose DQNViz, a visual analytics system to expose details of the blind training process in four levels, and enable users to dive into the large experience space of the agent for comprehensive analysis. As an initial attempt in visualizing DQN models, our work focuses more on Atari games with a simple action space, most notably the Breakout game. From our visual analytics of the agent's experiences, we extract useful action/reward patterns that help to interpret the model and control the training. Through multiple case studies conducted together with deep learning experts, we demonstrate that DQNViz can effectively help domain experts to understand. diagnose, and potentially improve DQN models.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available