期刊
COMPUTATIONAL LINGUISTICS
卷 34, 期 4, 页码 487-511出版社
MIT PRESS
DOI: 10.1162/coli.2008.07-028-R2-05-82
关键词
-
类别
资金
- European Commission under the FP6 project TALK: Talk and Look, Tools for Ambient Linguistic Knowledge [507802]
- FP7 project CLASSIC: Computational Learning in Adaptive Systems for Spoken Conversation [216594]
- EPSRC [EP/E019501/1]
- SHEFC [HR04016]
- EPSRC [EP/E019501/1] Funding Source: UKRI
- Engineering and Physical Sciences Research Council [EP/E019501/1] Funding Source: researchfish
We propose a method for learning dialogue management policies from a fixed data set. The method addresses the challenges posed by Information State Update (ISU)-based dialogue systems, which represent the state of a dialogue as a large set of features, resulting in a very large state space and a huge policy space. To address the problem that any fixed data set will only provide information about small portions of these state and policy spaces, we propose a hybrid model that combines reinforcement learning with supervised learning. The reinforcement learning is used to optimize a measure of dialogue reward, while the supervised learning is used to restrict the learned policy to the portions of these spaces for which we have data. We also use linear function approximation to address the need to generalize from a fixed amount of data to large state spaces. To demonstrate the effectiveness of this method on this challenging task, we trained this model on the COMMUNICATOR corpus, to which we have added annotations for user actions and Information States. When tested with a user simulation trained on a different part of the same data set, our hybrid model outperforms a pure supervised learning model and a pure reinforcement learning model. It also outperforms the hand-crafted systems on the COMMUNICATOR data, according to automatic evaluation measures, improving over the average COMMUNICATOR system policy by 10%. The proposed method will improve techniques for bootstrapping and automatic optimization of dialogue management policies from limited initial data sets.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据