3.8 Proceedings Paper

GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-supervised Learning and Explicit Policy Injection

Publisher

ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE

Keywords

-

Funding

  1. National Natural Science Foundation of China [61906185]
  2. Youth Innovation Promotion Association of CAS China [2020357]
  3. Shenzhen Science and Technology Innovation Program [KQTD20190929172835662]
  4. Shenzhen Basic Research Foundation [JCYJ20200109113441941]
  5. Alibaba Group through Alibaba Research Intern Program

Ask authors/readers for more resources

In this paper, a novel pre-trained dialog model called GALAXY is proposed, which learns dialog policy from limited labeled dialogs and large-scale unlabeled dialog corpora via semi-supervised learning. Empirical results show that GALAXY substantially improves the performance of task-oriented dialog systems and achieves new state-of-the-art results on benchmark datasets.
Pre-trained models have proved to be powerful in enhancing task-oriented dialog systems. However, current pre-training methods mainly focus on enhancing dialog understanding and generation tasks while neglecting the exploitation of dialog policy. In this paper, we propose GALAXY, a novel pre-trained dialog model that explicitly learns dialog policy from limited labeled dialogs and large-scale unlabeled dialog corpora via semi-supervised learning. Specifically, we introduce a dialog act prediction task for policy optimization during pre-training and employ a consistency regularization term to refine the learned representation with the help of unlabeled dialogs. We also implement a gating mechanism to weigh suitable unlabeled dialog samples. Empirical results show that GALAXY substantially improves the performance of task-oriented dialog systems, and achieves new state-of-the-art results on benchmark datasets: In-Car, MultiWOZ2.0 and MultiWOZ2.1, improving their end-to-end combined scores by 2.5, 5.3 and 5.5 points, respectively. We also show that GALAXY has a stronger few-shot ability than existing models under various low-resource settings. For reproducibility, we release the code and data at https://github.com/siat-nlp/GALAXY.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

3.8
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available