Switch-GPT: An Effective Method for Constrained Text Generation under Few-Shot Settings (Student Abstract)

In real-world applications of natural language generation, target sentences are often required to satisfy some lexical constraints. However, the success of most neural-based models relies heavily on data, which is infeasible for data-scarce new domains. In this work, we present FewShotAmazon, the first benchmark for the task of Constrained Text Generation under few-shot settings on multiple domains. Further, we propose the Switch-GPT model, in which we utilize the strong language modeling capacity of GPT-2 to generate fluent and wellformulated sentences, while using a light attention module to decide which constraint to attend to at each step. Experiments show that the proposed Switch-GPT model is effective and remarkably outperforms the baselines. Codes will be available at https://github.com/chang-github -00/S witch-GPT.

Automated summary

In this work, the authors present FewShotAmazon, the first benchmark for the task of Constrained Text Generation under few-shot settings on multiple domains. They also propose the Switch-GPT model, which utilizes the language modeling capacity of GPT-2 to generate fluent and well-formulated sentences, while using a light attention module to decide on the constraints to attend to. Experiments show that the proposed model outperforms the baselines.