期刊
IEEE ROBOTICS AND AUTOMATION LETTERS
卷 4, 期 4, 页码 3884-3891出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/LRA.2019.2926223
关键词
Deep learning in robotics and automation; domestic robots
类别
资金
- Japan Science and Technology Agency CREST
- SCOPE
- NEDO
In this letter, we address multimodal language understanding with unconstrained fetching instruction for domestic service robots. A typical fetching instruction such as Bring me the yellow toy from the white shelf requires to infer the user intention, i.e., what object (target) to fetch and from where (source). To solve the task, we propose a multimodal target-source classifier model (MTCM), which predicts the region-wise likelihood of target and source candidates in the scene. Unlike other methods, MTCM can handle region-wise classification based on linguistic and visual features. We evaluated our approach that outperformed the state-of-the-art method on a standard dataset. We also extended MTCM with generative adversarial nets, and enabled simultaneous data augmentation and classification.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据