Isolated Sign Language Recognition with Depth Cameras

In this paper, an approach to isolated sign language recognition with data provided by a depth camera is presented. In the introduced method, sequences of depth maps of dynamic sign language gestures are divided into smaller regions (cells). Then, statistical information is used to describe the cells. Since gesture executions have different lengths, the dynamic time warping (DTW) technique with the nearest neighbor (NN) rule is often employed for their comparison. However, due to time-consuming computations, the DTW limits the usability of the classifier. Therefore, in this paper, a selection of representative depth maps using statistics for cells is proposed. It is shown that such gesture representation can be successfully employed for isolated sign language recognition with the NN classifier using the city block distance. Furthermore, the NN rule with the DTW and the introduced statistics for cells provides superior gesture recognition performance. (C) 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://crativecommons.org/licenses/by-nc-nd/4.0) Peer-review under responsibility of the scientific committee of KES International.

Automated summary

This paper presents an approach to isolated sign language recognition using data from a depth camera. By dividing depth map sequences of dynamic sign language gestures into smaller regions and using statistical information, the method improves gesture recognition performance with the NN classifier in combination with DTW technique.