4.7 Article

Classification of Blind Users' Image Exploratory Behaviors Using Spiking Neural Networks

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNSRE.2019.2959555

Keywords

Trajectory; Visualization; Haptic interfaces; Shape; Training; Three-dimensional displays; Biological neural networks; Blind or visually impaired; dempster-Shafer theory; exploration procedures; image perception; spatio-temporal data; spiking neural networks

Funding

  1. Partnerships for Innovation Program through the National Science Foundation [1919214]
  2. Trask Trust fund
  3. Div Of Industrial Innovation & Partnersh
  4. Directorate For Engineering [1919214] Funding Source: National Science Foundation

Ask authors/readers for more resources

Individuals who are blind adopt multiple procedures to tactually explore images. Automatically recognizing and classifying users' exploration behaviors is the first step towards the development of an intelligent system that could assist users to explore images more efficiently. In this paper, a computational framework was developed to classify different procedures used by blind users during image exploration. Translation-, rotation- and scale-invariant features were extracted from the trajectories of users' movements. These features were divided as numerical and logical features and were fed into neural networks. More specifically, we trained spiking neural networks (SNNs) to further encode the numerical features as model strings. The proposed framework employed a distance-based classification scheme to determine the final class/label of the exploratory procedures. Dempster-Shafter Theory (DST) was applied to integrate the distances obtained from all the features. Through the experiments of different dynamics of spiking neurons, the proposed framework achieved a good performance with 95.89% classification accuracy. It is extremely effective in encoding and classifying spatio-temporal data, as compared to Dynamic Time Warping and Hidden Markov Model with 61.30% and 28.70% accuracy. The proposed framework serves as the fundamental block for the development of intelligent interfaces, enhancing the image exploration experience for the blind.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available