Journal
IEEE TRANSACTIONS ON MULTIMEDIA
Volume 23, Issue -, Pages 2782-2793Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TMM.2020.3016222
Keywords
Sensor fusion; Task analysis; Cameras; Semantics; Event detection; Image sensors; Physical social fusion; multimodal and multisource analysis; situation awareness; spatial temporal filtering
Categories
Funding
- National Research Foundation, Prime Ministers Office, Singapore under its International Research Centre in Singapore Funding Initiative
Ask authors/readers for more resources
This paper proposes a novel unified matrix factorization-based model to fuse physical and social sensor signals for spatio-temporal analysis, addressing challenges caused by data noise and heterogeneous data. Experimental results demonstrate that the proposed approach performs better in various situational understanding tasks.
Our world is witnessing the on-going substantial increase in the number of multimodal physical and social sensors that are ubiquitously distributed and observing or reporting what is happening in their surroundings. These sensors provide massive amounts of spatio-temporal digital footprints which can be analyzed for various tasks such as event detection or situation awareness. However, inherent noise due to the nature of these sensors result in imprecise data and hence imprecise analysis. Also, the heterogeneous data from different modalities, formats and sources make interpreting different levels of information a big challenge. To overcome these limitations, we propose a novel unified matrix factorization-based model to fuse physical and social sensor signals for spatio-temporal analysis. Readings of physical sensor signals are represented by a spatio-temporal situation matrix, which then incorporates social content that can provide explanations for the signal strengths. We test our framework on large-scale real-world data including PSI stations data, traffic CCTV camera images, and tweets for situation prediction as well as for filtering noise to detect events of diverse situations. The experimental results suggest that the proposed matrix factorization approach can utilize the sources correlation, resulting in better performances in various situational understanding tasks.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available