A deep-learning based multimodal system for Covid-19 diagnosis using breathing sounds and chest X-ray images

Covid-19 has become a deadly pandemic claiming more than three million lives worldwide. SARS-CoV-2 causes distinct pathomorphological alterations in the respiratory system, thereby acting as a biomarker to aid its diagnosis. A multimodal framework (Ai-CovScan) for Covid-19 detection using breathing sounds, chest X-ray (CXR) images, and rapid antigen test (RAnT) is proposed. Transfer Learning approach using existing deep-learning Convolutional Neural Network (CNN) based on Inception-v3 is combined with Multi-Layered Perceptron (MLP) to develop the CovScanNet model for reducing false-negatives. This model reports a preliminary accuracy of 80% for the breathing sound analysis, and 99.66% Covid-19 detection accuracy for the curated CXR image dataset. Based on Ai-CovScan, a smartphone app is conceptualised as a mass-deployable screening tool, which could alter the course of this pandemic. This app's deployment could minimise the number of people accessing the limited and expensive confirmatory tests, thereby reducing the burden on the severely stressed healthcare infrastructure. (C) 2021 Elsevier B.V. All rights reserved.

Automated summary

The study proposes a multimodal framework called Ai-CovScan for Covid-19 detection using breathing sounds, chest X-ray images, and rapid antigen tests. They developed a model called CovScanNet using transfer learning and Multi-Layered Perceptron to reduce false negatives. The model achieves an accuracy of 80% for breathing sound analysis and 99.66% for Covid-19 detection on the CXR image dataset.