Deep Learning-based Dementia Prediction using Multimodal Data

Abstract

In this project we propose a deep learning architecture to predict dementia, a disease which affects around 55 million people all over the world and makes them in some cases dependent people. The main aim is to predict the disease in the early stages, in order to start having professional treatment from the beginning, which can improve the quality of life of the patients. Another aim is to analyze how the combination of different modalities, like audio or text, can influence the results obtained by the model. Many research has been done over the different available dementia datasets as well as the classification tasks with audio and text data. To this end, we have used the DementiaBank dataset, which includes audio recordings as well as their transcriptions of healthy people and people with dementia. Different models have been used and tested, including Convolutional Neural Networks for the audio classification, Transformers for the text classification and a combination of both models in a multimodal one. These models have been tested over a test set, obtaining the best results from the text modality, achieving a 90.36% of accuracy on the detection of dementia task.