Assessing long-term user experience on a mobile health application through an in-app embedded conversation-based questionnaire

A satisfying experience is critical for the user to feel motivated over time, especially in mobile health (m-Health) applications. Long-term user experience (UX) assesses more extended periods of use, which may reveal the possible impact of a set of experiences. In this study, using an in-app embed questionnaire available through a conversational interface, we investigated long-term UX on an m-Health application to identify how it occurs over time. The methods were developed in 4 phases, which included problem characterization, search for UX aspects, elaboration and verification of the questionnaire, test period with questionnaire applied before, during and after use, interview with users, and information mapping. For 3 months, 37 users reported their experiences with the application, describing their impressions regarding usage. Results demonstrated that the most satisfying experiences occurred mainly within the first weeks of the study and were associated with the app's features, charts and visual resources, and the practicality of treatment monitoring. For the less satisfactory experiences, the main reasons identified were technical problems on the application, and the effort and difficulty of use. All users appreciated the use of a character in the conversational interface as a gatherer of the answers to the assessment questionnaire.