A Systematic Review and Taxonomy of Published Quality Criteria Related to the Evaluation of User-Facing eHealth Programs

The purpose of this review was to identify and classify key criteria concepts related to the evaluation of user-facing eHealth programs. In line with the PRISMA statement methodology, computer searches of relevant databases were conducted for studies published between January 1, 2000 and March 1, 2016 that contained explicit quality criteria related to mHealth and eHealth products. Reference lists of included articles, review articles, and grey literature (e.g., books, websites) were searched for additional sources. A team of nine experts led by the first author was gathered to support the classification of these criteria. Identified criteria were extracted, grouped and organized using an inductive thematic analysis. Eighty-four sources - emanating from 26 different courtiers - were included in this review. The team extracted 454 criteria that were grouped into 11 quality domains, 58 criteria concepts and 134 concepts' sub-groups. Quality domains were: Usability, Visual Design, User Engagement, Content, Behavior Change/Persuasive Design, Influence of Social Presence, Therapeutic Alliance, Classification, Credibility/Accountability, and Privacy/Security. Findings suggest that authors around the globe agree on key criteria concepts when evaluating user-facing eHealth products. The high proportion of new published criteria in the second half of this review time-frame (2008-2016), and more specifically, the high proportion of criteria relating to persuasive design, therapeutic alliance and privacy/security within this time-frame, points to the advancements made in recent years within this field.