Development and Validation of the Guide for Effective Nutrition Interventions and Education (GENIE): A Tool for Assessing the Quality of Proposed Nutrition Education Programs

Objective: To develop and validate the Guide for Effective Nutrition Interventions and Education (GENIE), a checklist of research-based quality indicators for nutrition education programs. Design: A prospective test of criterion validity and inter-rater reliability of a new tool comparing expert assessments and trained reviewer GENIE scores of the same nutrition education proposals. Participants: Ten nutrition education experts; 13 volunteer reviewers. Variables Measured: GENIE's face, content, and criterion validity and inter-rater reliability compared using expert assessments and reviewer objective and subjective scores. Analysis: Reviewer scores compared using Spearman correlation. Inter-rater reliability tested using intra-class correlation (ICC), Cronbach alpha, and ANOVA. Criterion validity tested using independent t test and point bi-serial correlation to compare reviewer with expert scores. Results: Correlation found between total objective and total subjective scores. Agreement found between reviewers across proposals and categories considering subjective scores (F = 7.21, P < .001; ICC = 0.76 [confidence interval, 0.53-0.92]) and objective scores (F = 7.88, P < .001; ICC = 0.82 [confidence interval, 0.63-0.94]). Relationship was not significant (r = .564, P = .06) between expert and reviewer proposal scoring groups (high, medium, and low). Conclusions and Implications: Results support the validity and reliability of GENIE as a tool for nutrition education practitioners, researchers, and program funding agencies to accurately assess the quality of a variety of nutrition program plans.