Methods and metrics challenges of delivery-system research

Background: Many delivery-system interventions are fundamentally about change in social systems (both planned and unplanned). This systems perspective raises a number of methodological challenges for studying the effects of delivery-system change-particularly for answering questions related to whether the change will work under different conditions and how the change is integrated (or not) into the operating context of the delivery system. Methods: The purpose of this paper is to describe the methodological and measurement challenges posed by five key issues in delivery-system research: (1) modeling intervention context; (2) measuring readiness for change; (3) assessing intervention fidelity and sustainability; (4) assessing complex, multicomponent interventions; and (5) incorporating time in delivery-system models to discuss recommendations for addressing these issues. For each issue, we provide recommendations for how research may be designed and implemented to overcome these challenges. Results and conclusions: We suggest that a more refined understanding of the mechanisms underlying deliverysystem interventions (treatment theory) and the ways in which outcomes for different classes of individuals change over time are fundamental starting points for capturing the heterogeneity in samples of individuals exposed to delivery-system interventions. To support the research recommendations outlined in this paper and to advance understanding of the why and how questions of delivery-system change and their effects, funding agencies should consider supporting studies with larger organizational sample sizes; longer duration; and nontraditional, mixed-methods designs. A version of this paper was prepared under contract with the Agency for Healthcare Research and Quality (AHRQ), US Department of Health and Human Services for presentation and discussion at a meeting on The Challenge and Promise of Delivery System Research, held in Sterling, VA, on February 16-17, 2011. The opinions in the paper are those of the author and do not represent the views or recommendations of AHRQ or the US Department of Health and Human Services.(1)