Request for Information: The HEALing Communities Study

Detailed Responses by Topic

This appendix contains a condensed inventory of selected specific suggestions and comments provided by respondents, organized according to the four thematic areas upon which the RFI was based. Items included are meant to be representative and not comprehensive.

Study Design

Many responses suggested ways to select communities for study; estimate reasonable duration of interventions; collect baseline data; think about confounding variables; and encourage collaboration, coordination, and data sharing.

Defining “heavily affected communities” for study

• Must include out-of-care populations
• Include online communities
• A level granular enough to account for such divergence, such as at the level of contiguous zip codes or census tracts 
• Diversity in age and ethnicity, as well as rural and urban
• Three communities may not be sufficient
• A community that statistically ranked above average for OUD across the continuum of prescription opioids, heroin and synthetics 
• Focus on specific populations, such as incarcerated individuals, veterans, and seniors
• CDC’s “high risk” counties could be used
• Include minorities and disadvantaged communities in the context of this crisis, and be mindful of disparities in resource allocation, contact with law enforcement, and medicalization vs. criminalization of substance use
• Using county-level data, it is possible to extract expected opioid death rates given the quantifiable aspects of a community.  From this, counties can be classified by the relationship between expected and observed death rates:
  • stability counties, where the overdose rate approaches the expected level,
  • momentum counties, where the overdose rate is above the expected level,
  • powder keg counties, where the overdose rate is below the expected level with reason to expect a precipitous increase, and
  • lessons-learned counties, where the overdose rate is below the expected level due to efficacious response & prevention. 
  • Both momentum counties and powder keg counties may be considered "heavily affected," though they will appear quite distinct.  They will require distinct intervention approaches.

Study designs for real-world care settings

• RCTs, cluster randomized designs, and natural experiments
• Mixed methods should be applied to collect quantitative and qualitative data, including secondary data analysis, longitudinal comparative data analysis, surveys and interviews
• Research designs should be community-informed/engaged, preferably multi-organization
• Use the PCORI Engagement Rubric
• Harm reduction providers will drive specific outcomes, while treatment/healthcare providers will drive others. These two groups of providers must be coordinated and have equal seats at the design table
• Include sex, gender, and gender identity as variables
• Don’t exclude pregnant women
• Children deserve to be a focus of the study with an eye towards implementing new models of care that support the whole family in myriad dimensions
• Include the chronic pain community
• People who use drugs can and should be incorporated as paid research assistants and have the ability to generate productive research questions and data collection.
• A difference-in-differences model may be appropriate, provided comparison data are available from non-intervention communities or key metrics can be tracked retrospectively
• Agent based modelling could be considered
• Qualitative methods are needed to understand the data in context
• Use adaptive and pragmatic evaluation methods that can assess comparative delivery strategies with real-time feedback loops to make mid-course corrections
• Focus not on what works, but rather on how to best deliver an intervention that works. Focusing on real-world effectiveness and sustainability, combined with study designs that permit attribution of observed outcomes to the program or intervention of interest, will improve the dissemination and uptake of effective interventions
• A fluid, mixed model evaluative and longitudinal approach to test integrated approaches
• Include qualitative inquiries, including those related to law, education, and public health along with environmental scans to identify existing interventions and metrics
• A cascade of analysis studies, starting with descriptive and inferential statistics, moving to relational and, finally, predictive statistics to associate predictors, with outcomes
• Use predictive models to predict outcomes, such as successful cure of OUD, and use those predictions to risk-adjust specific patients for those specific outcomes – to fairly compare the effectiveness of various treatments for OUD. Estimate the risk of a specific outcome among a specific cohort of patients before judging the effectiveness of a treatment intervention in producing that treatment outcome
• The proposed study needs millions of patients, standardized data collection and analysis to calculate their rates and ratios, and risk adjustment for the outcomes of interest. Some of the requirements, such as percent of patients screened for opioid misuse, will depend on how screening is captured in the billing system and the medical record. This study should call for a large consortium of health systems that already share a vast database of patients, clinical findings, medications prescribed and services rendered.
• Researchers can create hypotheses through retrospective analysis of tens of millions of people, and a survey of the published literature. This would eliminate the need to start from scratch in three cities with a patient population too small for predictive modeling.
• Study design can be left to the researcher but should be some type of staggered design with some randomization/stepped wedge, smart design, experiments (policy comparison) description, and natural. Not just RCT.
• Use RCTs and Pretest-posttest (pre-post) designs
• Quasi-experimental interrupted time series with matched comparison communities, in which outcomes are measured repeatedly in the intervention and comparison communities. One county gets the intervention first and the other waits until later in the program to get it if it is effective.

Estimating effect size

• Comprehensive metrics should leverage some rational weighting among outcomes to both establish baseline and evaluate success of integrated interventions over time.
• Rates of non-fatal and fatal OD, and prevalence and incidence of opioid misuse can be calculated on small population size – though to make the findings generalize accurately to many different communities, patient populations in the hundreds of thousands is needed. The Effect size depends on the number of predictors used to calculate the probabilities of specific outcomes.
• We recommend a national scope and data on tens of millions of people to generate and test hypotheses.
• A measure of reporting fidelity will help to characterize the otherwise unobserved effects of improved overdose reporting during the study period.
• Measuring outcomes in small communities is difficult. Year-to-year results in such regions are thus not particularly meaningful. Given the need for rapid development and deployment of interventions, it would make the most sense to include primarily larger regions, where intervention effects are more rapidly detectable. With a sufficiently large and well-resolved community, 5-7 months of post-deployment data should be sufficient to detect any findings.
• The effect size should be assessed through percentage-based cascades of care for OUD and related conditions (e.g., neonatal opioid withdrawal syndrome [NOWS]). Cascades of care are models of treatment that define sequential stages of interventions leading from detection of the problem through enrollment and delivery of treatments to final desired outcomes; a critical goal of these cascades is maintaining enrollment/compliance as patients progress from one stage to the next.

Confounding variables to consider when planning the study

• Environmental effects such as county or organizational policies, fidelity of interventions and variance in practices, and unmeasured endogeneity of clusters
• Legal and policy barriers. Consider adding an ethical, legal and social implications (ELSI) component, in much the same way related issues were integrated into Human Genome Project.
• State-, clinic-, provider- and patient-level variables.
• Race and ethnicity
• Community characteristics
• Access to treatment
• Diagnosis and comorbidities are essential to consider, along with individual social determinants of health.
• Data for helping metrics are prescribing, diagnosis, and if possible, inpatient data. Also EMS responses, police responses, arrests/criminal justice statistics, social services access, and births or deaths.
• Efforts should be made to control for geographic characteristics like employment rates, income level, and level of educational attainment.
• Rates of incarceration/institutionalization, housing stability and wealth/income disparity.
• State-mandated provider training

Potential threats to internal and external validity

• Many potential threats can be mitigated by employing near real-time data sets from large populations of patients from many locations of care, rather than data on patients from only three cities.
• Threats to internal and external study validity include the sheer overwhelming number of new studies and policies concerning opioids.
• Co-occurring interventions, such as changes in state policies. Acknowledging and accounting for these factors within the study analysis could help mitigate threats to study validity.

Coordination across research centers and data sets

• Consider a role for an unbiased and independent entity, not affiliated with the research org, as the data integrator and manager.
• Common data elements must be developed to enhance robust secondary and meta-data analysis, as well as individual participant meta-analysis. Secondary data analysis from judicial sources, treatment facilities, and clinical settings can be used to enhance evaluation and monitoring.
• Developing a data governance framework to determine data sharing processes, protect the needs of data stakeholders, and ensure transparency through a robust data use and reciprocal support agreement.
• Projects to facilitate data storage in such a way that reduces barriers for analytics while also maintaining security/reducing chance of data breach.
• Development of statistical, data scientific, and software engineering methods that integrate heterogeneous data: very structured medical outcomes, clinician reports, unstructured digital and ecological data.
• ADVANCE or PCORI model data standardization and datasets, and interventions across multiple industries. There should also be better models for data governance or linkage for unique encounters and modeling like the metric development (CMS).
• Collaborate with analytics organization to perform data collection and analysis.