You’ve designed your study in league with your community engagement and research teams. You have secured IRB approval, and you have drafted your funding proposal in collaboration with your site PIs and research team. Even with all the moving parts required for a study with human subjects, the research design has come together well.
Then you look at your estimated enrollment across study sites and stop short. They’re skewed. What to do now?
Maybe the estimated enrollment numbers meet your sample size for analysis of your primary and secondary outcomes, and you even have enough for a subgroup analysis or two. But when you see the numbers entered into the estimated enrollment table, your heart sinks. Although you have a fabulous recruitment team and strategy, your estimated enrollment is not as diverse as you—or, more importantly, grant reviewers—might expect. Perhaps the number of male and female participants is not roughly comparable for a study that is meant to provide outcomes applicable to both males and females, or the race and ethnicity numbers don’t mirror the population. The data resemble the communities you’re working in, but maybe your communities are unique and might appear skewed to reviewers unfamiliar with your community. If left unaddressed, this is a weakness in your proposal.
Reviewers want to fund scientifically sound projects with the potential for great impact on health outcomes for all patients. Understand and acknowledge these expectations, then address them with the appropriate evidence.
Solution
Here’s what I’ve learned from writing and reviewing research grant proposals:
- If the lack of diversity in the enrollment impacts the science and will produce outcomes that are not generalizable, look for research sites you can add to the study to reach an appropriate demographic mix that supports the science.
- If the demographics of the estimated enrollment does not impact the science or the outcome of the study, then address the issue head-on in the section of your proposal in which you provide the funder with your estimated enrollment numbers:
- Justify why the science is not impacted and how conclusions drawn from the study will be valid and generalizable.
- Provide demographic data for your community to demonstrate how the estimated enrollment of your study reflects your community. Since reviewers tend to review data presented in tables and may not look in the text, I would advise adding your data to the existing table or adding a table to the right or immediately below if an enrollment table is part of the application template and cannot be amended.
This approach applies to both prospective and retrospective studies but, in my experience, research teams are much more aware of the demographics of these numbers and the need to address their balance in prospective studies. There’s more of a willingness to claim an inability to adjust in retrospective studies, even when diversity and balance are critical to producing generalizable outcomes.
AI Model Development
In my work reviewing and critiquing AI model development research designs, I see teams planning to use only their institution’s patient data for training, validation, and testing a model they want to provide to institutions across the country. In very few of these instances are the data sets representative of the population of the country. The strongest proposals I see in AI development propose the use of large, diverse data sets resulting from the use of a network like PCORnet or collaboration with one or more institutions willing to provide data.
Reviewers want to fund scientifically sound projects with the potential for great impact on health outcomes for all patients. Inclusion and equity in recruitment contribute mightily to the generalizability that underpins health care equity. Understand and acknowledge these expectations, then address them with the appropriate evidence.