In spring 2025, the DC Education Research Collaborative, in partnership with the D.C. Policy Center, developed a questionnaire for school and program staff members of the District of Columbia Public Schools (DCPS) as part of our multiyear study examining career development opportunities (CDOs) available to DCPS students.
The questionnaire served as our primary data collection instrument to gather detailed information on the components and intended outcomes of around 40 CDOs across the city. During the survey design phase, our team noticed three challenges that could severely affect the quality of our data and our relationship with our DCPS partners.
First, we needed a 100 percent response rate to cover the universe of CDOs (a list co-created with DCPS). But conducting a full census would be challenging, given that the questionnaire was comprehensive, some program leads oversee multiple programs and would therefore have to fill it out multiple times, and survey fatigue lowers response rates.
Second, DCPS central services staff members said they wanted to minimize the time that any one individual spent on the questionnaire, which by necessity was quite detailed. Given that this phase of data collection was coming early in a multiyear project, it was critical that we ease response burden on DCPS.
Third, our DCPS partners advised us that, regardless of how carefully we crafted the questionnaire, program staff members would be susceptible to interpreting questions differently or providing data in formats different from what we intended. This created concerns about data uniformity and quality, and we wanted to avoid not only initial errors but having to go back to program staff members to review data for quality control.
These three challenges are not unique to our study. Balancing respondents’ time, capacity, and desire to provide data with researchers’ needs for complete, accurate data is a common issue. The need to preserve relationships with our respondents (and their supervisors, who connected us to them in the first place) is also a common theme in any sustained research effort.
To meet these challenges, we adopted a prefill strategy, using a wealth of public or easily requested information about many CDOs on our list. Some programs had websites describing how to apply, documentation for instructors on how many sessions the opportunity entailed, or aggregated data on the number and location of available seats. So before fielding the questionnaire, we conducted a desk review of available program documentation. This resulted in more than 1,000 data points that informed our understanding of CDOs and reduced the need to ask program staff members for information.
We then prefilled the questionnaires with the data from the desk review by manually loading each CDO’s unique survey link and filling out as many questions as we could from information we had already gathered. If we did not have information for a specific response, we left it blank. When the program leads filled out the survey themselves, we explained that some responses were prefilled and that they should change the responses only if any information was incorrect.
Of a survey that consisted of 35 questions, we were typically able to prefill responses for 15 to 20. We estimate that prefilling responses reduced response time by 5 to 15 minutes (anywhere from 20 seconds to a minute per question).
Prefilling responses also created consistency across the dataset. By entering information in a standardized format before we sent surveys out, we reduced the risk of variation in how key data points were recorded across the more than 40 programs, making our data more complete and accurate and our analyses more reliable. From my perspective as a researcher, prefilling responses reduced my time cleaning survey data and reduced the need to go back to program or central services staff members to verify information.
By using a careful desk review to prefill surveys, our team balanced the researchers’ need for census-level, error-free data with respect for respondents’ time and capacity, especially among school staff members. The quality of the work, and the relationships between researchers and practitioner partners, was strengthened as a result.
