Sometimes, the now-common wisdom goes, students just need a “nudge” in order to take small actions that will have a big effect on their schooling and their life.
In the past 10 years, the use of low-cost interventions, typically taking the form of a well-placed text message or mailer, have been used to try to encourage high school and college students to apply to college, enroll in classes and fill out financial aid forms.
According to many research studies, small-scale nudges have worked among specific populations. But according to new research, efforts to expand those interventions nationally have more mixed results.
“The approaches we’ve taken to take these strategies to scale haven’t worked,” said Ben Castleman, an associate professor of education and public policy at the University of Virginia. He directs the Nudge4 Solutions Lab at UVa, which uses tools honed by behavioral economics and social psychology to try to improve college and employment outcomes.
A National Bureau of Economic Research working paper published by Castleman and several other researchers in August examined a bevy of nudges to get a large amount of students — 800,000 high-schoolers who had filled out the Common Application for college and students at several stages of the college process in an unnamed state — to fill out the Free Application for Federal Student Aid. Students in varying groups received information about changes to the FAFSA or texts, emails or mailers about filling out the application to maximize the amount of aid they would receive. Some also received offers of one-on-one advising help, as well as encouragement to ask peers to fill out the application.
The result: None of the nudges had an effect on students’ college enrollment or the amount of aid they received.
“The broad takeaway is that when we implemented a nudge at a large scale, there wasn’t a large difference in college enrollment or persistence,” Castleman said.
Kelly Rosinger, an assistant professor of education at Penn State University, also worked on the study.
Her research focuses on how to help students make complex decisions, and whether policies and interventions can make small changes to help students who often don’t get the same level of resources and attention.
Students who use the Common App — a tool that lets them apply to multiple universities with the same set of essays and personal information — tend to enroll in college at high rates, and the nudges didn’t demonstrably alter that fact, according to the white paper. But the paper’s results, Rosinger said, indicate that to really help low-income, minority and first-generation students enroll and graduate from college, a better network of resources may be needed.
It’s possible that the smaller studies found that nudges were more effective, Rosinger said, because in those studies, students had a personal connection or trust with the people or organizations offering assistance.
“We can’t rule out the possible impact of one-on-one advising due to the small number of students we were able to provide this to,” she said. “And some smaller studies have found an effect on FAFSA completion, so we can’t rule out the impact of nudges on a local level.”
She thinks the field of behavioral economics still has a lot to teach the world of education, she said, and hopes that future studies will drill down into the possibility of creating local networks of advising organizations and nudges.
“We should still believe in nudge interventions at the local level and shouldn’t abandon them after one study,” she said. “Just as we shouldn’t believe they will always have an impact at scale based on smaller-scale studies with local partnerships. There is always more work to be done.”
A similar working paper by Castleman and other experts, published this month, which studied FAFSA completion and the likelihood of a 10,000-student sample receiving financial aid and finishing their degree, reached similar conclusions.
“While the intervention somewhat accelerated the timing of FAFSA fling for some students, slightly increasing the odds that they filed during the spring when they were being nudged to do so, it did not boost overall rates of FAFSA filing,” the authors wrote in the paper, though they noted the study had some limitation due to its timing in the calendar year. “Moreover, students neither received additional federal financial aid as a result of the intervention nor incurred positive benefits in terms of continued enrollment or graduation.”
Both papers, and the researchers, conclude that the efforts to scale nudges weren’t very effective, but, they say, some additional effort to target specific populations and work with more salient organizations could still have a real impact.
“I don’t see that nudges are no longer effective,” Castleman said. “But how we implement them matters a lot.”