Submitted by Dustin Beilke on
A recent foray into education "research" by the Bradley Foundation's Wisconsin Institute for Law and Liberty (WILL) is a case study not only in politically motivated research, but also in how the Bradley-funded echo chamber operates to inflate questionable claims and stave off critics.
The Milwaukee-based Lynde and Harry Bradley Foundation is perhaps best known for its advocacy of private and religious school vouchers. Bradley was instrumental in creating the nation's first voucher "experiment" in Milwaukee in 1990. A new cache of internal Bradley documents has come to light, which pull back the curtain on the $835 million organization and its efforts to advance school privatization, attack teachers unions and construct right-wing "infrastructure" nationwide.
The study in question is one that WILL released on March 1, 2017, purporting to show students in Milwaukee's private and religious voucher school program outperforming students in Milwaukee's public schools. Had WILL's conclusions been accurate, they would have contradicted consistent research findings from the last two decades showing Milwaukee voucher students fared worse or no better than their public school counterparts and several very recent studies showing voucher students throughout the nation faring extremely poorly.
Instead, the release of WILL's study showed that when one Bradley-backed organization speaks, numerous others rise up to echo it, and some mainstream news outlets are snookered by the noise.
The Center for Media and Democracy documented recently that the Bradley Foundation has poured at least $3.65 million into WILL since 2011. It has also received support from the Walton Family Foundation in 2014-2015, $585,000 according to IRS filings. Bradley funded WILL primarily to be a "litigation center" to aid the fight over school vouchers, to attack public sector unions and to backstop Wisconsin's right-wing infrastructure. For instance, Bradley notes, "WILL has represented School Choice Wisconsin in various actions against the Department of Public Instruction," (WILL, GPR, 11/10/15). And WILL filed an amicus brief in Duncan v. Nevada supporting the universal voucher program providing taxpayer money for private and religious schools, struck down by that state's highest court.
But Bradley's favorite litigation team is "morphing" into a think tank, say the Bradley documents, and producing studies on education. The stakes are high. The Bradley Foundation has spent untold millions supplementing Milwaukee voucher schools and advancing school vouchers in cities and states in partnership with the Walton's and Betsy DeVos' American Federation for Children. With DeVos now the head of the U.S. Department of Education, any study touting purported benefits of vouchers will be seized upon as Gospel by DeVos and Trump in their effort to expand vouchers nationwide.
Not "Apples to Apples"
WILL's March 1 report, Apples to Apples: The definitive look at school test scores in Milwaukee and Wisconsin, authored by WILL research director Will Flanders who has a PhD in political science, compared, for a single school year (2015-16), the average levels of student proficiency in math and English in K-8 schools and the averaged ACT college entrance exam scores for schools' high school juniors.
The report calculated the proficiency percentages on the standardized Forward Exam Wisconsin requires students in elementary and middle school to take, and high schools' averaged, composite ACT scores. It then compared the averages of private and religious voucher schools, charter schools and public schools and encouraged readers to draw the conclusion that voucher and charter schools are better than Milwaukee's public schools because their averaged scores are better. The report says it is making "apples to apples" comparisons because it makes an effort to account for student poverty, English language learning, and race, but it is unclear from the limited methodology Flanders describes if he does this correctly or not.
And that is just one of the flaws in the WILL study flagged by University of Colorado Boulder professor Benjamin Shear, who specializes in the misuse of educational testing data for research purposes, when he reviewed the report for the Think Twice think tank review project on April 25, almost eight weeks after WILL released its report. Think Twice is a project of the National Education Policy Center housed at Boulder, and is funded in part by the Great Lakes Center for Education Research and Practice.
To gauge how well one school does relative to another, researchers would have to look at the growth or regression of individual students over a number of years. WILL does rather the opposite, merging all students together in a school—regardless of grade level, or how long a student has been in a particular school or sector, or students' previous academic performance—for a single year and a single average. For K-8 students taking the Forward Exam, the average is relative to an arbitrary proficiency cutline that tells us nothing about how far below or above the cutline students scored.
Most crucially, and most ludicrously, WILL's study doesn't account for selection bias—differences between students whose parents and guardians decide to enroll them in voucher schools or charter schools and those who do not. Numerous studies show parental involvement is a key predictor of student performance, and the essential difference between a voucher student and a student in a traditional public school has to do with the action of a parent.
Shear says, "If policymakers or the public are interested in determining which schools or school choice policies in Wisconsin are most effective, this report cannot provide answers to such questions."
It is clear, however, in the way that Flanders presents the report, that WILL wants everyone to believe it has answered such questions. In the report's executive summary, where researchers state their main conclusions, Flanders writes, in bold letters, "We find that private schools in the choice programs and public charter schools in Milwaukee and Wisconsin perform significantly better on the ACT and Forward Exams than traditional public schools when a proper apples-to-apples comparison is made."
Referring to this sentence, Shear writes, "The report does not state precisely what is meant by an 'apples to apples' comparison. However, the implication is that the resulting comparisons can be interpreted in a causal manner. This is implied by the next paragraph stating that, 'we have to make the best use of the available data to provide parents with accurate information about what is working and what isn't.' Stating that this report indicates 'what is working and what isn't' infers that the differences between school sectors can be interpreted as the causal effect of attending a school in one sector relative to the others. This interpretation is further supported by use of phrases such as 'Effect of School Type on Forward Exam Performance,' elsewhere in the report."
Bradley Funded Echo Chamber
During the eight weeks between WILL's release of the study and Think Twice's academic rebuke, numerous Bradley-funded communication outlets ran with WILL's story.
On the same day WILL released the study, Wisconsin Watchdog posted an article praising the study and Wisconsin's private and religious voucher school programs. Wisconsin Watchdog, as noted by CMD, has been the recipient of $545,000 in Bradley money just in the time since Scott Walker was elected governor of Wisconsin in 2010. In addition to the direct donations, Bradley gave Wisconsin Watchdog's parent organization The Franklin Center for Government and Public Integrity another $250,000 to fund a fellowship at National Review magazine.
A week after Wisconsin Watchdog trumpeted WILL's voucher findings, National Review Online did the same in an article headlined, "New Evidence on School Choice Successes in Wisconsin: Higher test scores for students who attend schools their parents freely choose."
Meanwhile, the Heartland Institute, recipients of at least $648,000 from Bradley, posted an article praising the WILL study under the banner "Wisconsin Schools of Choice Outperform Public Schools, Study Finds."
As is so often the case, this concentrated effort in the right-wing echo chamber spilled over into the so-called liberal media, where, we are told, fake news abounds. Wisconsin Public Radio—yes, public radio—ran a report that rivaled the house organs for exuberance. The only source cited in the report other than WILL was from the president of the pro-voucher group School Choice Wisconsin, recipients of $3.36 million from Bradley from 2004 to 2015.
The Madison daily newspaper the Wisconsin State Journal ran a column from Flanders three weeks after the study was released, "Study shows need to expand Wisconsin's choice program." At least it was on the opinion page.
Amid this outpouring of enthusiasm for WILL and its research findings, the lone dissenter was Wisconsin Superintendent for Public Instruction Tony Evers. During the run-up to his eventual re-election, in a March 28 debate with his opponent Lowell Holtz, Evers said there is no credible research showing vouchers outperforming public schools. Of the WILL study, Evers volunteered, "They weren't comparing apples to apples, they were comparing apples to giraffes."
Wisconsin's independently elected school chief stood alone. No organizations, leaders or education researchers came forward to back his argument or confirm his skepticism until the Think Twice analysis was released four weeks later—three weeks after the election. His lone voice in the wilderness would likely have remained there had not Wisconsin Watchdog posted an article, two days later, condemning Evers for his remarks and exalting Holtz.
Studies Show Voucher Schools Harming Kids
Meanwhile, a spate of valid research shows deep, troubling problems with private school vouchers. In some states, the magnitude of the negative impacts is quite significant, calling into question the morality of continuing the "school choice" experiment after so many years of failure.
This April, an article in the Atlantic looked closely at a comprehensive study of voucher students in the Washington, D.C., voucher program—right under Betsy DeVos and Donald Trump's noses—and showed real, actual voucher students being outperformed by their peers in D.C's public schools. The Institute of Education Sciences (IES) found largely negative results for students who participated in D.C.'s Opportunity Scholarship Program.
The IES researchers compared test scores for two groups of students: students who, through a lottery process, were selected to receive vouchers, and students who applied for yet didn't receive them. The study compared the progress of both groups of students from spring of 2012 to 2014 and found that, a year after they applied for the scholarship, math scores were lower for students who won vouchers. For students in kindergarten through fifth grade, both reading and math scores were lower for students who won vouchers. While this study shares the WILL study's weakness of looking at just one school year, it is much more meaningful because it looks at each individual student's progress rather than a group's aggregated proficiency. Most crucially in terms of scientific validity, it examines two groups of students who are demographically identical except for the random selection of a lottery process, eliminating the selection bias.
While in an evidence-based world, voucher students' low scores would undermine the Trump administration's school privatization efforts, he and DeVos remain undaunted. Trump's proposed 2018 budget would allocate $1.4 billion more for school choice on top of the billions the federal government already spends with the goal of "ramping up to an annual total of $20 billion." The proposed budget would also apply $250 million toward "a new private school-choice program," which means even more vouchers. While the Trump/DeVos Department of Education budget will put billions more into privatization, as a whole there would be $9 billion less spent on education, a 13 percent cut.
The IES report adds to a growing body of recent, comprehensive, scientifically valid studies on state and local voucher programs that have found startlingly lousy academic results for voucher students. In February 2016, researchers published a major study of Louisiana's voucher program that showed plainly horrifying results in math. Public elementary school students who started at the 50th percentile in math and then used a voucher to transfer to a private school dropped, on average, to the 26th percentile in just one school year. Martin West, a professor at the Harvard Graduate School of Education, said the negative effects in Louisiana were "as large as any I've seen in the literature" in the history of American education research.
In an Economic Policy Institute report published on the last day of February 2017, the author found that vouchers have not only failed to improve student performance, but are succeeding in undermining public education programs and methods that have been successful. The study reviewed evaluations of voucher programs in cities including Milwaukee, Dayton, Ohio, and Washington, D.C., and the states of Indiana, Louisiana, and Florida, and shows that vouchers do not improve test scores and most likely do not improve graduation rates.
In Milwaukee, which, the report's author reminded us, has the nation's oldest voucher program, African American students rank second to last in eighth grade math scores and last in reading scores, significantly worse than when the voucher program began. Somehow, these two particular standardized test results were not included in WILL's research report promoting the expansion of school choice.
CMD's Mary Bottari contributed to this article. See CMD's series of articles on the Bradley Foundation here.
Printer-friendly version