EDUCATION Rank matters: The dubious business of school ranking


By Bruce Adams

 

Clockwise, from left: G. Scott Thomas, Dr. Mary Gresham, and Suzanne Miller; Illustration by Josh Flanigan.

Imagine that over the summer the top-ranked school district inBusiness First’s Guide to Western New York Schools—Williamsville—magically swapped places with a lower-ranked district. Not the bottom one; let’s say district number 46. Imagine that this fanciful exchange included everything: buildings and grounds, boards of education, administrations, teachers, support staffs, technology, supplies, books, district policies—everything but the students. So when the kids of Williamsville head back to the classroom this September, they will attend the lower-ranked district’s schools, and vice-versa.

Flash-forward to June and an amazing transformation has taken place. The students formerly attending the 46th-ranked district are now the highest-performing scholars in the region, having reaped the benefits of Williamsville’s secret pedagogical methods, while last year’s top-performing students have tragically dropped forty-five positions. Fat chance, you think? Of course, this is only a simplistic thought experiment, but it spotlights the tenuous premise behind Business First’s annual rankings. “We’re ranking school districts, not communities,” declares G. Scott Thomas, the man who originated the rankings and has overseen them since their rudimentary conception seventeen years ago. The school ranking issue is Business First’s top seller, and Thomas also edits a separately marketed 140-page extended report. A lengthy phone conversation found this Casey Kasem of schools to be gracious and earnest, willing to take on any question. “We’re trying to make [standardized test scores and other data supplied by the New York State Education Department (SED)] more understandable by making them comparative, and showing how individual schools and districts are performing relative to the other districts and schools in the area.”

Thomas’s assertion that it’s district and school performance that’s being ranked implies that accountability for success or failure lies with the institutions, not students, parents, or external factors such as poverty. Suzanne Miller, Ph.D., chair of the Department of Learning and Instruction at UB, sees that as the exception. “There are a few instances where schools in poor communities achieve at rates that ‘beat the odds’—and there are things [happening] that matter in those schools.” What matters, according to Miller, is taking a broad approach to education, rather than focusing primarily on assessment scores—“teaching to the test.” Thomas views poorer districts that excel as evidence that schools can outperform their community’s socioeconomic status. He cites several, including Akron, where math scores inexplicably soar, while other assessments there place about average. However, Business First’s high-profile ranking may actually workagainst sharing such singular achievements by cultivating a competitive climate among schools. With administrative jobs on the line, and districts measured against one another, successful schools may not be anxious to assist struggling ones.

Faulty reasoning
The assertion that Business First ranks the combined performance of bricks and mortar, books and teachers—but not the communities they serve—confuses correlation with causation. Do top-ranked schools make successful students, or is it the other way around? It’s a question the SED data doesn’t address. Research suggests that socioeconomic factors have the greatest impact on student achievement. Dr. Mary Gresham, dean of the Graduate School of Education at UB, cites numerous examples, and high among them are economic status and parental involvement. “Ranking gives people a false sense of ‘excellence,’” says Gresham. “Williamsville, Clarence, and Amherst have a lot of parents who are activists, who are educators themselves, and so there’s an expectation, and a partnership on the part of the parents and the school that is probably unique in terms of some of the other districts in the area.”

“That’s not the issue,” argues Thomas, “The issue is, does this disparity exist or not? And what I’m saying is that it’s our job to point out that this disparity exists. Somebody ought to be out there trying to adjust the system so that everybody really is equal.” However, Business First hasn’t called for economic parity between schools. There has been no editorial outcry for increased educational funding—though costly reforms such as smaller class sizes might help offset damaging educational factors such as poverty. “I don’t know if that’s the answer,” replies Thomas.

“People want stuff quick, fast, and in a hurry, so they’ll look at a ranking,” says Gresham. “I think people have a responsibility to understand exactly what they mean—what they account for and what they don’t account for—before they become overly reliant on them. Anyone who knows anything about education or about testing should not be surprised by the ranking results.” That’s partly because with some notable exceptions ranking order largely parallels economic status, with the wealthiest communities on top and the poorest at the bottom. “Without test scores,” says Miller, “you could run the poverty index of schools and come up with an ordered list that would be eerily similar [to Business First’s rankings].” Focusing on wealth alone, however, neglects the “socio” end of the socioeconomic formula. Research suggests that a myriad of social factors contribute to student achievement—such things as family communication, community values, religious involvement, drug abuse, and health care. In addition, certain ethnic cultures emphasize the importance of education, as illustrated by the archetypal non-English-speaking Asian student who arrives in a poorly performing school and graduates with honors several years later. Thomas agrees that many factors impact student achievement, but, he responds, “If you were going to do a study, we’d need a Ford Foundation grant to go into each school building and see why some are having improvement and others aren’t.”

The observer effect
If social factors largely determine academic success, why rank schools? “First of all, people are making choices about where to live, and they’ve found it extremely useful in that regard, whether they are relocating within the area or locating here,” explains Thomas, “They have a better sense of how the schools in the area they are looking at compare to the ones where they are at.” Miller sees a problem here. “These ‘rankings’ are converted immediately to real estate values, and pressure on school boards, on administrators, on teachers, on students, for quick fixes.” Thomas acknowledges that lower-ranked schools have claimed on occasion thatBusiness First stymies their attempts to improve. But he responds that a district in decline “is a situation that exists whether we rank [them] or not. Pointing that out to people is not damaging.” Yet, if involved parents use Business First’s rankings to move to higher-ranked schools, doesn’t this exacerbate the problem? Could a low school rank give struggling students an excuse to perform poorly, stoking a victim mentality that eschews personal responsibility and blames the system? Thomas states with Joe Friday just-the-facts somberness, “The test results are what they are.”

Rankings are sexy. “[They] are advertised and presented as competitions that reveal the status, importance, ability, and eminence of a school,” says Miller. Ranking distills complex topics down to easily digestible winners and losers, like an academic horse race. And that’s just how the media treats it. “The Williamsville Central School District is number one for the fourth straight year …” says WGRZ; WKBW announces, “Williamsville East climbed to second place in 2008—its best finish ever;” MSNBC proclaims, “Nardin Academy is the clear winner …”

“You won’t notice that [language] in our publication,” Thomas replies. Wrong. Emblazoned on the cover of Business First’s Guide to Western New York Schoolsare the words: “The dynasty continues, as Williamsville wins the district crown for the sixth year in a row. Clarence finishes second.” District leaders play along, too; highly ranked schools often publicly accept credit for their student’s achievements, while lower-ranked ones have been known to fudge the results. Thomas declines responsibility. “We can’t control what others do with that information. Do we understand that a lot of people see this, that they pick it up somewhere else [in the media]? Sure, but that’s the nature of this business; once you create information and put it out there, other people will use it for what they wish.” Business Firstattempts to counterbalance such distortions with in-depth analysis, but one suspects it’s the flashy sound bites most people remember.

Statistical inertia
No matter how much educational improvement takes place across Western New York, there will always be top- and bottom-ranked schools. About half the schools are destined to forever perform “below average,” and half “above.” As to why individual schools improve their scores yet sometimes fail to move up the ranks, Thomas replies, “Because everybody is improving.” But for the staff and students in schools that struggle to improve yet consistently rank low, it’s a morale buster. Here I speak from experience, having taught for thirty years in the eighty-third-ranked high school within the forty-sixth-ranked district cited in our thought experiment at the start of this article. Over my career, I willingly jumped on each new pedagogical bandwagon that rolled through town with the promise of revolutionizing education: mastery learning, Total Quality Schools, outcome-based education, authentic assessment, learning styles, Learning by Design, and professional learning communities, to name a few.

What I couldn’t do was overcome student apathy. In high-achieving districts where expectations are that children will attend good colleges, grade point average matters. In districts like mine where students often plan to go from high school into the military, community colleges, or the workforce, a passing grade of sixty-five is often the gold standard. This is the system we created.

Some of my peers experienced the cultural disparity that can occur between districts. A history teacher who went from my school to Williamsville marveled, “It’s so different here. You don’t challenge students; they challenge you!” A middle school art teacher came from Amherst and needlessly inquired whether it was safe to give students grades below ninety; “If you do that in Amherst you get angry calls from parents.”

Who profits?
Everyone agrees that there are appalling inequalities between schools that should be rectified, but in seventeen years, the Business First rankings have done little to change the educational landscape. “As an educator and researcher, I have seen unintended consequences of these rankings,” Miller states. “Schools at the top insisting on narrow ‘test-prep’ curricula to keep or increase their rankings; lower-ranked schools insisting on scripted English Language Arts programs that narrowly define literacy—to the exclusion of real reading and writing—because they believe this will help increase test scores; teachers called to administrators’ offices being asked if they really want to ‘give a failing grade’ on a student’s essay.” A former student of a high-ranking school once told me she dreaded the periodic auditorium assemblies when the principal would irately admonish them to maintain their ranking dominance.

Miller contends that the real issue is how individual schools do within their socioeconomic and community situations. Thomas argues that this is unquantifiable; unbiased data drives his rankings. But what data justifies the practice of ranking itself? Sales figures? Letters from satisfied readers? “Someone needs to accurately reflect the disparity that exists and I think we’re doing that,” says Thomas. Miller counters, “There is an embedded and underlying assumption that these schools are all on a level playing field to begin with and that the lists presented are indeed rankings.” Gresham adds, “Rankings should be taken with a grain of salt. Everyone looks at them; we don’t deny that, but it’s all relative.” Who then really benefits from ranking? It’s hard not to wonder if the answer lies in Business First’s name.

Bruce Adams is an artist, educator, and writer living in Buffalo.