We've entered the season when local news outlets look back on the past year and compile their best-of/worst-of lists. Invariably, those lists mention how Vermont stacks up relative to other states on some trend. As surely as you'll hear "Santa Claus Is Coming to Town" played ad nauseam in the local mall, Vermont will show up near the top, or bottom, of some state-by-state comparison.
What should readers make of these rankings? It depends on which ones you read. By some measures, the Green Mountain State is a paradise full of happy, healthy social-justice warriors who invest heavily in their children and pets. By others, Vermont is an opioid-addicted hellhole populated by heathens, racists and internet trolls.
Does the Green Mountain State's frequent appearance at or near the fringes of national surveys indicate Vermont's exceptionalism? Or is it just a statistical anomaly resulting from the state's tiny population?
Before a professional statistician weighs in, consider some of the best-of/worst-of lists on which Vermont appeared in 2017. U.S. News & World Report ranked Vermont No. 1 overall for "public safety," based on its low rates of juvenile incarceration, prison overpopulation, property crime and violent offenses. Meanwhile, the Federal Bureau of Investigation's annual report on hate crimes identified Vermont as the seventh worst state, per capita, for criminal activities motivated by the victim's race, religion or ethnicity.
By one measure, Vermont is home to America's biggest do-gooders, with more nonprofits per capita than any other state. Yet, also in 2017, Vermont ranked 33rd for its per-capita charitable giving. One possible explanation: Charitable donations often go to houses of worship; states ranked high on this measure, such as Utah, also ranked high in religiosity. Vermonters ranked dead last in their degree of religious affiliation, according to the Pew Research Center's division on religion and public life.
Vermonters are politically engaged and, presumably, satisfied with their elected officials. This year, U.S. Senators Bernie Sanders (I-Vt.) and Patrick Leahy (D-Vt.) ranked No. 1 and No. 2, respectively, among the nation's most popular members of Congress in their own states. Yet Vermont also ranked high on a national grumpiness index: Wired magazine identified it as the state with the most internet trolls per capita.
The Fiscal Times ranked Vermont among the top five states in the categories of general health and providing kids with safe and healthy childhoods. The international charity Save the Children ranked Vermont fourth best for kids, citing its high rate of high school graduations and low rates of teen births, infant mortality, and child homicides and suicides. Similarly, WalletHub ranked Vermont third among the "best states to raise a family."
What's keeping Vermonters so healthy? Perhaps all the poop we scoop and litter boxes we empty. Polls by the American Veterinary Medical Association have consistently shown Vermont to have the nation's highest rate of pet ownership, with nearly three out of every four households owning a cat, dog or both. Given that pet ownership has been linked to lower rates of depression, anxiety and hypertension, as well as other health benefits, Vermont's high quantity of kibble and kale seems to serve us well.
But Vermonters aren't all low-fat-granola munchers and 5 a.m. joggers. The state ranked sixth nationally for its rate of opioid-related hospital visits. It had higher-than-the-national-average rates of binge drinking among its college students and the fifth highest rate of per-capita beer consumption nationally.
How much stock should Vermonters put in this mixed bag of info?
"I don't give much credence to those rankings," said Jeff Buzas, who chairs the University of Vermont's department of mathematics and statistics. While he couldn't comment on specific studies without analyzing their data collection methods, he suggested multiple explanations for why Vermont tends to rise to the top or sink to the bottom.
Some results can be attributed to poorly done studies, with too-small samples, apples-to-oranges comparisons or inconsistencies in data collection, Buzas said. Others probably show that Vermont is legitimately exceptional — or at least unique.
Reports on such studies, Buzas elaborated, rarely note or quantify the uncertainty that goes into constructing rankings. In some cases, the margin of error — what statisticians call the "confidence interval" — is large enough to make the difference between the best-ranked and worst-ranked states statistically insignificant.
Sometimes, Buzas continued, results are skewed by the small sample size of the original survey or by data-collection methods that differ from state to state. Or they might be accurate but not indicative of anything we don't already know. Vermont's ranking in the rate of violent crime, for instance, reflects the state's lower population density and lower degree of "friction" among its residents.
Some polls and rankings are simply unscientific. Buzas pointed to the annual Town Meeting Day surveys created by former Washington County state senator Bill Doyle. For years, Vermont news outlets reported Doyle's survey results as barometers of public opinion on everything from wind energy to marijuana legalization. But, Buzas pointed out, those polls were never conducted scientifically or collected in a way that meaningfully represented Vermonters' views.
Another "totally unscientific" poll that made Buzas roll his eyes: Seven Days' formerly biannual sex survey.
"Oh, my God. That was the worst!" he said. "You guys should have had a huge disclaimer that this is not a representative sample."
We did — and discontinued that survey in 2013.
Finally, Buzas cautioned that, while these rankings can seem simple enough, interpreting the results is not.
"It's complex," he concluded. "Once you start really thinking carefully about it, it's not simple to draw conclusions. For questions that are simple to state, it's often very difficult to collect the data to answer them."