Think Harder

BY ILYA LOZOVSKY

JANUARY 27, 2015

Every year, American parents turn to U.S. News & World Report’s annual college rankings, a frequently consulted (and just as frequently derided) index indicating year after year the surprising fact that Princeton, Harvard, and Yale are, once again, the best places to send your kids to school.

What if you’re not an anxious parent looking to get your kids out of the house, but an anxious government official looking for policy advice? Where’s the best place to turn? This year, you might reference something called the “2014 Global Go To Think Tank Index Report,” just released by the University of Pennsylvania’s Think Tanks and Civil Societies Program.

Billed as “the most comprehensive ranking of the world’s top think tanks,” the report ranks 150 think tanks across several categories, including region and area of research. Of course, there’s also a “Top Think Tanks Worldwide” section, which allows the Princetons and Harvards of the think tank world (Brookings, Chatham House, and the Carnegie Endowment) to get their due.

According to James McGann, a University of Pennsylvania professor who heads the survey, he and his colleagues acquired the data by contacting thousands of journalists, policymakers, and, yes, think tanks and inviting them to nominate candidates for inclusion in the list. In a second round, participants ranked those that made the cut from “best” to “worst.” Smaller expert panels then reviewed and massaged the list according to their understanding of each think tank’s quality. Neither the identities of these expert panels nor the specific criteria for their selection are publically revealed. Though the report’s authors provide a list of 28 criteria that participants are “encouraged” to apply to their decisions, the actual ranking is subjective, the process known only to the experts themselves.

Some commentators have offered suggestions on how think tanks might be ranked more objectively. I think the real problem is more fundamental.Some commentators have offered suggestions on how think tanks might be ranked more objectively. I think the real problem is more fundamental. Just as the U.S. News & World Report is frequently criticized for measuring the wrong thing — SAT scores of incoming students and artificially deflated acceptance rates, rather than quality of education — so, too, does the Penn report fail to address the single most important ingredient of a think tank’s quality: the integrity of its research.

Last September, The New York Times published an exhaustively researched investigation that implicated some of Washington’s most prominent think tanks in accepting funding from foreign governments eager to advance their own interests in the U.S. capital. Such concerns aren’t new. In 1999, David Callahan of the Washington Monthly published a blistering attack on right-wing think tanks, including the American Enterprise Institute and the Cato Institute, which he accused of shilling for private business. In a widely noted article in September 2013, Molly Ball chronicled the transformation of the Heritage Foundation from a bastion of conservative intellectualism to a policy arm of the GOP’s most conservative faction. (The Cato Institute and Heritage Foundation rank eighth and ninth in the Penn report’s ranking of U.S. think tanks. The American Enterprise Institute ranks twelfth.)

Thomas Medvetz, a professor at the University of California San Diego and author of a recent book on the history of American private policy research, says that the very concept of a “think tank” needs to be approached more skeptically. Brookings and institutions like it, says Medvetz, have deliberately positioned themselves at a murky intersection of academia, business, media, and advocacy. And, in the absence of any vigorous challenge, they will only continue to profit from a symbiosis with politicians and journalists all too eager to consume their products. Though the Brookings Institution may use an “.edu” web domain, it’s not a university and it’s certainly no ivory tower.

The Penn report, Medvetz says, “takes for granted precisely what we should be asking about: how do you come to be recognized as an expert in the United States?” The think tanks criticized in the New York Timesinvestigation — Brookings, the Atlantic Council, and the Center for Global Development — rank first, sixteenth, and twenty-seventh respectively in the survey’s “top think tanks in the United States” category. (And in the interests of full disclosure: Though I’m an FP employee, Democracy Lab is a joint venture between FP and the Legatum Institute, a London-based think tank that wasn’t included in the report.)

McGann, the report’s lead author, insists that he’s not “ducking the issue.” That New York Times investigation, he says, was an unfair and sensationalistic “witch hunt” that presented “not a shred of evidence” that the think tanks in question were compromised by their reliance on foreign donors. He underscores his report’s numerous references to “quality of research” as well as the inclusion of “independence” as one of 28 criteria for participants to consider. He dismisses criticism of think tanks for failing to disclose their donors as too simplistic, though he also promises that future versions of his report will measure their commitment to integrity in a more comprehensive way.

Medvetz agrees that a think tank’s donors are only part of the picture: there are other forms of dependence too, personal and political. But he embracesthe New York Times article as a “remarkable piece of investigative journalism,” describing its conclusions as “pretty much beyond dispute.” What’s really needed, he says, is a transformation of the private policy world through a “norm of voluntary transparency” that will push think tanks to be more forthright about their connections. Efforts by civil society organizations like Transparify to grade think tanks by financial transparency are a good start.

Yet Medvetz sees the current system as so entrenched that he’s skeptical about the likelihood of change. 

Think tanks, in Washington and elsewhere, are no less susceptible than any other institution to the temptations of money and power.Think tanks, in Washington and elsewhere, are no less susceptible than any other institution to the temptations of money and power. Neither are they any shyer about defending the turf they’ve carved out.

At the very least, journalists can do their part by treating claims of impartiality more skeptically and providing context about possible conflicts of interest. And those who devote their careers to studying the world of private research institutions can do more to address these issues head-on. It’s a pity that, as “the premier database and measure of world think tanks,” the Penn report does little to go beyond the status quo. Unless this particular paradigm shifts, may I be the first to congratulate the Brookings Institution on its inevitable victory next year.