Of the six studies that Gov. Romney cites to defend his mathematically-challenged tax plan, a few were blogs and were thus often dismissed by critics as being de facto inadequate evidence simply by dint of being blog posts. As an active, evidence-based blogger, I can only say…quelle horreur!
Actually, I think it’s entirely fair to heavily discount blogs as evidence. Obviously, quality varies widely and there are no gate keepers on the net, and nothing approaching rigorous peer review. However, there’s room for nuance.
First, some blogs, like the ones at CBPP and EPI (not to leave anyone out, but those are places I’ve worked so I know how they operate), generally report shortened, reader-friendlier versions of their studies, so judging those blogs as evidence depends on your view of their studies.
Which gets to my larger point—I wouldn’t generally trust blogs as evidence but neither would I trust every think tank report. The most reliable basis for trustworthy evidence in social science is peer review, a process by which experts of the established knowledge on the topic, along with rigorous application of the rules of statistical evidence, evaluate the claims in the study before it can be published.
This process too, of course, is far from fool proof. There are statistical outliers, but more problematic, embedded prejudices in social science—assumptions and even “fads” that undermine the peer review process. For years, economic journals printed studies supporting self-regulation by rational actors that gave succor to Greenspanian laissez faire regulatory practices which in turn helped spawn the Great Recession.
Now, slowly, those articles are being replaced by new takes on an older set of ideas once espoused by Kindleberger and Minsky about inherent instabilities in the economic system.
But back to blogs, think tanks, and what constitutes evidence. The thing I’ve always found useful about the work of the think tanks I trust is that they tend to take a pretty-much “just the facts ma’am” approach in their reporting. CBPP and EPI reports, for example—again, places I cite because I know first hand how they make the sausage—tend to be based on data series from established data sources with little manipulation. They offer interpretation, but the facts and data processes are right out there and derived from trustworthy source material.
In terms of using blogs for evidence, you’ve got to know with whom you’re dealing. I can most reliably speak for myself and much of what I do in this space is report evidence from sources I judge to be reliable in ways that are intended to go down more easily. When I present my own statistical evidence, I’m trying to follow the standards noted above re EPI/CBPP—plotting/interpreting reliable data.
But blogs are by nature quick, impressionistic takes on issues. They can point you in the direction of deeper dives, but they will very rarely, I think, provide adequate evidence for policy decisions.
Case in point: Romney’s tax plan. It would be ludicrous to trust a blog or an oped that threw some rough numbers together over the Tax Policy Center’s results, as the latter are based on a well-established tax model built by non-partisan experts that has a long track record of generating reliable facts about tax policy. Certainly the blogs can play a very useful function in raising questions about the deeper studies, and even send the authors of the study back to the model for tweaks suggested by the bloggers.
But it’s actually pretty shocking to me—though I know I should develop a thicker skin—that the Romney campaign is now lurching around, daily trotting out new “studies” and “solutions” to their math problem–“we’ll broaden the base (but can’t say how)”…”we’ll cap deductions at $17,000”…”whoops…that doesn’t work…we’ll cap them at $25,000.”
[See here for TPC evidence that these caps also fail to raise enough revenue to offset the $5 trillion, 10-year cost of the tax cuts. Since most deductions and credits accrue to higher income households, the caps do, however, raise revenue in a progressive manner.]
The Romney team clearly threw out their tax plan—20% cuts across the board—without any of the requisite spade work to see if it actually made sense. And now that real studies are challenging it, they’re veering from “just trust us” to setting the evidentiary bar down so low that anything with numbers on it can clear it.
Not surprising, given the beating the facts have taken in this election cycle, but not so good for democracy, not to mention the truth.
Shares