In 2003, Michael Lewis wrote a book called Moneyball: The Art of Winning an Unfair Game, a book that is probably the best combination ever of good writing and important analysis about the sport of baseball. It went on to be massively popular and was eventually made into a movie starring Brad Pitt and Jonah Hill (a very good movie, too). The central premise of the book is that the so-called “wisdom” that has dominated baseball thinking for the last century is deeply flawed. Baseball has always been heavily oriented to statistics and numbers, but the numbers used are a really bad measurement of how good players are and they should be replaced by better numbers, using more scientific analysis to improve results. This seems obvious, but despite the fact that newer, better statistics have been developed (usually under the blanket term Sabremetrics), at the time of the book’s topic, almost nobody was even beginning to look at those better ways of analyzing the facts. In fact, traditional baseball analysts not only reject the new statistics they ridicule them and make it difficult for people who recognize the value of the new statistics to find work or keep a job. Even after teams like Oakland and Boston showed significant success with the new, more scientific approach to the game, most teams still reject the new ways for the old, preferring to rely on the conventional wisdom that really, really has no validity.
While the book is about baseball, it is nearly perfectly analogous to politics, particularly left-wing Democratic politics.
Democratic politics is heavily focused on numbers, but few, if any of the people examining those numbers really have any training in statistical analysis and even fewer of them know what numbers are important or how to study them. Logical fallacies abound in the “study” of elections and policymaking. “Gut feelings” and “experience” hold more sway than empiricism. Hell, people still spend money on yard signs, despite the clear evidence that they are useless.
And the adherents to the old way of doing things, the conventional wisdom, not only staunchly defend their way of doing things — even though they consistently lose — they keep training the next generation of candidates, staffers and consultants and they keep getting paid large sums of money. And they keep losing. They win often enough to claim that their methods work, despite the fact that there are many other factors that determine election outcomes and the obvious fact that even a terrible candidate can win if they have enough money and a favorable enough climate for their party or even bigger incompetence from their opponent (see: George W. Bush). These same people explain away any loss as based on the “specific circumstances of that particular race,” while always claiming it is their own expertise and knowledge that led to victory. They are exactly like the football players who thank Jesus when they score a touchdown but forget to mention his name when they fumble the ball. This is how Democrats consistently lose, despite the fact that they have more registered voters in their party and the fact that the people overwhelmingly support the more liberal position on almost every issue.
You can see some of this on the right, as evidenced by this year’s assault on Nate Silver, the New York Times blogger who has predicted nearly every Senate race and electoral college state in the last two elections. Prior to Tuesday, right-wingers were assaulting him as a partisan (despite the fact that he doesn’t vote and prefers Libertarian candidates) and dismissing his analysis. But scientific research never lies and it is only wrong very rarely (if properly conducted). Democrats were scared to death about this election despite the fact that all the available scientific evidence said, from the beginning, that Obama was very likely to win. But if you reject science — which even most Democrats frequently do — it’s much easier to be scared and it’s much easier to have no idea of what to expect.
But there is a better way. We need to take the lessons from Moneyball and apply them to politics. And it isn’t the case that we have to somehow come up with money to engage in new research or anything like that. This is what political scientists do. And what they’ve been doing for decades. Any serious political analyst needs to regularly be reading the work of political scientists, which can conveniently be found in the political science journals. While these aren’t widely published, they also aren’t very hard to find. Every college library has access to them and Democratic organizations should be making a point to obtain subscriptions, particularly since they aren’t exactly cheap. A great list of political science journals can be found here. If you aren’t reading these journals on a regular basis, then you aren’t up-to-date on the latest research on politics. And if you haven’t read them in the past you really don’t understand how politics actually works in the real world. And you probably lose a lot.
A lot of people like to act like politics is some kind of vague, amorphous thing that we can’t really understand and we just have to get a “feel” for it. This is complete and utter nonsense. While people’s opinions are subjective and often illogical outside of their specific context, the simple fact is we can study those opinions and beliefs and actions and it is clear that there are patterns and that if we know and understand these patterns based on scientific research it becomes really easy to predict behavior. Opinion polls are often seen as this crazy, unpredictable thing that are highly inaccurate, but that’s really not the case. Why has Nate Silver been so accurate? Is he lucky? Is he just really good at having the right gut feeling? Or does he really understand how polling works and has trust in what the polls tell him if they are conducted properly? It is definitively that last one.
To take the polling example a little deeper. Properly conducted polls are highly accurate. For decades, the final polls before election day are almost always within 1-2% of being completely accurate. This is why Silver has been so correct, because he’s paying attention to the polls and using them correctly. Here’s how polling works when it’s done properly. There are two key principles for any scientifically valid poll. The first is randomness. The participants in the poll have to be randomly selected. You can’t just pick your friends and poll participants can select themselves for participation (one of the many reasons online polls like those favored by networks like CNN are meaningless). Second, but equally important, is the fact that the poll has to be representative of the population it claims to represent. If you want to know what “Americans” think and you poll only people who live in Miami, your poll will be invalid, since the population of Miami is not very similar to the overall population of the U.S. The most accurate political polls closely represent the demographic breakdown of the population they claim to represent. While the percentages don’t have to be exact, they should closely represent the population the claim to be representative of — in terms of gender, race, party identification, cell phone vs. landline, etc. The more different these numbers are in the sample compared to the general population, the more inaccurate the poll is likely to be. If you noticed the pattern of more conservative polls being more wrong than the mainstream polls, it was because they oversampled Republicans and undersampled Hispanics and cell phone-only users.
If a sample is representative and random, it only needs to be big enough to be accurate. The larger the sample, the more accurate it tends to be, although after a few thousand, the difference is so miniscule as to make the cost of expanding the sample not worth it. Polls in the 600-800 range tend to be a really good cost-to-accuracy ratio, it seems. And while non-scientists can question how such a small group of people can represent a whole population, suffice it to say that this has been tested over and over and over and over again and found to be highly accurate. Also keep in mind that any individual poll, even if correctly administered, could still be an outlier. This is why everyone looks at the averages of the polls rather than just an individual poll. And, of course, the wording of the questions in a poll can be highly influential on the outcome and can greatly reduce the validity of the poll. Also, polls are a snapshot in time and the farther away you get from the event, the less predictive they become of the event. The overall point, finally, as Nate Silver shows, is that if properly conducted and interpreted correctly, polls are very accurate. Particularly national polls that are taken frequently. The more local the poll gets, the less likely it is to be accurate, because local polling tends to violate many of these principles, often out of necessity, particularly in the lack of money to conduct accurate polls.
The rest of scientific research, even political science research, is just as accurate as the polls are when conducted properly and interpreted correctly. More on this, and Political Moneyball, later.