The Signal and the Noise: Why So Many Predictions Fail — but Some Don’t by Nate Silver

There is a tendency in our planning to confuse the unfamiliar with the improbable. The contingency we have not considered seriously looks strange; what looks strange is thought improbable; what is improbable need not be considered seriously. –Thomas Schelling (419)

Nate Silver’s award winning book The Signal and the Noise enlightens us to our many biases and proposes a way to sift the signal of a trend from the noise created by unmeaningful data. He recommends thinking about prediction and probability in a Bayesian way. Bayes’s theorem of probability and prediction makes us reduce our biases by stating our beliefs upfront. (451) It forces us close the gap “between what we know and what we think we know” (448) However, we must not forget about the unknown unknowns.

Political pundits are no better than average at guessing political races, and often receive camera time because of histrionics instead of prediction accuracy. Weathermen have a more challenging fiet. Weatherman must compensate for the wet bias because “people notice one type of mistake-the failure to predict rain-more than another kind, false alarms. If it rains when it isn’t supposed to, they curse the weatherman for ruining their picnic, whereas an unexpectedly sunny day is taken as serendipitous bonus.” (135) Forecasters must weigh the economic value of each forecast and ask does this “help the public and policy makers to make better decisions?” (134) By factoring economic value into the forecast for the public, the forecast is reduced in accuracy. Forecasting a hurricane warning that will trigger an evacuation is expensive and dangerous. Forecasters have to juggle believability and the economical value of each prediction. “That we can sometimes predict nature’s course, however, does not mean we can alter it. Nor does a forecast do much good if there is no one willing to listen to it. (like in Katrina)” (110)

Silver intelligently explains Moody’s oversight in the credit debacle, the herding behavior of mutual fund managers to avoid getting fired which causes market bubbles, the strategies of professional poker players, and the heuristics behind a chess computer program. In regard to baseball GM Billy Beane’s philosophy to “collect as much information as possible, but then be as rigorous and disciplined as possible when analyzing it,” (100) Silver cautions, “but statheads can have their biases too. One of the most pernicious ones is to assume that if something cannot easily be quantified, it does not matter.”(92) Further, if someone confidently presents “an overly specific solution to a general problem” he may be guilty of overfitting his model and omitting data. (163) He reminds us that “Computers are very,very fast at making calculations. Moreover, they can be counted on to calculate faithfully-without getting tired or emotional or changing their mode of analysis in midstream” (289) Humans, however, must decide what we must calculate and identify what cannot calculate. The book is a fun read and a necessity for everyone as the world becomes more datafied.

I highly recommend this book.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s