Emphasizing the Information we know versus the information that matters

Malcolm Gladwells's new book, Blink, has this terrific anecdote on the Cook County emergency room.  Turns out the Director of the  ER wanted to figure out a better way to diagnose when people show up to the ER complaining of chest pain whether they are having a heart attack or just a heartburn.  Since the costs of being wrong in the diagnosis is catastrophic, physicians tended to be conservative and keep patients in observation for a day if they had the slightest doubt.  This resulted in lots of folks staying overnight (personal inconvenience) and a lot of hospital beds lost unnecessarily (major financial and health inconvenience). 

A young cardiologist (then) by the name of Lee Goldman (full disclosure: he is a colleague of my wife) developed a decision tree that, using only four factors, evaluates the likelihood of heart attacks.  Here is a summary of the results using Goldman's decision tree:

For two years, data were collected, and in the end, the result wasn't even close. Goldman's rule won hands down in two directions: it was a whopping 70 percent better than the old method at recognizing patients who weren't actually having a heart attack. At the same time, it was safer. The whole point of chest pain prediction is to make sure that patients who end up having major complications are assigned right away to the coronary and intermediate units. Left to their own devices, the doctors guessed right on the most serious patients somewhere between 75 and 89 percent of the time. The algorithm guessed right more than 95 percent of the time. (pp. 135-136)

Goldman's decision tree needed drastically less information than the usual physical exam and questioning.  In fact additional research showed that extensive exam and questioning by the doctors actually made their diagnosis WORSE.  Turns out that the additional information obtained by the doctors led them to weigh it more than the simple 4 item criteria laid out by Lee Goldman.

This is not surprising.

As a venture capitalist, I do various kinds of diligence on prospective companies.  Occasionally I meet a company who is in a market or whose founders are intimately acquainted with someone I know.  In those cases, I reach out to the contact and ask him/her for their opinion on the company or the person  I end up weighing their opinion very highly in my decision making even though the importance of that information is not clear.  I am weighing more heavily the information I can access versus the information that *MATTERS*. 

Of course the trick is knowing what is the information that MATTERS.  There is no easy answer here but its important to be cognizant that just because you don't know what is the information that really MATTERS does not mean you should value the information you have received much higher than other pieces of data.