Fooled by Randomness (26 page)

You Should Be Dead by Now

This brings to mind another common violation of probability by prime-time TV financial experts, who may be selected for their looks, their charisma, and their presentation skills, but certainly not for their incisive minds. For instance, a fallacy that I saw commonly made by a prominent TV financial guru goes as follows: “The average American is expected to live seventy-three years. Therefore if you are sixty-eight you can expect to live five more years, and should plan accordingly.” She went into precise prescriptions of how the person should invest for a five-more-years horizon. Now what if you are eighty? Is your life expectancy
minus
seven years? What these journalists confuse is the unconditional and conditional life expectancy. At birth, your unconditional life expectancy may be seventy-three years. But as you advance in age and do not die, your life expectancy increases along with your life. Why? Because other people, by dying, have taken your spot in the statistics, for expectation means average. So if you are seventy-three and are in good health, you may still have, say, nine years
in expectation.
But the expectation would change, and at eighty-two, you will have another five years, provided of course you are still alive. Even someone one hundred years old still has a positive conditional life expectation. Such a statement, when one thinks about it, is not too different from the one that says: Our operation has a mortality rate of 1%. So far we have operated on ninety-nine patients with great success; you are our one hundreth, hence you have a 100% probability of dying on the table.

TV financial planners may confuse a few people. This is quite harmless. What is far more worrying is the supply of information by nonprofessionals to professionals; it is to the journalists that we turn next.

The Bloomberg Explanations

I have, on my desk, a machine eponymously called a
Bloomberg
(after the legendary founder Michael Bloomberg). It acts as a safe e-mail service, a news service, a historical-data retrieving tool, a charting system, an invaluable analytical aid, and, not least, a screen where I can see the price of securities and currencies. I have gotten so addicted to it that I cannot operate without it, as I would otherwise feel cut off from the rest of the world. I use it to get in contact with my friends, confirm appointments, and solve some of those entertaining quarrels that put some sharpness into life. Somehow, traders who do not have a Bloomberg address do not exist for us (they have to have recourse to the more plebeian Internet). But there is one aspect of Bloomberg I would dispense with: the journalist’s commentary. Why? Because they engage in explaining things and perpetuate the right-column, left-column confusion in a serious manner. Bloomberg is not the sole perpetrator; it is just that I have not been exposed to newspapers’ business sections over the past decade, preferring to read real prose instead.

As I am writing these lines I see the following headlines on my Bloomberg:

and so on for an entire page. If I translate it well, the journalist claims to provide an explanation for something that amounts to
perfect noise.
A move of 1.03 with the Dow at 11,000 constitutes less than a 0.01% move. Such a move does not warrant an explanation. There is nothing there that an honest person can try to explain; there are no reasons to adduce. But like apprentice professors of comparative literature, journalists being paid to provide explanations will gladly and readily provide them. The only solution is for Michael Bloomberg to stop paying his journalists for providing commentary.

Significance:
How did I decide that it was perfect noise? Take a simple analogy. If you engage in a mountain bicycle race with a friend across Siberia and, a month later, beat him by one single second, you clearly cannot quite boast that you are faster than him. You might have been helped by something, or it can be just plain randomness, nothing else. That second is not in itself significant enough for someone to draw conclusions. I would not write in my pre-bedtime diary:
Cyclist A is better than cyclist B because he is fed with spinach whereas cyclist B has a diet rich in tofu. The reason I am making this inference is because he beat him by 1.3 seconds in a 3,000 mile race.
Should the difference be one week, then I could start analyzing whether tofu is the reason, or if there are other factors.

Causality:
There is another problem; even assuming statistical significance, one has to accept a cause and effect, meaning that the event in the market can be linked to the cause proffered.
Post hoc ergo propter hoc
(it is the consequence because it came after). Say hospital A delivered 52% boys and hospital B delivered the same year only 48%; would you try to give the explanation that you had a boy because it was delivered in hospital A?

Causality can be very complex. It is very difficult to isolate a single cause when there are plenty around. This is called multivariate analysis. For instance, if the stock market can react to U.S. domestic interest rates, the dollar against the yen, the dollar against the European currencies, the European stock markets, the United States balance of payments, United States inflation, and another dozen prime factors, then the journalists need to look at all of these factors, look at their historical effect both in isolation and jointly, look at the stability of such influence, then, after consulting the test statistic, isolate the factor if it is possible to do so. Finally, a proper confidence level needs to be given to the factor itself; if it is less than 90% the story would be dead. I can understand why Hume was extremely obsessed with causality and could not accept such inference anywhere.

I have a trick to know if something
real
in the world is taking place. I have set up my Bloomberg monitor to display the price and percentage change of all relevant prices in the world: currencies, stocks, interest rates, and commodities. By dint of looking at the same setup for years, as I keep the currencies in the upper left corner and the various stock markets on the right, I managed to build an instinctive way of knowing if something serious is going on. The trick is to look only at the large percentage changes. Unless something moves by more than its usual daily percentage change, the event is deemed to be noise. Percentage moves are the size of the headlines. In addition, the interpretation is not linear; a 2% move is not twice as significant an event as 1%, it is rather like four to ten times. A 7% move can be several billion times more relevant than a 1% move! The headline of the Dow moving by 1.3 points on my screen today has less than one billionth of the significance of the serious 7% drop of October 1997. People might ask me: Why do I want everybody to learn some statistics? The answer is that too many people read explanations. We cannot instinctively understand the nonlinear aspect of probability.

Filtering Methods

Engineers use methods to clean up the noise from the signal in the data. Did it ever occur to you while talking to your cousin in Australia or the South Pole that the static on the telephone line could be distinguished from the voice of your correspondent? The method is to consider that when a change in amplitude is small, it is more likely to result from noise—with its likelihood of being a signal increasing exponentially as its magnitude increases. The method is called a smoothing kernel, which has been applied in Figures 11.1 and 11.2. But our auditory system is incapable of performing such a function by itself. Likewise our brain cannot see the difference between a significant price change and mere noise, particularly when it is pounded with unsmoothed journalistic noise.

We Do Not Understand Confidence Levels

Figure 11.1 Unfiltered Data Containing Signal and Noise

Figure 11.2 Same Data with Its Noise Removed

Professionals forget the following reality. It is not the estimate or the forecast that matters so much as the degree of confidence with the opinion. Consider that you are going on a trip one fall morning and need to formulate an idea about the weather conditions prior to packing your luggage. If you expect the temperature to be 60 degrees, plus or minus 10 degrees (say in Arizona), then you would take no snow clothes and no portable electric fan. Now, what if you were going to Chicago, where you are told that the weather, while being 60 degrees, will nevertheless vary by about 30 degrees? You would have to pack winter and summer clothes. Here the expectation of the temperature carries little importance concerning the choice of clothing; it is the variance that matters. Your decision to pack is markedly different now that you are told that the variability would be around 30 degrees. Now let us push the point further; what if you were going to a planet where the expectation is also going to be around 60 degrees, but plus or minus 500 degrees? What would you pack?

We can see that my activity in the market (and other random variables) depends far less on where I think the market or the random variable is going so much as it does on the degree of error I allow around such a confidence level.

An Admission

We close this chapter with the following information: I consider myself as prone to foolishness as anyone I know, in spite of my profession and the time spent building my expertise on the subject. But here is the exception; I know that I am very, very weak on that score. My humanity will try to foil me; I have to stay on my guard. I was born to be fooled by randomness. That will be explored in Part III.

Part III

•

WAX IN MY EARS

Living with Randomitis

O
dysseus, the Homerian hero, had the reputation of using guile to overcome stronger opponents. I find the most spectacular use of such guile was against no other opponent than himself.

In Book 12 of the
Odyssey,
the hero encounters the sirens, on an island not far from the rocks of Charybdis and Scylla. Their songs are known to charm the sailors into madness, causing them irresistibly to cast themselves into the sea off the sirens’ coast, and perish. The indescribable beauty of the sirens’ songs is contrasted with the moldering corpses of sailors who strayed into the area around them. Odysseus, forewarned by Circe, contrives the following ruse. He fills the ears of all his men with wax, to the point of total deafness, and has himself tied to the mast. The sailors are under strict instructions not to release him. As they approach the sirens’ island, the sea is calm and over the water comes the sound of a music so ravishing that Odysseus struggles to get loose, expending an inordinate amount of energy to unrestrain himself. His men tie him even further, until they are safely past the poisoned sounds.

The first lesson I took from the story is not to even attempt to be Odysseus. He is a mythological character and I am not. He can be tied to the mast; I can merely reach the rank of a sailor who needs to have his ears filled with wax.

I AM NOT SO INTELLIGENT

The epiphany I had in my career in randomness came when I understood that I was not intelligent enough, nor strong enough, to even try to fight my emotions. Besides, I believe that I need my emotions to formulate my ideas and get the energy to execute them.

I am just intelligent enough to understand that I have a predisposition to be fooled by randomness—and to accept the fact that I am rather emotional. I am dominated by my emotions—but as an aesthete, I am happy about that fact. I am just like every single character whom I ridiculed in this book. Not only that, but I may be even worse than them because there may be a negative correlation between beliefs and behavior (recall Popper the man). The difference between me and those I ridicule is that I try to be aware of it. No matter how long I study and try to understand probability, my emotions will respond to a different set of calculations, those that my unintelligent genes want me to handle. If my brain can tell the difference between noise and signal, my heart cannot.

Such unintelligent behavior does not just cover probability and randomness. I do not think I am reasonable enough to avoid getting angry when a discourteous driver blows his horn at me for being one nanosecond late after a traffic light turns green. I am fully aware that such anger is self-destructive and offers no benefit, and that if I were to develop anger for every idiot around me doing something of the sort, I would be long dead. These small daily emotions are not rational. But we need them to function properly. We are designed to respond to hostility with hostility. I have enough enemies to add some spice to my life, but I sometimes wish I had a few more (I rarely go to the movies and need the entertainment). Life would be unbearably bland if we had no enemies on whom to waste efforts and energy.

The good news is that there are tricks. One such trick is to avoid eye contact (through the rearview mirror) with other persons in such traffic encounters. Why? Because when you gaze into someone’s eyes, a different part of your brain, the more emotional one, is activated and engaged as the result of the interaction. I try to imagine that the other person is a Martian, rather than a human being. It works sometimes—but it works best when the person presents the appearance of being from a different species. How? I am an avid road cyclist. Recently, as I was riding along with other cyclists, slowing down traffic in a rural area, a small woman in a giant sports utility vehicle opened her window and heaped curses at us. Not only did it not upset me but I did not even interrupt my thoughts to pay attention. When I am on my bicycle, people in large trucks become a variety of dangerous animals, capable of threatening me but incapable of making me angry.

I have, like anyone with strong opinions, a collection of critics among finance academics and economists, annoyed by my attacks on their misuse of probability and unhappy about my branding them as pseudoscientists. I am incapable of taming my emotions when reading their comments. The best I can do is just not read them. Likewise with journalists. Not reading their discussions of markets spares me plenty of emotional expenditure. I will do the same with unsolicited comments on this book. Wax in my ears.

WITTGENSTEIN’S RULER

What is the mechanism that should convince authors to avoid reading comments on their work, except for those they solicit from specified persons for whom they have intellectual respect? The mechanism is a probabilistic method called conditional information: Unless the source of the statement has extremely high qualifications, the statement will be more revealing of the author than the information intended by him. This applies, of course, to matters of judgment. A book review, good or bad, can be far more descriptive of the reviewer than informational about the book itself. This mechanism I also call Wittgenstein’s ruler:
Unless you have confidence in the ruler’s reliability, if you use a ruler to measure a table you may also be using the table to measure the ruler.
The less you trust the ruler‘s reliability (in probability called the
prior
),the more information you are getting about the ruler and the less about the table. The point extends way beyond information and probability. This conditionality of information is central in epistemology, probability, even in studies of consciousness. We will see later extensions with “ten sigma” problems.

The point carries practical implications: The information from an anonymous reader on
Amazon.com
is all about the person, while that of a qualified person, is going to be all about the book. This plays equally in court: Take the O. J. Simpson trial once again. One of the jurors said, “There was not enough blood,” meaning to assess the statistical evidence of what was offered: Such statement reveals very little about the statistical evidence as compared with what it shows about the author of the statement’s ability to make a valid inference. Had the juror been a forensic expert, the ratio of information would have tilted the other way.

The problem is that while such reasoning is central to my thinking, my brain knows it though not my heart: My emotional system does not understand Wittgenstein’s ruler. I can offer the following evidence: A compliment is always pleasant, regardless of its authorship—something manipulators know rather well. Likewise with book reviews or comments on my risk-management strategy.

THE ODYSSEAN MUTE COMMAND

Recall that the accomplishment from which I derive the most pride is my weaning myself from television and the news media. I am currently so weaned that it actually costs me more energy to watch television than to perform any other activity, like, say, writing this book. But this did not come without tricks. Without tricks I would not escape the toxicity of the information age. In the trading room of my company, I have the television set turned on all day with the financial news channel CNBC staging commentator after commentator and CEO after CEO murdering rigor all day long. What is the trick? I have the volume turned completely off. Why? Because when the television set is silent, the babbling person looks ridiculous, exactly the opposite effect as when the sound is on. One sees a person with moving lips and contortions in his facial muscles, taking themselves seriously—but no sound comes out. We are visually but not auditorily intimidated, which causes a dissonance. The speaker’s face expresses some excitement, but since no sound comes out, the exact opposite is conveyed. This is the sort of contrast the philosopher Henri Bergson had in mind in his
Treatise on Laughter,
with his famous description of the gap between the seriousness of a gentleman about to walk on a banana skin and the comical aspect of the situation. Television pundits lose their intimidating effect;they even look ridiculous. They seem to be excited about something terribly unimportant. Suddenly pundits become clowns, which is a reason the writer Graham Greene refused to go on television.

I had this idea of stripping people of language while, on a trip, I listened (while brutally jet-lagged) to a speech in Cantonese, a language I do not understand, without the benefit of translation. Since I had no possible clue about his subject, the animated orator lost a large share of his dignity. The idea came to me that perhaps I could use a built-in bias, here prejudice, to offset another built-in bias, our predisposition to take information seriously. It seems to work.

This part, the conclusion of this book, presents the human aspect of dealing with uncertainty. I have personally failed in achieving a general insulation from randomness, but I have managed a few tricks.