Fooled by Randomness (21 page)

The Earnings Season: Fooled by the Results

Wall Street analysts, in general, are trained to find the accounting tricks that companies use to hide their earnings. They tend to (occasionally) beat the companies at that game. But they are neither trained to reflect nor to deal with randomness (nor to understand the limitations of their methods by introspecting—stock analysts have both a worse record and higher idea of their past performance than weather forecasters). When a company shows an increase in earnings once, it draws no immediate attention. Twice, and the name starts showing up on computer screens. Three times, and the company will merit some buy recommendation.

Just as with the track record problem, consider a cohort of 10,000 companies that are assumed on average to barely return the risk-free rate (i.e., Treasury bonds). They engage in all forms of volatile business. At the end of the first year, we will have 5,000 “star” companies showing an increase in profits (assuming no inflation), and 5,000 “dogs.” After three years, we will have 1,250 “stars.” The stock review committee at the investment house will give your broker their names as “strong buys.” He will leave a voice message that he has a hot recommendation that necessitates immediate action. You will be e-mailed a long list of names. You will buy one or two of them. Meanwhile, the manager in charge of your 401(k) retirement plan will be acquiring the entire list.

We can apply the reasoning to the selection of investment categories—as if they were the managers in the example above. Assume you are standing in 1900 with hundreds of investments to look at. There are the stock markets of Argentina, Imperial Russia, the United Kingdom, Unified Germany, and plenty of others to consider. A rational person would have bought not just the emerging country of the United States, but those of Russia and Argentina as well. The rest of the story is well-known; while many of the stock markets like those of the United Kingdom and the United States fared extremely well, the investor in Imperial Russia would have no better than medium-quality wallpaper in his hands. The countries that fared well are not a large segment of the initial cohort; randomness would be expected to allow a few investment classes to fare extremely well. I wonder if those “experts” who make foolish (and self-serving) statements like “markets will always go up in any twenty-year period” are aware of this problem.

COMPARATIVE LUCK

A far more acute problem relates to the outperformance, or the comparison, between two or more persons or entities. While we are certainly fooled by randomness when it comes to a single times series, the foolishness is compounded when it comes to the comparison between, say, two people, or a person and a benchmark. Why? Because
both
are random. Let us do the following simple thought experiment. Take two individuals, say, a person and his brother-in-law, launched through life. Assume equal odds for each of good and bad luck. Outcomes: lucky-lucky (no difference between them), unlucky-unlucky (again, no difference), lucky-unlucky (a large difference between them), unlucky-lucky (again, a large difference).

I recently attended for the first time a conference of investment managers where I sat listening to a very dull presenter comparing traders. His profession is to select fund managers and package them together for investors, something called “funds of funds” and I was listening to him as he was pouring out numbers on the screen. The first revelation was that I suddenly recognized the speaker, a former colleague biologically transformed by the passage of time. He used to be crisp, energetic, and nice; he became dull, portly, and inordinately comfortable with success. (He was not rich when I knew him—can people react to money in different ways? Do some take themselves seriously while others do not?) The second revelation was that while I suspected that he was fooled by randomness, the extent had to be far greater than one could imagine, particularly with the survivorship bias. A back of the envelope calculation showed that at least 97% of what he was discussing was just noise. The fact that he was
comparing
performances made the matter far worse.

Cancer Cures

When I return home from an Asian or European trip, my jet lag often causes me to rise at a very early hour. Occasionally, though very rarely, I switch on the TV set searching for market information. What strikes me in these morning explorations is the abundance of claims by the alternative medicine vendors of the curing power of their products. These no doubt are caused by the lower advertising rates at that time. To prove their claim, they present the convincing testimonial of someone who was cured thanks to their methods. For instance, I once saw a former throat cancer patient explaining how he was saved by a combination of vitamins for sale for the exceptionally low price of $14.95—in all likelihood he was sincere (although of course compensated for his account, perhaps with a lifetime supply of such medicine). In spite of our advances, people still believe in the existence of links between disease and cure based on such information, and there is no scientific evidence that can convince them more potently than a sincere and emotional testimonial. Such testimonial does not always come from the regular guy; statements by Nobel Prize winners (in the wrong discipline) could easily suffice. Linus Pauling, a Nobel Prize winner in chemistry, was said to believe in vitamin C’s medicinal properties, himself ingesting massive daily doses. With his bully pulpit, he contributed to the common belief in vitamin C’s curative properties. Many medical studies, unable to replicate Pauling’s claims, fell on deaf ears as it was difficult to undo the testimonial by a “Nobel Prize winner,” even if he was not qualified to discuss matters related to medicine.

Many of these claims have been harmless outside of the financial profits for these charlatans—but many cancer patients may have replaced the more scientifically investigated therapies, in favor of these methods, and died as a result of their neglecting more orthodox cures (again, the nonscientific methods are gathered under what is called “alternative medicine,” that is, unproven therapies, and the medical community has difficulties convincing the press that there is only one medicine and that alternative medicine is not medicine). The reader might wonder about my claims that the user of these products could be sincere, without it meaning that he was cured by the illusory treatment. The reason is something called “spontaneous remission,” in which a very small minority of cancer patients, for reasons that remain entirely speculative, wipe out cancer cells and recover “miraculously.” Some switch causes the patient’s immune system to eradicate all cancer cells from the body. These people would have been equally cured by drinking a glass of Vermont spring water or chewing on dried beef as they were by taking these beautifully wrapped pills. Finally, these spontaneous remissions might not be so spontaneous; they might, at the bottom, have a cause that we are not yet sophisticated enough to detect.

The late astronomer Carl Sagan, a devoted promoter of scientific thinking and an obsessive enemy of nonscience, examined the cures from cancer that resulted from a visit to Lourdes in France, where people were healed by simple contact with the holy waters, and found out the interesting fact that, of the total cancer patients who visited the place, the cure rate was, if anything, lower than the statistical one for spontaneous remissions. It was lower than the average for those who did not go to Lourdes! Should a statistician infer here that cancer patients’ odds of surviving deteriorates after a visit to Lourdes?

Professor Pearson Goes to Monte Carlo (Literally):
Randomness Does Not Look Random!

At the beginning of the twentieth century, as we were starting to develop techniques to deal with the notion of random outcomes, several methods were designed to detect anomalies. Professor Karl Pearson (father of Egon Pearson of Neyman-Pearson fame, familiar to every person who sat in a statistics 101 class) devised the first test of nonrandomness (it was in reality a test of deviation from normality, which, for all intents and purposes, was the same thing). He examined millions of runs of what was called a Monte Carlo (the old name for a roulette wheel) during the month of July 1902. He discovered that, with a high degree of statistical significance (with an error of less than one to a billion), the runs were not purely random. What! The roulette wheel was not random! Professor Pearson was greatly surprised at the discovery. But this result in itself tells us nothing; we know that there is no such thing as a pure random draw, for the outcome of the draw depends on the quality of the equipment. With enough minutiae one would be able to uncover the nonrandomness somewhere (e.g., the wheel itself may not have been perfectly balanced or perhaps the spinning ball was not completely spherical). Philosophers of statistics call this the
reference case problem
to explain that there is no true attainable randomness in practice, only in theory. Besides, a manager would question whether such nonrandomness can lead to any meaningful, profitable rules. If I need to gamble $1 on 10,000 runs and expect to make $1 for my efforts, then I would do much better in the part-time employment of a janitorial agency.

But the result bears another suspicious element. Of more practical relevance here is the following severe problem about nonrandomness. Even the fathers of statistical science forgot that a random series of runs need not exhibit a pattern to look random; as a matter of fact, data that is perfectly patternless would be extremely suspicious and appear to be man-made. A single random run is bound to exhibit some pattern—if one looks hard enough. Note that Professor Pearson was among the first scholars who were interested in creating artificial random data generators, tables one could use as inputs for various scientific and engineering simulations (the precursors of our Monte Carlo simulator). The problem is that they did not want these tables to exhibit any form of regularity. Yet real randomness does not look random!

I would further illustrate the point with the study of a phenomenon well-known as cancer clusters. Consider a square with 16 random darts hitting it with equal probability of being at any place in the square. If we divide the square into 16 smaller squares, it is expected that each smaller square will contain one dart on average—but only on average. There is a very small probability of having exactly 16 darts in 16 different squares. The average grid will have more than one dart in a few squares, and no dart at all in many squares. It will be an exceptionally rare incident that no (cancer) cluster would show on the grid. Now, transpose our grid with the darts in it to overlay a map of any region. Some newspaper will declare that one of the areas (the one with more than the average of darts) harbors radiation that causes cancer, prompting lawyers to start soliciting the patients.

The Dog That Did Not Bark: On Biases
in Scientific Knowledge

By the same argument, science is marred by a pernicious survivorship bias, affecting the way research gets published. In a way that is similar to journalism, research that yields no result does not make it to print. That may seem sensible, as newspapers do not have to have a screaming headline saying that nothing new is taking place (though the Bible was smart enough to declare
ein chadash tachat hashemesh—
“nothing new under the sun,” providing the information that things just do recur). The problem is that a finding of absence and an absence of findings get mixed together. There may be great information in the fact that
nothing took place.
As Sherlock Holmes noted in the
Silver Blaze
case—the curious thing was that the dog did not bark. More problematic, there are plenty of scientific results that are left out of publications because they are not statistically significant, but nevertheless provide information.

I HAVE NO CONCLUSION

I am frequently asked the question “When is it truly not luck?” There are professions in randomness for which performance is low in luck, like casinos, which manage to tame randomness. In finance? Perhaps. All traders are not speculative traders: There exists a segment called market makers whose job is to derive, like bookmakers, or even like store owners, an income against a transaction. If they speculate, their dependence on the risks of such speculation remains too small compared to their overall volume. They buy at a price and sell to the public at a more favorable one, performing large numbers of transactions. Such income provides them some insulation from randomness. Such category includes floor traders on the exchanges, bank traders who “trade against order flow,” money changers in the souks of the Levant. The skills involved are sometimes rare to find: Fast thinking, alertness, a high level of energy, an ability to guess from the voice of the seller her level of nervousness; those who have them make a long career (that is, perhaps a decade).They never make it big, as their income is constrained by the number of customers, but they do well probabilistically. They are, in a way, the dentists of the profession.

Outside of this very specialized bookmaker-style profession, to be honest, I am unable to answer the question of who’s lucky or unlucky. I can tell that person A seems less lucky than person B, but the confidence in such knowledge can be so weak as to be meaningless. I prefer to remain a skeptic. People frequently misinterpret my opinion. I never said that every rich man is an idiot and every unsuccessful person unlucky, only that in absence of much additional information it is preferable to reserve one’s judgment. It is safer.

Ten

•

LOSER TAKES ALL—ON THE NONLINEARITIES OF LIFE

N
ext I put the platitude
life is unfair
under some examination, but from a new angle. The twist: Life is unfair in a
nonlinear
way. This chapter is about how a small advantage in life can translate into a highly disproportionate payoff, or, more viciously, how no advantage at all, but a very, very small help from randomness, can lead to a bonanza.

THE SANDPILE EFFECT

First we define
nonlinearity.
There are many ways to present it, but one of the most popular ones in science is what is called the sand-pile effect, which I can illustrate as follows. I am currently sitting on a beach in Copacabana, in Rio de Janeiro, attempting to do nothing strenuous, away from anything to read and write (unsuccessfully, of course, as I am mentally writing these lines). I am playing with plastic beach toys borrowed from a child, trying to build an edifice—modestly but doggedly attempting to emulate the Tower of Babel. I continuously add sand to the top, slowly raising the entire structure. My Babylonian relatives thought they could thus reach the heavens. I have more humble designs—to test how high I can go before it topples. I keep adding sand, testing to see how the structure will ultimately collapse. Unused to seeing adults build sandcastles, a child looks at me with amazement.

In time—and much to the onlooking child’s delight—my castle inevitably topples to rejoin the rest of the sand on the beach. It could be said that the last grain of sand is responsible for the destruction of the entire structure. What we are witnessing here is a nonlinear effect resulting from a linear force exerted on an object. A very small additional input, here the grain of sand, caused a disproportionate result, namely the destruction of my starter Tower of Babel. Popular wisdom has integrated many such phenomena, as witnessed by such expressions as “the straw that broke the camel’s back” or “the drop that caused the water to spill.”

These nonlinear dynamics have a bookstore name, “chaos theory,” which is a misnomer because it has nothing to do with chaos. Chaos theory concerns itself primarily with functions in which a small input can lead to a disproportionate response. Population models, for instance, can lead to a path of explosive growth, or extinction of a species, depending on a very small difference in the population at a starting point in time. Another popular scientific analogy is the weather, where it has been shown that a simple butterfly fluttering its wings in India can cause a hurricane in New York. But the classics have their share to offer as well: Pascal (he of the wager in
Chapter 7
) said that if Cleopatra’s nose had been slightly shorter, the world’s fate would have changed. Cleopatra had comely features dominated by a thin and elongated nose that made Julius Caesar and his successor, Marc Antony, fall for her (here the intellectual snob in me cannot resist dissenting against conventional wisdom; Plutarch claimed that it was Cleopatra’s skills in conversation, rather than her good looks, that caused the maddening infatuation of the shakers and movers of her day; I truly believe it).

Enter Randomness

Things can become more interesting when randomness enters the game. Imagine a waiting room full of actors queuing for an audition. The number of actors who will win is clearly small, and they are the ones generally observed by the public as representative of the profession, as we saw in our discussion on survivorship bias. The winners would move into Bel Air, feel pressure to acquire some basic training in the consumption of luxury goods, and, perhaps owing to the dissolute and unrhythmic lifestyle, flirt with substance abuse. As to the others (the great majority), we can imagine their fate; a lifetime of serving foamed
caffe latte
at the neighboring Starbucks, fighting the biological clock between auditions.

One may argue that the actor who lands the lead role that catapults him into fame and expensive swimming pools has some skills others lack, some charm, or a specific physical trait that is a perfect match for such a career path. I beg to differ. The winner may have some acting skills, but so do all of the others, otherwise they would not be in the waiting room.

It is an interesting attribute of fame that it has its own dynamics. An actor becomes known by some parts of the public because he is known by other parts of the public. The dynamics of such fame follow a rotating helix, which may have started at the audition, as the selection could have been caused by some silly detail that fitted the mood of the examiner on that day. Had the examiner not fallen in love the previous day with a person with a similar-sounding last name, then our selected actor from that particular sample
history
would be serving
caffe latte
in the intervening sample
history.

Learning to Type

Researchers frequently use the example of QWERTY to describe the vicious dynamics of winning and losing in an economy, and to illustrate how the final outcome is more than frequently the undeserved one. The arrangement of the letters on a typewriter is an example of the success of the least deserving method. For our typewriters have the order of the letters on their keyboard arranged in a nonoptimal manner, as a matter of fact in such a nonoptimal manner as to slow down the typing rather than make the job easy, in order to avoid jamming the ribbons as they were designed for less electronic days. Therefore, as we started building better typewriters and computerized word processors, several attempts were made to rationalize the computer keyboard, to no avail. People were trained on a QWERTY keyboard and their habits were too sticky for change. Just like the helical propulsion of an actor into stardom, people patronize what other people like to do. Forcing rational dynamics on the process would be superfluous, nay, impossible. This is called a
path dependent outcome,
and has thwarted many mathematical attempts at modeling behavior.

It is obvious that the information age, by homogenizing our tastes, is causing the unfairness to be even more acute—those who win capture almost all the customers. The example that strikes many as the most spectacular lucky success is that of the software maker Microsoft and its moody founder Bill Gates. While it is hard to deny that Gates is a man of high personal standards, work ethics, and above-average intelligence, is he the best? Does he
deserve
it? Clearly not. Most people are equipped with his software (like myself ) because other people are equipped with his software, a purely circular effect (economists call that “network externalities”). Nobody ever claimed that it was the best software product. Most of Gates’ rivals have an obsessive jealousy of his success. They are maddened by the fact that he managed to win so big while many of them are struggling to make their companies survive.

Such ideas go against classical economic models, in which results either come from a precise reason (there is no account for uncertainty) or the good guy wins (the good guy is the one who is more skilled and has some technical superiority). Economists discovered path-dependent effects late in their game, then tried to publish wholesale on the topic that otherwise would be bland and obvious. For instance, Brian Arthur, an economist concerned with nonlinearities at the Santa Fe Institute, wrote that chance events coupled with positive feedback rather than technological superiority will determine economic superiority—not some abstrusely defined edge in a given area of expertise. While early economic models excluded randomness, Arthur explained how “unexpected orders, chance meetings with lawyers, managerial whims . . . would help determine which ones achieved early sales and, over time, which firms dominated.”

MATHEMATICS INSIDE AND OUTSIDE THE REAL WORLD

A mathematical approach to the problem is in order. While in conventional models (such as the well-known Brownian random walk used in finance) the probability of success does not change with every incremental step, only the accumulated wealth, Arthur suggests models such as the Polya process, which is mathematically very difficult to work with, but can be easily understood with the aid of a Monte Carlo simulator. The Polya process can be presented as follows: Assume an urn initially containing equal quantities of black and red balls. You are to guess each time which color you will pull out before you make the draw. Here the game is rigged. Unlike a conventional urn, the probability of guessing correctly depends on past success, as you get better or worse at guessing depending on past performance. Thus, the probability of winning increases after past wins, that of losing increases after past losses. Simulating such a process, one can see a huge variance of outcomes, with astonishing successes and a large number of failures (what we called skewness).

Compare such a process with those that are more commonly modeled, that is, an urn from which the player makes guesses with replacement. Say you played roulette and won. Would this increase your chances of winning again? No. In a Polya process case, it does. Why is this so mathematically hard to work with? Because the notion of independence (i.e., when the next draw does not depend on past outcomes) is violated. Independence is a requirement for working with the (known) math of probability.

What has gone wrong with the development of economics as a science? Answer: There was a bunch of intelligent people who felt compelled to use mathematics just to tell themselves that they were rigorous in their thinking, that theirs was a science. Someone in a great rush decided to introduce mathematical modeling techniques (culprits: Leon Walras, Gerard Debreu, Paul Samuelson) without considering the fact that either the class of mathematics they were using was too restrictive for the class of problems they were dealing with, or that perhaps they should be aware that the precision of the language of mathematics could lead people to believe that they had solutions when in fact they had none (recall Popper and the costs of taking science too seriously). Indeed the mathematics they dealt with did not work in the real world, possibly because we needed richer classes of processes—and they refused to accept the fact that no mathematics at all was probably better.

The so-called
complexity theorists
came to the rescue. Much excitement was generated by the works of scientists who specialized in nonlinear quantitative methods—the mecca of those being the Santa Fe Institute near Santa Fe, New Mexico. Clearly these scientists are trying hard, and providing us with wonderful solutions in the physical sciences and better models in the social siblings (though nothing satisfactory there yet). And if they ultimately do not succeed, it will simply be because mathematics may be of only secondary help in our real world. Note another advantage of Monte Carlo simulations is that we can get results where mathematics fails us and can be of no help. In freeing us from equations it frees us from the traps of inferior mathematics. As I said in
Chapter 3
, mathematics is merely a way of thinking and meditating, little more, in our world of randomness.

The Science of Networks

Studies of the dynamics of networks have mushroomed recently. They became popular with Malcolm Gladwell’s book
The Tipping Point,
in which he shows how some of the behaviors of variables such as epidemics spread extremely fast beyond some unspecified critical level. (Like, say, the use of sneakers by inner-city kids or the diffusion of religious ideas. Book sales witness a similar effect, exploding once they cross a significant level of word-of-mouth.) Why do some ideologies or religions spread like wildfire while others become rapidly extinct? How do fads catch fire? How do idea viruses proliferate? Once one exits the conventional models of randomness (the bell curve family of charted randomness), something acute can happen. Why does the Internet hub Google get so many hits as compared to that of the National Association of Retired Veteran Chemical Engineers? The more connected a network, the higher the probability of someone hitting it and the more connected it will be, especially if there is no meaningful limitation on such capacity. Note that it is sometimes foolish to look for precise “critical points” as they may be unstable and impossible to know except, like many things, after the fact. Are these “critical points” not quite points but progressions (the so-called Pareto power laws)? While it is clear that the world produces clusters it is also sad that these may be too difficult to predict (outside of physics) for us to take their models seriously. Once again the important fact is knowing the existence of these nonlinearities, not trying to model them. The value of the great Benoit Mandelbrot’s work lies more in telling us that there is a “wild” type of randomness of which we will never know much (owing to their unstable properties).

Our Brain

Our brain is not cut out for nonlinearities. People think that if, say, two variables are causally linked, then a steady input in one variable should
always
yield a result in the other one. Our emotional apparatus is designed for linear causality. For instance, you study every day and learn something in proportion to your studies. If you do not feel that you are going anywhere, your emotions will cause you to become demoralized. But reality rarely gives us the privilege of a satisfying linear positive progression: You may study for a year and learn nothing, then, unless you are disheartened by the empty results and give up, something will come to you in a flash. My partner Mark Spitznagel summarizes it as follows: Imagine yourself practicing the piano every day for a long time, barely being able to perform “Chopsticks,” then suddenly finding yourself capable of playing Rachmaninov. Owing to this nonlinearity, people cannot comprehend the nature of the rare event. This summarizes why there are routes to success that are nonrandom, but few, very few, people have the mental stamina to follow them. Those who go the extra mile are rewarded. In my profession one may own a security that benefits from lower market prices, but may not react at all until some critical point. Most people give up before the rewards.