In the fall of 1906, the well-bred British statistician Francis Galton left his home to check out the West of England Fat Stock and Poultry Exhibition, a regional fair. Galton was a man obsessed with two things: breeding and statistics. What better place for an eminent Victorian to be than a fair where he could witness first-hand the effects of good and bad breeding? As he walked through the fair, Galton stumbled across a weight-judging competition.

The fair's organizers had selected one fat ox and soon hundreds of fairgoers were lining up to place bets on the weight of the finely dressed carcass.

These bettors came from all walks of life, some may hardly have known as much as the butchers and farmers with their inside information. Based on no more than smalltalk, these non-experts competed offering their best guess. Galton did not think much of the enterprise —employing the argument that the bettors like most voters were rationally ignorant. "The average competitor," he later wrote, "was probably as well fitted for making a just estimate of the dressed weight of the ox, as an average voter is of judging the merits of most political issues on which he votes."

Alas, the poor folks would have no clue, surmised Galton because they were merely average. In keeping with the consciousness of his refined class (this was the turn of the century, after all) Galton believed that average people were capable of very little. So being the inquisitive man of science and having an open mind, Galton decided to run an experiment. After the contest was over and prizes distributed, Galton borrowed the tickets from the fair's organizers and ran statistical tests on them. He then calculated the mean value and found much to his surprise (or shock) that the crowd collectively found that the poor old fat ox weighed 1,197 pounds. Its actual weight was just one pound more: 1,198. "The result seems more creditable to the trustworthiness of a democratic judgment than might have been expected." And thus did the effete gentlemen from Plymouth discover the wisdom of the crowd.

James Surowiecki recalls Galton's experiment on an autumnal afternoon as a perfect introduction to his remarkable and fascinating polemic on the wisdom that two heads are clearly better than one. Or put another way: There is something to be said for groupthink, properly understood.

Surowiecki, a financial writer for the New Yorker, thinks that groups — or better yet aggregates of individuals — can collectively make wise decisions. Wisdom is more widespread than we think. Surowiecki finds the wisdom of crowds at work in Iowa Electronic Markets and Google; the open software movement and most collaborative scientific research.

But more telling is how, without any central authority, we manage each day to walk into crowds of pedestrians without knocking them down or drive upon interstate highways without much damage considering the sheer numbers of people on the road. People, if we care to look, have a great ability to coordinate their actions without many rules or rulers.

Nobel Prize economist Vernon Smith proved that "naive, unsophisticated agents" can coordinate themselves "to achieve complex, mutually beneficial ends." Surowiecki is a market enthusiast and he wishes more people could marvel, like Frederic Bastiat, at how well markets move products from the people who can produce them most cheaply to the people who want them most fervently. "What's mysterious is that this is supposed to happen without anyone seeing the whole picture of what the market is doing, and without anyone knowing in advance what a good answer will look like," notes the author.

On the other hand for example, he shows where there is no wisdom: NASA (as it confronted the Challenger and Columbia catastrophes), network television executives (who continue to rely on the increasingly obsolete Neilson television ratings system), movie theater owners (who refuse to part with a one-price-fits-all strategy for turkeys like Gigli) and Long Term Capital Management, the hedge fund that nearly brought the world to financial calamity in the late 1990s. These, along with stock market bubbles are examples, where very smart people working collectively produce poor results.

Surowiecki's thesis runs up against the mythic ethos of someone like Henry David Thoreau who once said, "The mass never comes up to the standard of its best member, but on the contrary degrades itself to a level with the lowest." The author Charles Mackay of the 1841 classic, Extraordinary Popular Delusions and the Madness of Crowds scoffed that crowds could ascertain anything. More recently, Ralph Cordiner, former chairman of General Electric, once boasted that "if you can name for me one great discovery or decision that was made by a committee, I will find you the one man in that committee who had the lonely insight — while he was shaving or on his way to work, or maybe while the rest of the committee was chattering away — the lonely insight that solved the problem and was the basis for the decision."


What exactly did MacKay and Thoreau and later Galton and Cordiner overlook? While there is much room for the pursuits of entrepreneurs and rugged individualists (Surowiecki gives an approving nod to "methodological individualism"), the author places enormous faith in a synthesis where independence — even to the point of irrationality within a group — guards against errors. More specifically, the more influence a member of a group has upon another member the less likely good decisions will be made.

He writes, "Diversity and independence are important because the best collective decisions are the product of disagreement and contest, not consensus and compromise. An intelligent group, especially when confronted with cognition problems, does not ask its members to modify their positions in order to let the group reach a decision everyone can be happy with. Instead, if figures out how to use mechanisms — like market prices, or intelligent voting systems — to aggregate and produce collective judgments that represent not what any one person in the group thinks but rather, in some sense, what they all think. Paradoxically, the best way for a group to be smart is for each person in it to think and act as independently as possible."

This is distinct from the idea of herding. Professional money managers rarely beat the market because they herd and according to Surowiecki "in doing so, they destroy whatever information advantage they might have had, since the mimicking managers are not trading on their own information but are relying on the information of others."

To make his argument cogent and compelling, Surowiecki relies on the work of many economists from diverse schools of thought from John Maynard Keynes to Ronald Coase to the growing adherents of behavioral economics. But no one can approach the enormous influence of the Austrian Frederich von Hayek upon Surowiecki where the Nobel Laureate's theory of market coordination holds sway.

In his seminal work "The Uses of Knowledge in Society," Hayek stressed the importance of how a price system could coordinate the resources of many disparate individuals.(1) This was possible because many of the players — without regard to a central authority — retained "tacit knowledge." This knowledge emerges from the experience of each player and is often contingent on local conditions that no administrator can replicate. This tacit knowledge is crucial to efficiency because the ensuing order enables producers to properly allocate their products to consumers who are most willing to pay for them. A market system based on prices that serve as signals demonstrates the allocative superiority of capitalism over socialism.

But as Surowiecki acknowledges, Hayek did not view 'the market as a kind of giant calculating machine." For Hayek, the virtues of the market lay in its decentralized nature with buyers and sellers achieving their ends only through the price system. Surowiecki believes, with good reason, that Hayek's fear of socialism prevented him from realizing the benefits of aggregating diverse opinions to glean the wisdom of the crowd. Hayek was right to worry about socialist calculation but he did not foresee the dominance of technology and the way that technology could be used to centralize decision making away from his decentralized ideal. Both Hayek and Surowiecki rely on the premise that local knowledge —intuition, limited personal information, rule of thumb — are the endgame of coordination. The decline in price for computing power has enabled industrial organizations to substitute aggregated information for local knowledge

The legal scholar Richard Epstein notes that this shift away from local knowledge is the result of cheaper and readily available technology. "It is not that local knowledge is disreputable. It is that it can become ineffective in at least some walks of life relative to more systemic, number-crunching attacks on certain problems." (2) The increasing role of technology deployed by both governments and corporations even at the expense of local knowledge does not diminish Surowiecki's brilliant "autobiography of an idea." The author's thesis is vital because it renews a faith in a democratic ethos that can generate as good a solution as that of an expert, public official or a computer. "State a moral case for a ploughman and a professor, wrote Thomas Jefferson, "the former will decide it as well and often better than the latter because he has not been led astray by artificial rules."

(Endnotes)
1 For an exposition of Hayek's argument in favor of local knowledge, see F.A. Hayek, "The Use of Knowledge in Society," American Economic Review, XXXV, No. 4; September, 1945, pp. 519-30; Internet, available at http://www.econlib.org/library/Essays/hykKnw1.html, accessed February 28, 2005.
2 Richard A. Epstein, "The Uses and Limits of Local Knowledge: A Cautionary Note on Hayek," New York University Journal of Law and Liberty, I, No. 0; Internet; available at http://www.law.nyu.edu/journals/liberty/Images/11%20-%20Epstein.pdf;accessed February 28, 2005.