Illustration: Viktor Koen

In August 2004, Todd Proebsting, a researcher in Microsoft’s platform and services division, was approached by a manager in the company’s testing organization who had spent months helping to create a piece of software to be used by other Microsoft programmers. Although it was an internal product, the software still had a rigid development schedule and an official launch date: November 2004, just a few months away.

The manager had heard a talk by Proebsting about something called a prediction market, a sort of stock market for ideas, in which Microsoft employees would in effect place bets on predictions, instead of on racehorses or football teams. A lot was riding on the timely completion of the testing software. “You said that a market could be used to predict schedules,” the manager said. “I want to know when my team will finish writing the software.”

Proebsting created a market with six possible bets: that the product would ship before November, in November, in December, in January, in February, or later than February. His pool of bettors included members of the development team itself, other developers, and program managers from related teams, as well as internal “customers”—the programmers within Microsoft who would use the software. He showed them all how to use the market, gave them each US $50 with which to wager, and then sat back and watched prices fluctuate.

“All six months were started equally at 16 2/3 cents on the dollar,” Proebsting says, meaning that you only had to bet that amount to win $1 if you were right. “Within seconds, the pre-November market went to $0.00 and never moved from there.” So much for beating the deadline. “The November date went down to 1.2 cents in about 3 minutes.” So much for meeting the deadline.

“The director of the group came to see me. He asked, ‘What have you done?’ ”

“No one believes your product will ship on time,” Proebsting told him. The director replied, “No one on the team is telling me this.”

After discussing things with his development team, the director came to accept what the market was “saying.” He decided to cut some of the software features that were holding things up. “And the price of the markets started to reflect that—the November price rose,” Proebsting says. “Then the internal customers got wind of the fact that some of their favorite features were being cut and demanded their features back. So the market then reflected that!” In other words, the markets that predicted the software would be very late went back up. “In the end,” Proebsting says, “the product shipped in February, which is what the market predicted.”

ArcelorMittal, Best Buy, General Electric, Hewlett-Packard, Nokia, and Samsung have all begun tapping into the “wisdom of crowds” to help them predict public reaction to new products, the future price of a commodity, or sales revenue in the next quarter. In the past few years, the technique has really taken off, with at least a dozen start-ups competing for business in the field. Some offer software and services to help companies tap the wisdom of their workers or the outside world. Others create markets that allow anyone to go to a Web site to bet or even to pose a question that can be bet on.

Chris F. Masse, a financial consultant in Sophia Antipolis, France, who specializes in prediction markets, says that by 2010, “10 percent of Fortune 500 companies will have gone public about their use of internal prediction markets, and probably another 10 percent will be testing some projects.”

Among the leaders in the emerging field are Consensus Point, in Nashville, which counts GE and Best Buy among its clients, and Inkling, a Chicago start-up that designs internal markets. Computer-game manufacturer Electronic Arts, in Redwood City, Calif., uses Inkling to predict industry assessments of its products. There is, inevitably, an open-source software for prediction markets: the Zocalo project, which is run by software engineer Chris Hibbert and affiliated with North Carolina State University.

Meanwhile, the number of public markets is growing at an astonishing rate. You can already predict the popularity of Web sites, new movies, computer game hardware, financial instruments, and the eventual success of a book proposal or a musical artist’s first CD. You can bet on the success of sports stars or entire teams in an absurdly varied number of ways—including how many goals a team will score in a season and the number of fans who will attend its games. You can guess how many inches of snow will fall in New York City’s Central Park in December, when Osama bin Laden will be captured, and the outcome of a 2008 Senate race. At Smarkets, based in Austin, Texas, you can even buy shares representing relative sales of Amazon products, guessing if the retailer will sell more books, iPods, or 500-thread-count sheets next month.

Prediction markets have caught on so well in the United States that they’ve even attracted the attention of the state and federal regulators who oversee lotteries, casino gambling, and racetrack wagering. So in May of this year, a group of distinguished economists including Nobel laureate Kenneth Arrow, of Stanford, issued a statement asking that prediction markets be exempt from gambling regulations. In the statement, the group declared that “using these markets as forecasting tools could substantially improve decision making in the private and public sectors.”

Users bet on one outcome (the month of a product launch, a political candidate, or a sports team) more than another, which establishes a favorite and a long shot, just as in a horse race. As explained by financial journalist James Surowiecki, who wrote the 2004 book The Wisdom of Crowds, “under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them.”

Prediction markets aren’t perfect. They failed spectacularly to predict Howard Dean’s startling 2004 defeat in the Iowa caucuses and Michael Jackson’s high-profile acquittal in 2005. In the 2007 National Collegiate Athletic Association men’s basketball tournament, they trailed 30 different expert sports analysts. But all in all, they consistently do better than other methods of predicting events. In May, Intel published the results of a comprehensive 18-month study of prediction markets. It found that they were as least as accurate as official forecasts by Intel management, and often better by as much as 20 percent.