Stats for Fun and Profit: Revenge of the Nerds


As performance improvement professionals we need to not only improve processes at a macro level but also to improve the decision tools embedded in the process. The article below highlights several clever techniques for improving how capacity is matched to demand, increasing sales rates for things like online sales of cars, analyzing the effect of selection simplicity on buying behaviors, and customizing the algorithms that determine whether you see an online ad as well as which ad.

One particularly interesting use of data analysis is to use the frequency of search terms such as “job,” “benefits,” or “solitaire” (people out of work often have more time to kill) and the correlation to unemployment claims. As one can see in the chart at the bottom, the correlation as measured by standard deviation from the mean are striking, and could help policy makers use more timely measures rather than rear-view mirror data such as GDP and unemployment figures that lag by weeks.

From the November 24th 2012 edition of The Economist:

On the face of it, economics has had a dreadful decade: it offered no prediction of the subprime or euro crises, and only bitter arguments over how to solve them. But alongside these failures, a small group of the world’s top microeconomists are quietly revolutionising the discipline. Working for big technology firms such as Google, Microsoft and eBay, they are changing the way business decisions are made and markets work.

Take, for example, the challenge of keeping costs down. An important input for a company like Yahoo! is internet bandwidth, which is bought at group level and distributed via an internal market. Demand for bandwidth is quite lumpy, with peaks and troughs at different times of the day. This creates a problem: because spikes in demand must be met, firms run with costly spare capacity much of the time.

This was one of the first questions that Preston McAfee, a former California Institute of Technology professor, looked at when he arrived at Yahoo! in 2007. Mr McAfee, who now works for Google, found that uses of bandwidth fall into two categories: urgent (displaying a web page) and delayable (backups and archiving). He showed how a two-part tariff (high prices when demand peaks, low ones otherwise) could shift less time-sensitive tasks to night-time, allowing Yahoo! to use costly bandwidth more efficiently.

The solution—two types of task, two prices—has intuitive appeal. But economists’ ideas on how to design markets can seem puzzling at first. One example is the question of how much detail an online car auctioneer should reveal about the condition of the vehicles on offer. Common sense would suggest some information—a car’s age and mileage—is essential, but that total transparency about other things (precise details on subpar paintwork) might deter buyers, lowering the auctioneer’s commissions. Academic theory suggests otherwise: in some types of auction more information always raises revenues.

To test the idea, Steve Tadelis of the University of California at Berkeley (now also working for eBay) and Florian Zettelmeyer of Northwestern University set up a trial, randomly splitting 8,000 cars into two groups. The first groups were auctioned with standard information, including age and mileage. The second had a detailed report on the car’s paintwork. The results were striking: cars in the second group had better chances of a sale and sold for higher prices. This effect was most pronounced for cars in poorer condition: the probability of a sale rose by 23%, with prices up by 5%. The extra information meant that buyers were able to spot the type of car they wanted. Competition for cars rose, even the scruffier ones.

But more information is not always better. Studies show that shoppers overwhelmed by choice may simply walk away. Mr Tadelis tested whether it would be better to tailor eBay’s auctions to users’ experience level. The options for new users were narrowed, by removing sellers who are more difficult to assess (for example those who had less-than-perfect feedback on things like shipping times). When new users had a simpler list of sellers to choose from, the number of successful auctions rose and buyers were more likely to use eBay again. Tailoring the market meant gains for buyers, sellers and eBay.

The desire to use theory to challenge conventional thinking is one reason economists are valuable to firms, says Susan Athey, of Stanford University and Microsoft. When Ms Athey arrived at the software giant in 2007 it faced what was seen as an unavoidable trade-off: online advertising was good for revenues, but too much would deter users. If advertisers gained, users would lose. But economic theory challenges this, showing that if firms are dealing with two groups (advertisers and users, say), making one better off often benefits the other too.

Ms Athey and Microsoft’s computer scientists put that theory to work. One idea was to toughen the algorithm that determines whether an ad is shown. This means ads are displayed fewer times, so advertisers lose out in the short-term. But in the longer run, other forces come into play. More relevant ads improve the user experience, so user numbers rise. And better-targeted ads mean more users click on the advert, even if it is shown less often. Empirical evidence showed that although advertisers would respond only after some time, the eventual gain was worth the wait. Microsoft made the change.

Microeconomists have their sights on problems outside their home turf too. At the moment the policies picked by central banks and finance ministries are based on old news, since things like GDP, inflation and unemployment are measured with long lags. A team at Google headed by its chief economist, Hal Varian, is using search-engine data to provide more timely measures. Search terms like “job”, “benefits” and “solitaire” are closely correlated with unemployment claims (see chart). These types of relationship help construct new indexes that offer a real-time picture of the economy. If policymakers start to use these in a systematic way, their decisions could be based on how the economy looked yesterday, rather than months ago.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.