HAL, How Are You Feeling Today?

Interpretation of data and rational decision-making are the implicit assumptions underlying many performance improvement philosophies, methodologies and tools. In many cases these assumptions are largely met. Yet in many cases a thoughtful and systematic consideration of facts, risks, options and consequences is not performed or if conducted is often interpreted through the lenses of ego, fear, and sometimes old-fashioned laziness, habit or fatigue. From computer-assisted flying and driving we are now adding computer-assisted (even computer-conducted) surgery, medical diagnosis, and financial management.

Michael Nairne of Tacita Capital wrote:

In a host of endeavours, quantitative models are surpassing human experts in decision accuracy. Software programs that predict the location and frequency of criminal activity have repeatedly beaten the forecasts of experienced police analysts. Models that assess credit worthiness have proved so superior to human judgment that they are now used across the credit card and mortgage industries. Several studies have found that certain statistical models do a better job of predicting academic performance than admissions officers.

The superior accuracy of quantitative models is extending into realms undreamt of a few years ago, including medicine and law. Models have proven more accurate than doctors in forecasting survival rates for patients with coronary heart disease. A statistical model has even proven more accurate than legal experts in forecasting the decisions of the U.S. Supreme Court.

The ability of models to outstrip experts derives from their inherent advantages. Good models are constructed through rigorous statistical testing using appropriate data samples that ensure their findings are valid at a defined level of confidence. Experts, on the other hand, use subjective judgment based on diverse criteria and, hence, are prone to a higher error rate.

Overall, models are consistent in their application; experts are influenced by emotion and extraneous variables. It shouldn’t be a surprise then that a good investment model outshines the typical investor.

The point of this blog isn’t so much whether or not computer-driven investment models are better but rather to reinforce that much of the challenge of performance improvement professionals is to convince senior decision makers to support and participate in analytic and decision processes that are designed to maximize individual and group wisdom while attempting to mitigate the biases, group-think and other distortions humans introduce into the interpretation of data.

In this author’s experience even when it is in the best interests of executives to pay heed to the analytic models, ego is sometimes a barrier and the conclusions of the analysis, and the team that generated it, is water-down or rejected. It is a bit like a pilot over-riding a plane’s autopilot during a particularly tough landing (storms conditions with limited visibility etc.) because he or she was annoyed at a computer usurping their authority.

But there are also examples where over-reliance on technology, such as software programs, can lead to problems. For example, the U.S. National Transportation Safety Board report on a crash landing at San Francisco’s airport by Asiana Airlines stated:

The Asiana Airlines captain who crashed a Boeing 777 at San Francisco International Airport in July told investigators he was “very concerned” about attempting a visual approach because the runway’s automatic landing aids were out of service due to construction, according to an investigative report released Wednesday.

Lee Kang Kuk, a 46-year-old pilot who was landing the big jet for his first time at San Francisco, “stated it was very difficult to perform a visual approach with a heavy airplane.” The jet crash landed after approaching low and slow in an accident that left three dead and more than 200 injured, according to the National Transportation Safety Board.

A visual approach involves lining the jet up for landing by looking through the windshield, as well as using numerous automated cues.

Though Lee was an experienced pilot with the Korea-based airline, he was a trainee in the Boeing 777. NTSB investigator Bill English said Lee had less than 45 hours experience in the Boeing 777 and he last piloted a jet into San Francisco in 2004.

Lee told investigators that he realized others had been safely landing at San Francisco without the glide slope indicator, an array of antennas that transmits a signal into the cockpit, helping ensure the plane is landing correctly.

That system was out of service while the runway was expanded, and has since been restarted.

Lee was nervous about attempting to land using “stick and rudder” flying skills. Pilots spend more time managing computer systems than manually flying planes, systems that are more precise and use less fuel than a human pilot.

The sophistication of tools for the acquisition, analysis, and interpretation of data will continue to increase. As they do so it is important that people know when to listen to the technology and when to override it. A key question is whether the mindsets, habits, and capabilities of senior leaders will keep pace.