A Paradigm Shift in Performance Measurement

Abstract

 The authors offer a two step performance procedure that purports to identify mutual funds that offer downside protection while providing upside potential.   They propose measuring return in terms of upside potential instead of the average return, and measuring downside risk instead of standard deviation.  They recommend replacing the information ratio with a risk-adjusted return called the omega excess return.

Frank A. Sortino and Bernardo Kuan

Bernardo Kuan received his MBA from San Francisco State University and is Director of Operations at DAL Investments, San Francisco.

 Introduction

The thesis of this paper is that popular performance measures, like the Sharpe ratio and information ratio, are not designed for your clients needs. The, "one size fits all" approach of these ratios does not recognize the fact that your clients have different ages, different amounts of wealth, and different goals. We propose a new two step procedure for performance measurement that is designed to help the individual investor accomplish his or her goals.

We take the position of aircraft designers talking to pilots about a radically new plane that has just passed its test flights. Pilots don’t have to know how to build an airplane in order to fly one; and passengers don’t have to know how to fly a plane in order to use them for transportation. Therefore, we will avoid tedious formulas and focus on what you need to know in order to get your clients to their destination more effectively with greater safety. We will first answer the question, "does it fly any better"?

The Results

Most new investment concepts are born out of catastrophe. That is, a catastrophe happens in the financial markets and researchers sift through the ashes of the ruins to find out what could have been done to avoid the catastrophe. Pensions & Investments magazine (P&I) provided a unique opportunity to test a new strategy just before the NASDAQ melt down of 2000.

In the first quarter of 2000, a new performance measure was introduced in P&I called the Upside Potential ratio. This U-P ratio was used to rank the 100 mutual funds that received the most new money in the previous year. The second step selected funds from the top quartile of the ranking that had a positive Omega excess (a downside risk-adjusted return). The combination of these two steps we will refer to as the U-P strategy.

The U-P strategy will be compared with a naive strategy that simply picks the fund that had the highest return in the previous year. We will refer to this naive strategy as the high return strategy. The comparison of the U-P strategy with the high return strategy will help answer the question, "who needs a financial planner to tell me what to invest in? I’ll just pick the one who’s winning the race."

Figure 1 shows how the top three funds in the U-P strategy did in the first market sell-off of 2000, relative to the three funds that had the highest return in the previous year.

From peak to trough, the three top funds in the U-P strategy were up an average of 13% in a market that saw a 32% decline in the NASDAQ. All three of the funds in the high return strategy were down substantially, and on average, were off more than the NASDAQ.

Figure 2, shows the relative performance of these same funds in the second decline from September 1^st to November 30^th, which witnessed a 38% decline in the NASDAQ. While the three funds in the high return strategy were off an average of 44%, the U-P strategy funds were up an average of 4/10^ths of one percent.

Figure 2

Style Rankings

The rankings that produced Figures 1 and 2 showed a bias toward value funds. So, in the third quarter report for P&I the format was changed in order to compare performance within style categories (large growth, large value and small cap stocks) In Figure 3, the top ranked fund in each style category is compared to the bottom ranked fund in each style category.

The top ranked U-P strategy funds did better in all three style categories, and on average, did approximately 4 times better than the bottom ranked funds. In this instance, the U-P strategy was able to identify the bad as well as the good performers.

Figure 3

Well, the U-P strategy flew better than anything else we know of in the financial storm of 2000, but how would it have done relative to the average fund in all kinds of weather? Since most of the funds in the P&I list are large growth funds, we back tested the U-P strategy relative to the average large cap growth fund for the past 20 years. The first three years were used to identify the fund’s style. Figure 4 shows how the top U-P strategy fund performed relative to the average fund in this category.

The cumulative result shown in Figure 4 shows the U-P strategy had a compound growth rate of 18% versus approxamately 16% for the average fund. Figure 5 shows that this was accomplished with less downside risk. The U-P strategy produced fewer negative returns than the average fund and, half the time the average fund was down, the U-P strategy fund was up.

                                     U-P Strategy Took Less Downside Risk

Why does it fly better?

The first reason is, it is conceptually better for solving your client’s investment problem than the Capital Asset Pricing Model (CAPM) approach. The second reason has to do with superior estimation procedure.

Sally Atwater [2001] points out that financial planners focus on the client’s goal, while pension consultants focus on who will manage the money. This leads pension consultants to measure performance in terms of beating the market. However, financial planners should measure performance in terms of accomplishing the client’s goal. For each of your clients, with sufficient time and money to accomplish his or her goals, there is some rate of return that must be earned at minimum in order to accomplish the client’s goal. This is called the minimal acceptable rate of return (MAR). Therefore, any performance measure that does not specifically consider the client’s MAR is not measuring performance relative to your client’s goal.

The Sharpe ratio subtracts the risk free rate from the manager’s return and divides by the standard deviation of the manager’s returns. The Information ratio subtracts the return on the market index (e.g., the S&P 500) from the manager’s return and divides by the tracking error (the standard deviation of the term in the numerator). The MAR is ignored in both of these measures. These performance measures are designed to identify managers who beat the market. It will be shown that the MAR is in both the numerator and denominator of the U-P ratio and is incorporated in the Omega excess return.

Because the Sharpe ratio and information ratio are estimated by calculating the average return they ignore some important information about how often and how far the manager might exceed the MAR. Table 1 provides a simple example to show the important difference between upside potential and the average, or mean, return.

Table 1

Fund 1 and Fund 2 have the same average return (9.6%). Which one has demonstrated the most upside potential? Fund 1 exceeded the MAR of 8% 70% of the time while fund 2 only exceeded the MAR 60% of the time. But frequency alone ignores how far above the MAR each manager got. Fund 1 never exceeded the MAR by more than 3 percentage points, while fund 2 exceeded the MAR by 6 and 7 percentage points. A measure of upside potential should incorporate both frequency and magnitude. The way to accomplish this is to weight each value above the mar by its probability of occurrence. In this example, each return (above and below the MAR) is assumed to have the same probability of occurrence. Therefore, the total of excess returns above the MAR is divided by 10, not 7 for fund 1 and 6 for fund 2.

The estimation procedure shown in Table 1 suffers from the same problem as the Sharpe ratio and Information ratio. That is, they only look at what did happen in the portfolio, not what could have happened. A much better estimation procedure called the bootstrap is described on www.sortino.com.

The Omega excess Return

The second step involves calculating the Omega excess return, which is a way to determine whether a manager outperformed a passive set of indexes. First, the manager’s style is replicated by a set of passive indexes called a style benchmark. Then the downside risk of the manager’s style benchmark is subtracted from the manager’s return, creating a risk-adjusted return. Similarly, a risk-adjusted return is calculated for the style benchmark. The difference between the two risk-adjusted returns is called the Omega excess return.

Intuition or logic

Upside potential is not a familiar concept like the probability of success, and therefore may not be as intuitively appealing. For that reason, some investors might be inclined to choose Fund 1 because it exceeded their MAR more often than Fund 2. However, to do so would be an error in logic. The relatively new field of behavioral finance documents the errors in judgment investors make over and over again. Shefrin [1999] provides a good reference book on this subject. It should come as no surprise that the logic of statistics is not intuitively obvious to most humans.

The late Amos Taversky, the father of behavioral finance, found that investors tended to have an aversion to making high returns and tended to ignore the risks of huge losses. This prescription for disaster is called prospect theory, and flies in the face of the old adage, ride your gains and cut short your losses. One example of this behavior would be to sell covered calls on technology stocks after a modest run up, thus giving away the prospect for very high returns when the stock is called away. Another example would be to buy tech stocks at the top in early 2000 and ride them all the way down in 2001. Our task is to find ways to help investors avoid systematic errors in judgment.

A possible means to that end is the U-P strategy. The U-P ratio is simply the upside potential divided by the deviations of the returns below the MAR, or, the downside risk.

The top ranked funds from the 4^th quarter P&I analysis is shown in Table 2 to illustrate how this information can be used to assist your clients toward the accomplishment of their goal.

                                        Table 2

Wells Fargo diversified Equity fund had a U-P ratio of 1.78, meaning, that fund’s style had 78% more upside potential than downside risk. The R-squared indicates that a combination of passive indexes accounted for 98% of the returns of the mutual fund.

The Omega excess indicates the Wells Fargo fund demonstrated an ability to beat the passive indexes by an average of 2.5% per year, after adjusting for the risk of failing to average 8%. However, AMCAP fund had a very similar U-P ratio but had a downside risk-adjusted return that was 4 times better than Wells Fargo. For that reason, AMCAP fund was recommended over the Wells Fargo fund. Notice American Century Income and Growth had a negative Omega excess return. In other words, an investor would have made 1.7 % more on a risk-adjusted basis by investing in a set of passive indexes that replicated this funds style.

Conclusion

The Sharpe ratio and Information ratio are derived from the Capital Asset Pricing Model (CAPM). The CAPM explains how all assets should be priced in equilibrium, so that, on a risk-adjusted basis, all returns are equal. This implies that everyone has the same goal, beat the market. The CAPM is designed to solve the investment problem for all investors, simultaneously. It is not designed to solve the investment problem of an individual investor, like your client. Each of your clients has some rate of return that must be earned at minimum in order to accomplish their goal. If that minimal acceptable return (MAR) is not in the equation, it cannot be measuring performance relative to your client’s goal.

One limitation to a wide use of the Upside potential ratio is the time and skill required to write a computer program to make these calculations. To facilitate the use of the U-P ratio, The Pension Research Institute will make free software available to the investment community in a book titled "Managing Downside Risk In Financial Markets" (See New Book on Home Page.

References

Shefrin, Hersh. "Beyond Greed and Fear," Harvard Business School Press, 1999.

Sortino, Frank and Satchell, Stephen. "Managing Downside Risk in Financial Markets", Ch 3, Butterworth Heinemann, September, 2001.