Managing Uncertainty with the Bootstrap Procedure

Revised 11/3/99

Managing Uncertainty With The Bootstrap Procedure

Frank Sortino and Hal Forsey

Overview

Almost all quantitative models assume that what did happen in the past, best predicts what will happen in the future. The most popular statistical forecasting tool is linear regression, which assumes that the past is linked to the future along a straight line fitted to a series of past outcomes. We think there is a link between the past, the present, and the future, in that financial markets vascilate around a fairly valued state; sometimes becoming over valued, other times, under valued. Each market valuation state has a stable distribution that is estimatable. Furthermore, knowing the current valuation state of the market allows one to manage future financial uncertainty with greater skill.

It is amazing how much time and effort is spent analyzing a past event like, the crash of '87, or the "Asian Crisis" of 1997, as if that event is likely to happen again. This is a testament to our inability to see events as draws from a wide range of possible outcomes. To recognize that uncertainty encompases past events as well as future events (see Exhibit 1).

Exhibit 1

Uncertainty_1.gif (8939 bytes)

Looking back from the blue line labeled "Now", it is clear that manager C never fell below the MAR. But all we see is what did happen, not what could have happened. At each point in time there was a whole distribution of returns that could have occured, as shown by the distribution with the green star. We need a procedure for bringing up this underlying distribution by its bootstraps so we can get an insight as to what could happen in the future.

Unfortunately, the statistical technique most often used is linear regression, which looks only at what did happen. Regression implies the link between the past and the future is a straight line drawn through a series of past events, as shown in Exhibit 2. Linear regression is a notoriously poor way to predict the future.

Exhibit 2

While linear regression does produce an ex ante estimate of the mean and standard deviation, these moments may not accurately describe the underlying distribution, i.e., that it is a standard normal distribution. The merit to this statistical tool is that it is readily available and easy to use. If there is a link between the past and the future, it is probably not this simple.

Exhibit 2 shows how the Bootstrap Procedure allows us to see what could have happened from what did happen. A portfolio manager with this series of returns could claim that the worst year they ever had was their first year, when, in spite of three negative months, they still earned 30.56% compounded. But is that the worst year one could expect in the future? One of the possible draws from the bootstrap procedure shown above indicates that the manager could lose 35.3%, based on this same set of data. This procedure assumes that the 48 observations constitute a representative sample,drawn from the true underlying distribution. Each return that occurred was just one of many that could have occurred.

To estimate what could have happened from what did happen, we randomly draw one return from the sample and replace it. We then draw a second return and replace it. This is repeated until we have one annual return that could have occurred. The entire process is repeated to generate a distribution of thousands of returns that could have occurred, as shown in exhibits 3 and 4. To accomplish this, we used 27 years of monthly observations, not just 4 years. The bootstrap procedure indicates the return could have been much worse than history indicated.

Exhibit 2

Exhibit 2 shows how one such annual return could have been drawn. The first return of 4% is drawn and replaced, so that it could be drawn again. Notice, the second and 12th returns are the same historic observation of -6%. This is consistent with the random walk hypothesis. Even though patterns seem to appear in the data, e.g., a negative return is followed by another negative return, the bootstrap procedure ensures that the returns are independent. For an example of how this procedure identified a potential loss in the Japanese market that went up every year for ten years, see the Journal of Portfolio Management article on this web site.

Research shows people are not good at predicting future returns [Martin 1985] and simple extrapolation of historical returns has been a poor predictor of future returns [Michaud 1998]. Since many consultants make judgements about whether or not the market is currently fairly valued, they might find it easier to use the shape of uncertainty current valuations imply, than forecast the future. Managing uncertainty is difficult at best. But how can one manage something one cannot describe? The alternative, as Peter Bernstein so aptly discloses [1996] is to rely on fate or intervention from God.

Hsu [1984] has shown that historic distributions have been very unstable over time, e.g., the standard deviation of returns for five year intervals ranged from less than 10% to more than 25%. Therefore, the distribution of returns that did happen in some previous interval of time, will likely prove to be a poor predictor for the next interval. Hsu assumed the instability was due to some economic phenomenon. We agree, but rather than focus on the economic cause, we focus on the results. While markets tend toward equilibrium, they sometimes become overvalued or undervalued. The economic cause of this over (under) valuation may change over time but the result will be the same; and that result is a probability distribution. The shape of the distribution of a particular asset will change dramatically as the market moves from an undervalued level to an overvalued level, but its distribution is stable for a given state. Therefore, an estimate of the distribution for any asset based on when the market was overvalued in the past, will also be valid for that asset when the market is overvalued in the future.

Exhibit 3 identifies three different market valuation levels for the ten year interval beginning August, 1987. Is the shape of uncertainty the same at relatively high points in the market as it is at relatively low points? If the distributions for stocks, bonds, and cash, never change, then the allocation among them should remain constant. Alternatively, if the potential for loss in the stock market is greater now than it was in November of 1987, then the shape of the underlying distribution must be different than it was. Also, the location point (mean) of that distribution must be different. To the extent one can estimate the shape and location point of the underlying distributions, one can profit from active asset allocation. Style-based scenario identification is an attempt to change the shape and location point of style indexes in a way that is statistically sound and intuitively appealing. The sequence of questions we attempt to answer is: where am I now (valuation wise), what does the uncertainty of investing in various assets look like from here, what asset allocation decisions are consistent with these beliefs?

Exhibit 3

We begin by using 20/20 hindsight to identify the periods in the past when the market was over, under, or fairly valued (see Exhibit 4). We used a discounted cash flow model developed by KS Global Strategies Inc. to identify periods in the past when the S&P 500 was overvalued or undervalued by 10% or more (see Exhibit 3). While the discount function used by KS may result in different market valuations for some months than other models, the methodology is directly applicable to any valuation process. It matters not if this valuation model is able to predict the future. What matters is whether it was able to identify past periods when the market was under or over valued. Almost all valuation models are able to prduce evidence to support that claim. Exhibit 4 indicates that the market was overvalued 32% of the time and undervalued 24% of the time. Then the bootstrap procedure developed by Bradley Effron [1993] is used to identify what could have happened to various styles in those valuation periods. Thousands of annual returns are generated that could have occurred for each market valuation. The implicit assumption is that the cause of the stock market becoming incorrectly valued is not always the same, but the distribution of returns in overvalued or undervalued markets may be similar enough to produce better estimates of risk and return and thus improve asset allocation decisions.

The combination of styles that will do best in an overvalued market is probably not the combination that will do best in an undervalued market. The task then, is to allow the consultant to affect the return distributions of the indexes, based on the consultant’s opinion of the degree to which the market is currently misvalued. In a previous article [1997] we generated distributions for various styles given no ability to determine the level of market valuation. In this article we examine how various levels of market valuation might alter the shape of the distribution, and therefore, the asset allocation.

Exhibit 4

If the consultant making the asset allocation decision believes the market is currently fairly valued, all of the data is used to produce the distribution of returns, e.g., the distribution for large cap stocks would be as shown in Exhibit 5. The implicit forecast is that the market could remain fairly valued, become overvalued, or become undervalued. The distribution is generated by bootstrapping all the monthly returns from Dimensional Fund Advisors from 1/64 through 12/95.

Exhibit 5

What if the consultant believes the market is so overvalued that the returns associated with overvalued markets are not feasible. The distribution for large cap stocks that is consistent with that belief is shown in Exhibit 6. All of the returns above the 10% overvalued line were eliminated in the bootstrap procedure for Exhibit 6. The downside risk associated with large cap stocks would increase 86% and the expected return would drop from approximately 11% to a little over 1%. The same methodology can be used to construct a probability distribution for undervalued markets. In which case, the mean becomes 21% and the tenth percentile shifts up to 1.7%.

Exhibit 6

Hsu, D.A. "The Behavior of Stock Returns: Is it Stationary or Evolutionary?" Journal of Financial and Quantitative Analysis, March 1984.

Martin, Linda J. "Uncertain? How do you Spell Relief?", Journal of Portfolio Management, Spring 1985.

Sharpe, William F. "Asset Allocation: Management Style and Performance Measurement," March 1992.

Sortino, Frank A., and Hal J. Forsey. "On the Use and Misuse of Downside Risk." Journal of Portfolio Management, 1996.

Sortino, Frank A., Gary A. Miller, and Hal J. Forsey. "Estimating Risk-adjusted Performance: a Style-based Analysis", Journal of Investing, Summer 1997.

Endnotes