Investigating Structural Change in Predictive Regressions with Applications to Forecasting Stock Returns

A vast body of empirical research involving econometric methods has been undertaken investigating stock return predictability. These methods have largely been based on predictive regression models and have investigated a wide array of financial and macroeconomic variables as putative predictors for returns. Popular variables investigated for their predictive ability for returns have included valuation ratios such as the dividend-price ratio, earnings-price ratio, book-to-market ratio and various interest rate spreads. Overall, these empirical studies have tended to find either no or only relatively weak statistical evidence of in-sample predictability in stock returns. Yet for many variables there are good theoretical reasons to expect them to provide some form of predictive power for returns. For example, a finding that the price-dividend ratio has some predictive power for returns is consistent with orthodox financial theory, because if the dividend-price ratio is time-varying then according to the present value model it must forecast either the dividend growth rate or returns to some extent. A more general explanation for predictability is that expected returns vary with expected business conditions for which those variables found to have some predictive ability are good proxies.

The overwhelming majority of empirical studies of stock return predictability have assumed that the coefficients in the prediction regression models do not change over the available sample data. However, market efficiency arguments suggest that if stock returns are predictable then it is likely to be a temporary rather than a permanent phenomenon. More specifically, if exploiting the predictive power of a regression model can be used to generate abnormal trading profits, then in an efficient market the model will be exploited by large numbers of investors thereby causing the predictive power of the relevant predictor to be eliminated. If a variable begins to have predictive power for stock returns, then a short window of predictability might exist before investors learn about the new relationship between that variable and returns, but market efficiency implies that it will eventually disappear. In such cases, standard predictive regression models based on the full sample of available data will have almost no power to detect these short predictive regimes.

It therefore seems reasonable to consider the possibility that the predictive relationship might change over time, so that over a long span of data one may observe windows of time during which predictability occurs and others where it does not. This project will develop the rigorous econometric methods that are needed to: (i) be capable of effectively detecting the presence of structural change in the coefficients of predictive regression models when applied to the full sample of data, and (ii) use methods of sub-sample data analysis designed to be effective in uncovering such windows of predictability and to enable real-time monitoring for early detection of these windows of predictability as they occur. The project will additionally investigate the role and impact of the forecasting horizon (either short, eg quarterly or monthly, or longer) being considered when using these methods.

The theoretical developments will be conducted using large sample econometric theory and will involve state-of-the-art bootstrap methods. The practical relevance of the theoretical results will be explored using simulation experiments. Clear guidance will also be given to empirical researchers through worked examples and applications of the techniques to key international equity data sets. The empirical aspects of the project will investigate both traditional predictors (such as those discussed above) as well as more recently considered so-called technical analysis indicators (where only price or volume data is used to predict returns).

Practitioners from a range of sectors will benefit: those in research divisions of central banks, government treasuries and US federal reserve banks. Researchers in private sector economic and financial consultancy firms, particularly those providing investment portfolio advice to investors and groups of investors, will also benefit from being able to apply the new techniques developed in the project to the prediction of stock returns and other equity data.

The expected impact to these researchers will take the form of the provision of new theoretically rigorous econometric techniques designed to allow for predictability testing in unstable environments. Specifically, the project will deliver two main features that will substantially increase the scope and reliability of the regression-based methods for detecting predictability in practice:

(i) Tests will be developed to detect, for a given set of sample data and putative predictors, the presence of structural instability in the coefficients of estimated predictive regression models;

(ii) Sub-sample modelling and testing procedures will be developed, for a given set of data and putative predictors, designed to uncover any windows of predictability that may exist in the data and to also enable real-time monitoring for early detection of such windows of predictability as they occur. The latter will provide the practitioner with an early indication of the presence of a valid prediction window.

Principally, the proposed methods are aimed at enabling practitioners to investigate whether and from what predictor variables any episodes of predictability are present in data such as stock returns. Scenarios where predictability is episodic within the available sample could explain both the lack of strong empirical evidence for predictability and the documented weak out-of-sample predictive power of the typical predictor candidates, and allowing for this richer class of time-varying models will provide practitioners with the necessary techniques to uncover such relationships.

To ensure longer-term impact of the research, the PI will discuss with established econometric software manufacturers the inclusion of the procedures developed in the project in their software programs. Producing better econometric software will not only benefit manufacturers but also practitioners that use this software.