NOVEL APPROACHES TO COMPARING THE PREDICTIVE ACCURACY OF NESTED MODELS IN DATA RICH AND HETEROGENEOUS PREDICTOR ENVIRONMENTS
Lead Research Organisation:
University of Southampton
Department Name: Sch of Economic, Social & Political Sci
Abstract
Comparing the out of sample predictive accuracy of competing statistical models is an essential component of data science and a key metric for choosing a suitable specification for the purpose of generating forecasts or discriminating between competing hypotheses. Unlike the explanatory power of such models which is commonly evaluated via in-sample goodness of fit measures and specification tests, predictive accuracy and predictive modelling are instead concerned with how well models can cope with unseen data and produce accurate forecasts of some outcome of interest.
The purpose of this project is to develop a novel toolkit for comparing the relative accuracy of time series forecasts produced by two or more nested predictive regression models with the end-goal of detecting key drivers of predictability or the lack of it. We consider an environment where one is confronted with not only a potentially large pool of predictors but also with these predictors allowed to display a mixture of dynamic characteristics, some (or all) being highly persistent and others noisier as it commonly occurs in economic and financial data. A macroeconomist interested in forecasts of GDP growth for instance faces hundreds of potentially useful predictors ranging from noisy indicators with very little memory such as financial returns to more persistent series with much longer memory or trending behaviours such as interest rates. Bundling such predictors together in a predictive accuracy contest or ignoring the persistence properties of the data all-together is likely to affect the reliability of inferences regardless of whether there are few or many such predictors. Despite the relevance and omnipresence of such scenarios in applied work the predictive accuracy testing literature has devoted little attention to such considerations. The novel aspects of this research concern both the specific criteria introduced for implementing predictive accuracy comparisons which will considerably simplify and generalise existing approaches and the richer environment under which they can be applied.
Furthermore and in the course of empirical research or policy analysis, researchers are often faced with the need to compare the forecasting ability of a simple model with a more complicated one, with the simple model being a special case of the more complicated model. Such model pairs are typically referred to as nested while model pairs with no such similarities are referred to as non-nested. Nested models are one of the most commonly encountered setting in empirical research and help answer fundamental questions such as: does the inclusion of a set of additional predictors significantly improve the predictive power of a smaller model or a non-predictability benchmark?
Irrespective of whether one operates in a big data environment combined with heterogeneous predictor types or in a more idealised environment with few well behaved and purely stationary predictors, conducting out of sample predictive accuracy comparisons between nested models raises many technical challenges that have also not been resolved in a satisfactory way despite a voluminous literature on the subject (e.g. the fact that two nested models collapse into the same specification under the hypothesis of equal predictive accuracy typically results in ill-defined test statistics with degenerate variances).
The overarching objective of this proposal is to introduce a totally new technical framework that can accommodate predictive accuracy comparisons between models irrespective of whether they have a nested structure or not. This framework will then be used to develop a toolkit for conducting predictive accuracy tests and predictor screening in data rich environments.
The purpose of this project is to develop a novel toolkit for comparing the relative accuracy of time series forecasts produced by two or more nested predictive regression models with the end-goal of detecting key drivers of predictability or the lack of it. We consider an environment where one is confronted with not only a potentially large pool of predictors but also with these predictors allowed to display a mixture of dynamic characteristics, some (or all) being highly persistent and others noisier as it commonly occurs in economic and financial data. A macroeconomist interested in forecasts of GDP growth for instance faces hundreds of potentially useful predictors ranging from noisy indicators with very little memory such as financial returns to more persistent series with much longer memory or trending behaviours such as interest rates. Bundling such predictors together in a predictive accuracy contest or ignoring the persistence properties of the data all-together is likely to affect the reliability of inferences regardless of whether there are few or many such predictors. Despite the relevance and omnipresence of such scenarios in applied work the predictive accuracy testing literature has devoted little attention to such considerations. The novel aspects of this research concern both the specific criteria introduced for implementing predictive accuracy comparisons which will considerably simplify and generalise existing approaches and the richer environment under which they can be applied.
Furthermore and in the course of empirical research or policy analysis, researchers are often faced with the need to compare the forecasting ability of a simple model with a more complicated one, with the simple model being a special case of the more complicated model. Such model pairs are typically referred to as nested while model pairs with no such similarities are referred to as non-nested. Nested models are one of the most commonly encountered setting in empirical research and help answer fundamental questions such as: does the inclusion of a set of additional predictors significantly improve the predictive power of a smaller model or a non-predictability benchmark?
Irrespective of whether one operates in a big data environment combined with heterogeneous predictor types or in a more idealised environment with few well behaved and purely stationary predictors, conducting out of sample predictive accuracy comparisons between nested models raises many technical challenges that have also not been resolved in a satisfactory way despite a voluminous literature on the subject (e.g. the fact that two nested models collapse into the same specification under the hypothesis of equal predictive accuracy typically results in ill-defined test statistics with degenerate variances).
The overarching objective of this proposal is to introduce a totally new technical framework that can accommodate predictive accuracy comparisons between models irrespective of whether they have a nested structure or not. This framework will then be used to develop a toolkit for conducting predictive accuracy tests and predictor screening in data rich environments.
Publications
Gonzalo J
(2024)
Out-of-sample predictability in predictive regressions with many predictor candidates
in International Journal of Forecasting
Gonzalo J
(2025)
Detecting Sparse Cointegration
Gonzalo J
(2021)
Spurious relationships in high-dimensional systems with strong or mild persistence
in International Journal of Forecasting
Pitarakis J
(2025)
Serial-Dependence and Persistence Robust Inference in Predictive Regressions
Pitarakis J
(2023)
A NOVEL APPROACH TO PREDICTIVE ACCURACY TESTING IN NESTED ENVIRONMENTS
in Econometric Theory
| Description | Understanding Forecasting and Model Selection in Complex Data Environments: Forecasting is a crucial part of decision-making in economics, finance, and policy-making. Whether predicting stock market movements, economic growth, or inflation, accurate forecasts are essential for making informed choices. However, comparing and selecting the best forecasting models is not simple. Traditional statistical tools often fail when working with large, complex datasets or when models contain many possible predictors. This research, funded by the ESRC, has developed new techniques to improve how we assess forecasting models, ensuring that decision-makers use the most reliable methods. The Challenge: Comparing Models and Selecting Predictors One of the biggest challenges in forecasting is determining whether adding more predictors genuinely improves predictions. In practice, forecasters often test multiple models, but conventional comparison methods can be misleading, especially when models are closely related or when predictors behave unpredictably over time. Another challenge comes from the explosion of available data. Today, economists and analysts have access to hundreds or thousands of potential predictors-from macroeconomic indicators to financial ratios and market sentiment measures. Choosing which variables to include in a forecasting model is a difficult problem. Using too many can lead to misleading results, while using too few risks missing valuable insights. Key Achievements of the Research This project has developed a new toolkit for evaluating and selecting forecasting models, addressing the above challenges. 1. New Statistical Tests for Forecast Accuracy The research introduced new methods for comparing forecasting models, ensuring that analysts can reliably assess whether adding new predictors genuinely improves predictions. Unlike previous techniques, these methods remain valid even when predictors display unusual patterns, such as high persistence (where past values strongly influence future values). 2. Techniques for Handling Large Data Sets With modern forecasting models incorporating vast amounts of data, this research developed new tools for selecting the most relevant predictors from large datasets. One major contribution was the creation of a method that identifies which few variables truly matter while ignoring the noise. 3. New Techniques for detecting relevant predictors in regression models under complex settings. 4. Robust Methods for Cointegration Analysis Many economic and financial variables are linked over the long term. This project introduced a novel way to detect these long-run relationships in high-dimensional settings, ensuring that models correctly identify deep connections between economic indicators, with the purpose of improving forecasts. 5. Software Tools for Practitioners The findings have been translated into easy-to-use software in R and MATLAB, allowing researchers, policymakers, and financial analysts to apply these techniques without needing deep statistical expertise. 6. International Knowledge Sharing The project's discoveries have been shared through academic conferences, an international workshop, and an open-access website, ensuring that these methods reach economists, policymakers, and analysts worldwide. The Impact: Better Forecasting, Smarter Decisions By improving the way, we compare forecasting models and select predictors, this research helps ensure that businesses, policymakers, and researchers make more informed, data-driven decisions. |
| Exploitation Route | Taking the Research Forward: Practical Applications and Future Impact The outcomes of this research provide new methods for evaluating forecasting models and selecting relevant predictors, offering direct applications across multiple fields, from economic policy-making to financial markets and data science. The methodologies developed can be taken forward and put to use in several key ways: 1. Adoption by Economists, Policymakers, and Financial Analysts Forecasting is fundamental in economic policy, investment decisions, and risk management. Central banks, financial institutions, and government agencies can apply these new tools to improve economic forecasting, inflation modeling, and financial risk assessments. By ensuring that only the most relevant predictors are included in forecasting models, decision-makers can rely on more accurate and transparent predictions, reducing uncertainty in economic planning. For instance, a central bank could use these techniques to determine whether adding new macroeconomic indicators (such as consumer confidence or trade balances) improves inflation forecasts. Similarly, an investment firm could apply these methods to assess whether new financial indicators enhance stock market predictions. 2. Integration into Forecasting Software and Machine Learning Models The research outcomes have been translated into open-source software, available in R and MATLAB, making them accessible to a broad audience. Econometricians and data scientists can integrate these tools into existing forecasting frameworks, applying them to various problems, from financial forecasting to climate modeling and energy demand prediction. Additionally, the methods can complement machine learning models, helping researchers identify which variables to prioritize in predictive analytics. By combining traditional econometric techniques with machine learning, practitioners can improve model accuracy while maintaining interpretability. 3. Teaching and Training the Next Generation of Economists The methods developed in this research can be incorporated into university curricula and professional training programs in econometrics, statistics, and data science. Courses on forecasting and time-series analysis can use these techniques to help students understand how to evaluate model performance and deal with high-dimensional data. Furthermore, the project's open-access website and workshop materials provide a valuable educational resource, ensuring that researchers worldwide can learn and apply these new methods. 4. Future Research and Cross-Disciplinary Collaboration This research lays the groundwork for further advancements in predictive analytics and model evaluation. Future studies could extend these methods to applications in climate forecasting, healthcare analytics, and AI-driven decision-making. For example, researchers in environmental economics might use these techniques to assess the predictive power of climate indicators for long-term policy planning. Additionally, collaborations with industry partners, government agencies, and financial institutions could refine these techniques for specialized applications, ensuring that they are both theoretically robust and practically useful. Conclusion: A Lasting Impact on Forecasting and Decision-Making By making forecasting more accurate, reliable, and scalable, this research provides tools that can be widely adopted across multiple fields. From improving economic policy to enhancing machine learning models, the methods developed will continue to shape how organizations and researchers approach predictive modeling, ensuring data-driven decision-making in an increasingly complex world. |
| Sectors | Education Energy Financial Services and Management Consultancy Government Democracy and Justice |
| URL | https://sites.google.com/view/jpitarakis-esrc/home?authuser=0 |
| Description | The new tools for forecast evaluation and predictor selection developed in this research have practical applications in finance, policy-making, and technology, helping businesses, financial institutions, and government agencies make more informed, data-driven decisions. The research outcomes have been translated into open-source software tools in R and MATLAB, ensuring accessibility beyond academia. By lowering technical barriers, these tools empower economists, data scientists, and financial analysts to apply state-of-the-art econometric methods without requiring deep expertise in statistical theory. While the project is still in its early stages of dissemination, its key outputs have already begun gaining recognition and citations from academic researchers. Notably, papers such as New Tests of Equal Forecast Accuracy for Factor-Augmented Regressions with Weaker Loadings (Margaritella & Stauskas, 2024) and Dissecting the Sentiment-Driven Green Sector Premium in China with a Large Language Model (Chen et al., 2024) have referenced and built upon these methodologies. As awareness grows and more practitioners integrate these tools into real-world applications, the research is poised to significantly influence forecasting and model selection practices. By improving the accuracy and reliability of predictive models, these advancements have the potential to reshape decision-making in economic policy, investment strategies, and risk assessment across multiple industries. |
| First Year Of Impact | 2024 |
| Sector | Education |
| Impact Types | Economic |
| Title | A novel and highly robust approach to detecting significant covariates in dynamic regression models |
| Description | The research tools developed in this agenda are designed to detect predictability and, more generally, to identify relevant covariates in non-standard regression models. These models are considered "non-standard" because they may include covariates with heterogeneous dynamic structures (e.g., varying degrees of persistence) alongside complications such as serial correlation and heteroskedasticity. The unique contribution of this agenda is the introduction of a novel method that can simultaneously accommodate all of these complexities. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2025 |
| Provided To Others? | Yes |
| Impact | Matlab package made available to the public and potential users via a dedicated github site. |
| URL | https://arxiv.org/abs/2502.00475 |
| Title | A novel approach for detecting equilibrium relationships in high dimensional settings |
| Description | With the proliferation of large data sets, high-dimensional methods for statistical estimation and inference have become increasingly prominent in research. However, many of these approaches-such as regularization and sparsity-based estimation-are grounded in restrictive assumptions that limit the degree of dependence in the data and otherwise seldom hold for economic time series. The research tools introduced in this agenda help bridge that gap by extending high-dimensional methods to more commonly encountered yet less conventional settings. In particular, they address scenarios in which we need to determine whether highly persistent covariates (e.g., those behaving like random walks) are relevant or irrelevant in explaining a given phenomenon. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2025 |
| Provided To Others? | Yes |
| Impact | Software package for implementing the methods developed in this agenda have been made available in a public github repository |
| URL | https://doi.org/10.48550/arXiv.2501.13839 |
| Title | A novel implementation of the forecast encompassing principle for model selection purposes |
| Description | The research tools developed in this agenda are geared towards making model comparisons based on their forecasting ability. The main originality of this agenda is in its novel way of implementing the forecast encompassing principle in a way that facilitates making predictive accuracy comparisons across nested models, a context plagued with difficulties and not easily amenable to making such comparisons. Equally importantly, the method I propose and whose properties I derive is shown to be robust to the stochastic properties of the predictors under consideration, allowing them to be highly persistent or much noisier (or both). This is particularly important when dealing with econometric models that use economic and financial time series with quite heterogeneous dynamic properties. Some of these series are highly persistent with an almost trending like behaviour (e.g., interest rates, stock market valuation ratios, inflation) while others are much noisier (e.g., any proxy for economic growth, stock returns etc.). Developing methods that are robust to the stochastic properties of the predictors under consideration is therefore of paramount importance for practitioners. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2023 |
| Provided To Others? | Yes |
| Impact | R package made publicly available on the CRAN repository; Matlab package made publicly available on Github (see URL below) |
| URL | https://sites.google.com/view/jpitarakis-esrc?authuser=0 |
| Title | New methods for detecting the predictability of an outcome when the pool of potential predictors can be very large |
| Description | In empirical work one is often confronted with the need to assess the ability of a large pool of predictors (e.g., economic and financial variables) to predict an outcome of interest (e.g., inflation, interest rates). How to do that in an environment where there may be hundreds of predictor candidates each having distinct time-series dynamics (e.g., some being highly persistent and others much noisier)? In this research I develop a simple methodology for doing that. The test I introduce can accommodate environments where the number of predictors exceeds the available sample size. Equally importantly the proposed method is shown to be robust to the stochastic properties of predictors. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2023 |
| Provided To Others? | Yes |
| Impact | The methods developed in this agenda have been disseminated internationally at prestigious international events and institutions (see URL below). In order to incentivize their use by a wider audience I am currently coding dedicated Matlab and R packages which will be made available in public repositories. |
| URL | https://www.sciencedirect.com/science/article/pii/S0169207023001048 |
| Title | Novel techniques for comparing the predictive accuracy of competing econometric models with a nested structure |
| Description | A key component of this grant's research agenda is the development of new techniques for comparing the predictive accuracy of competing econometric models. A long standing and unresolved problem in this literature has to do with our ability to conduct formal tests of predictive accuracy when the models under consideration are nested. Suppose for instance that a particular model aims to explain an outcome of interest with a certain number of predictors. We wish to assess whether a larger model that contains the same predictors augmented by an additional set improves or deteriorates the model's ability to generate out of sample predictions (i.e., using new data). In the now published paper titled "A novel approach to predictive accuracy testing in nested environments" (In Press at ECONOMETRIC THEORY) I introduce novel test statistics designed to implement such inferences. These test statistics do not suffer from the drawbacks and limitations of existing methods. Their distribution is derived formally, and implementation guidelines provided for practitioners (including software applications written in matlab and R). Importantly, the methods I developed are shown to be robust to the stochastic properties of the predictors under consideration, allowing them to be highly persistent or much noisier (or both). This is particularly important when dealing with econometric models that use economic and financial time series with quite heterogeneous dynamic properties. Some of these series are highly persistent with an almost trending like behaviour (e.g., interest rates, stock market valuation ratios, inflation) while others are much noisier (e.g., any proxy for economic growth, stock returns etc.). Developing methods that are robust to the stochastic properties of the predictors under consideration is therefore of paramount importance for practitioners. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Matlab and R programmes implementing the proposed techniques have been made publicly available. |
| URL | https://sites.google.com/view/jpitarakis-esrc?authuser=0 |
| Title | Detecting long run equilibrium relationships linking a small number of variables when the candidate pool is very large |
| Description | Economic and financial time series are often linked through long run relationships the knowledge of which is important for forecasting purposes. A novel method designed to uncover such relationships in big data environments with complex dynamic properties has been developed using state of the art regularization methods. |
| Type Of Material | Data analysis technique |
| Year Produced | 2025 |
| Provided To Others? | Yes |
| Impact | As this component of the methodological developments has only been recently circulated, it is too early to describe potential impact. |
| URL | https://arxiv.org/abs/2501.13839 |
| Title | Detecting the presence of predictability when there is a large pool of predictor candidates |
| Description | In this research I developed new test statistics designed to detect the presence of out-of-sample predictability when predictability can be induced by a very large number of predictor candidates (possibly larger than the available sample size). In empirical work one is often confronted with the need to assess the ability of a large pool of predictors (e.g., economic and financial variables) to predict an outcome of interest (e.g., inflation, interest rates). How to do that in an environment where there may be hundreds of predictor candidates each having distinct time-series dynamics (e.g., some being highly persistent and others much noisier)? In this research I develop a simple methodology for doing that. The tests I introduce can accommodate environments where the number of predictors exceeds the available sample size. Equally importantly my proposed methods are shown to be robust to the stochastic properties of predictors. This is particularly important when dealing with econometric models that use economic and financial time series with quite heterogeneous dynamic properties. Some of these series are highly persistent with an almost trending like behaviour (e.g., interest rates, stock market valuation ratios, inflation) while others are much noisier (e.g., any proxy for economic growth, stock returns etc.). Developing methods that are robust to the stochastic properties of the predictors under consideration is therefore of paramount importance for practitioners. |
| Type Of Material | Data analysis technique |
| Year Produced | 2023 |
| Provided To Others? | Yes |
| Impact | This research is still in it early stages of dissemination as the paper has been accepted for publication at the International Journal of Forecasting but is still in Press. |
| URL | https://sites.google.com/view/jpitarakis-esrc/home?authuser=0 |
| Title | Inferring the relevance of covariates for prediction purposes in complex data settings |
| Description | A new method for testing the statistical significance of estimated parameters in predictive regressions. The approach features a new family of test statistics that are robust to the degree of persistence of the predictors. Importantly, the method accounts for serial correlation and conditional heteroskedasticity without requiring any corrections or adjustments. This is achieved through a mechanism embedded within the test statistics that effectively decouples serial dependence present in the data. |
| Type Of Material | Data analysis technique |
| Year Produced | 2025 |
| Provided To Others? | Yes |
| Impact | The research develops an entirely new way of detecting the relevance of covariates in complex and "messy" data environments. |
| URL | https://arxiv.org/abs/2502.00475 |
| Title | Methods for comparing the predictive accuracy of competing econometric models using novel test statistics that are specifically designed to handle nested model comparisons. |
| Description | A key component of the grant's research agenda is the development of new techniques for comparing the predictive accuracy of competing econometric models. A long standing and unresolved problem in this literature has to do with our ability to conduct formal tests of predictive accuracy when the models under consideration are nested. Suppose for instance that a particular model aims to explain an outcome of interest with a certain number of predictors. We wish to assess whether a larger model that contains the same predictors augmented by an additional set improves or deteriorates the model's ability to generate out of sample predictions (i.e., using new data). How can we formally test such hypotheses? In the paper titled "A novel approach to predictive accuracy testing in nested environments" (Econometric Theory, In Press) I introduce novel test statistics designed to implement such inferences. These new test statistics do not suffer from the drawbacks and limitations of existing methods. Their distribution is derived formally, and implementation guidelines provided for practitioners. |
| Type Of Material | Data analysis technique |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Matlab and R packages for implementing the proposed techniques made available to anyone wishing to use them on their own data. |
| URL | https://sites.google.com/view/jpitarakis-esrc/home |
| Title | Techniques for comparing the out-of-sample prediction capabilities of competing econometric models using the forecast encompassing principle and with a focus on nested model comparisons |
| Description | A key component of the grant's research agenda is the development of new techniques for comparing the predictive accuracy of competing econometric models. A long standing and unresolved problem in this literature has to do with our ability to conduct formal tests of predictive accuracy when the models under consideration are nested. Suppose for instance that a particular model aims to explain an outcome of interest with a certain number of predictors. We wish to assess whether a larger model that contains the same predictors augmented by an additional set improves or deteriorates the model's ability to generate out of sample predictions (i.e., using new data). How can we formally test such hypotheses? In the working paper titled "Direct Multi-Step Forecast based Comparison of Nested Models via an Encompassing Test " I introduce a novel methodology that relies on the forecast encompassing principle and designed to implement such inferences. The methods I developed do not suffer from the drawbacks and limitations of existing methods. Equally importantly their properties are shown to be robust to the stochastic properties of predictors. |
| Type Of Material | Data analysis technique |
| Year Produced | 2023 |
| Provided To Others? | Yes |
| Impact | Matlab and R packages for implementing the proposed techniques made available to anyone wishing to use them on their own data. R package made publicly available on the CRAN repository; Matlab package made publicly available on Github (see URL below) |
| URL | https://sites.google.com/view/jpitarakis-esrc/home |
| Description | International collaboration with researchers from Universidad Carlos III de Madrid, Spain |
| Organisation | Charles III University of Madrid |
| Country | Spain |
| Sector | Academic/University |
| PI Contribution | Collaboration with the Department of Economics at the Universidad Carlos III de Madrid takes the form of co-authorship and joint dissemination of research outcomes related to this grant. Specifically, The PI based at the University of Southampton and Professor Jesus Gonzalo (based at UC3M) who is a named collaborator on the grant have collaborated on one of the outputs of the grant's research agenda. This has resulted in a publication in a leading academic journal dedicated to forecasting methods. In on-going work, we are extending the techniques developed in this already published research to a high dimensional setting. To encourage practitioners to consider using these methods we have also developed software applications (matlab and R) that will allow anyone wishing to use the methods on their own datasets to do so. These components of the agenda and collaboration have been disseminated in various international conferences and workshops. |
| Collaborator Contribution | Co-authorship of a research paper. |
| Impact | "Out of sample predictability in Predictive Regressions with many predictor candidates" (with Jesus Gonzalo). DOI: https://doi.org/10.48550/arXiv.2302.02866 |
| Start Year | 2022 |
| Title | Matlab Package to implement the methods developed in the paper "A Novel Approach to Predictive Accuracy Testing in Nested Environments", Econometric Theory (In Press, 2023) |
| Description | The package provides the means to implement the methods developed in "A Novel Approach to Predictive Accuracy Testing in Nested Environments", Econometric Theory (In Press, 2023)" on one's own data. The dedicated github site also includes a "demonstration" file illustrating usage via an example. These programs allow users to formally test the predictive power of competing models in very flexible environments that can accommodate features commonly encountered in time-series data (e.g., persistence). The focus is on the comparison of nested models. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2023 |
| Open Source License? | Yes |
| Impact | A dedicated open-access github site has been created to disseminate the methods used in this research. Anyone wishing to implement these model comparison methods can do so with the information and files provided on the site. |
| URL | http://doi.org/10.1017/s0266466623000154 |
| Title | Matlab Package to implement the methods developed in the paper "Direct Multi-Step Forecast based Comparison of Nested Models via an Encompassing Test" (https://arxiv.org/abs/2312.16099v1) |
| Description | The package provides the means to implement the methods developed in "Direct Multi-Step Forecast based Comparison of Nested Models via an Encompassing Test" (ArXiv https://arxiv.org/abs/2312.16099, December 2023) on one's own data. The dedicated github site also includes a "demonstration" file illustrating usage via an example. These programs allow users to formally test the out-of-sample predictive power of competing models in very flexible environments that can accommodate common features encountered in time-series data. The focus is on the comparison of nested models via a novel and original implementation of the encompassing principle. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2023 |
| Open Source License? | Yes |
| Impact | A dedicated open-access github site has been created to disseminate the methods used in this research. Anyone wishing to implement these model comparison methods can do so with the information and files provided on the site. |
| URL | https://arxiv.org/abs/2312.16099 |
| Title | Matlab package to implement the methods developed in "Serial dependence and Persistence robust inference in predictive regressions" |
| Description | TBC |
| Type Of Technology | Webtool/Application |
| Year Produced | 2025 |
| Open Source License? | Yes |
| Impact | TBC |
| Title | Matlab package to implement the methods developed in the paper "Detecting Sparse Cointegration" |
| Description | The package provides the means to implement the methods developed in "Detecting Sparse Cointegration". |
| Type Of Technology | Webtool/Application |
| Year Produced | 2025 |
| Open Source License? | Yes |
| Impact | Anyone wishing to implement these methods on their own data can do so with the information and files provided on the dedicated github site/repository. |
| URL | https://github.com/jpitarakis/sparse_cointegration |
| Title | R Package to implement the methods developed in the paper "A Novel Approach to Predictive Accuracy Testing in Nested Environments", Econometric Theory (In Press, 2023) |
| Description | The R package provides the means to implement the methods developed in "A Novel Approach to Predictive Accuracy Testing in Nested Environments", Econometric Theory (In Press, 2023)" on one's own data. The package allows users to formally test the predictive power of competing models in very flexible environments that can accommodate common features encountered in time-series data (e.g. persistence). The focus is on the comparison of nested models. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2023 |
| Open Source License? | Yes |
| Impact | Anyone wishing to implement these model comparison methods using the R package can do so with the information and files provided on the CRAN repository |
| URL | https://cran.r-project.org/package=pretest |
| Title | R Package to implement the methods developed in the paper "Direct Multi-Step Forecast based Comparison of Nested Models via an Encompassing Test" (December 2023, https://arxiv.org/abs/2312.16099) |
| Description | The package provides the means to implement the methods developed in "Direct Multi-Step Forecast based Comparison of Nested Models via an Encompassing Test" (ArXiv https://arxiv.org/abs/2312.16099, December 2023) on one's own data. The dedicated github site also includes a "demonstration" file illustrating usage via an example. These programs allow users to formally test the out-of-sample predictive power of competing models in very flexible environments that can accommodate common features encountered in time-series data. The focus is on the comparison of nested models via a novel and original implementation of the encompassing principle. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2024 |
| Open Source License? | Yes |
| Impact | Anyone wishing to implement these model comparison methods using the R package can do so with the information and files provided on the CRAN repository |
| URL | https://arxiv.org/abs/2312.16099 |
| Description | Dedicated website aggregating outcomes (papers, talks, publications,events) and disseminating software applications aimed at engaging with practitioners |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | I have created a dedicated website aggregating all forms of outputs, events and software associated with the grant. Research outputs are initially placed on ArXiv for them to be disseminated to a wide international audience. Linked pdfs and publication updates are provided on the website. The same website also contains links to R and Matlab packages that implement the methods developed as part of the grant. Anyone wishing to use these methods on their own data can do so with the information and illustrations provided (a dedicated GitHub site also provides the same information to potential users). The R packages are also made available on the CRAN repository under a public licence. Finally, the website also contains a "dissemination" section in which I advertise the events relevant to the grant's research agenda (e.g., forthcoming, and past conferences, dedicated workshops etc.) |
| Year(s) Of Engagement Activity | 2022,2023,2024 |
| URL | https://sites.google.com/view/jpitarakis-esrc/home |
| Description | Econometrics Workshop on Model Selection, Predictability and High Dimensionality: Theory and Applications. |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | As part of this research grant, multiple engagement activities were undertaken to ensure that the project's findings reached research users. These activities aimed to facilitate the adoption of advanced forecasting methodologies in finance, policy-making, and economic analysis while fostering broader awareness of the project's contributions. One of the major engagement activities was the international workshop organized under the theme "High-Dimensional Forecasting and Model Evaluation". This event brought together a mix of academic researchers, and policymakers to discuss recent advances in predictive modeling, forecast evaluation, and variable selection in large datasets. Participants: The workshop was attended by academic economists and analysts based at major Banks (e.g. Deutsche Bank). The workshop featured presentations and discussions on practical applications of the research, ensuring that non-academic users could understand and implement the methods. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://sites.google.com/view/sotoneconometrics2024/home |
| Description | Github site disseminating the software packages for potential users of the methods developed in this award |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The research conducted as part of this award resulted in numerous novel statistical methodologies. To support wider adoption of the research findings, the project team developed and publicly released open-source software tools in R and MATLAB, allowing industry professionals and policymakers to implement the forecasting techniques without deep technical expertise. A dedicated website was created to provide non-specialists with access to the tools, explanations, and research findings in an accessible format. The GitHub repository hosting the forecasting comparison tools has been promoted among data scientists, quantitative analysts, and econometricians. |
| Year(s) Of Engagement Activity | 2023,2024,2025 |
| URL | https://github.com/jpitarakis |