cemmap Legacy Centre

Lead Research Organisation: Institute for Fiscal Studies
Department Name: IFS Research Team

Abstract

This project aims to work with business and public sector users to develop a focused sequence of conferences, masterclasses, and training courses to bring frontier research methods to the fingertips of non-academic economists. To further this aim, the project will found a new blog, CeMMAP "Econometric Insights", to produce accessible versions of frontier research tools.

The project will start with a pair of workshops that bring together economists and researchers from institutions including the Bank of England, the FCA, the CMA, OFCOM, KPMG, and others. At these workshops, private and public sector economists will present practical experience using 'big data' methods and machine learning techniques. Academics will present recent advances in core research. The workshops will be used to share ideas, spread knowledge, and to discuss topics and formats for a sequence of training courses and masterclasses to further share knowledge across the network of researchers.

Topics for the series of conferences and training courses will likely stick close to the core research of CeMMAP but will respond to core needs of users. CeMMAP's core research develops and applies tools to extract information about individual behaviour, and influences on it, from data on individuals' actions and outcomes in their normal environment. Typically, this environment is not a controlled laboratory-like situation. The 'individuals' the project studies can be, for example, people, households or enterprises.

This core research is organised under five headings.
1) Big data and machine learning. We study the performance of big data and machine learning methods, develop models and research applications in which economic and social science poses deep questions and guides model construction. Applications study the impact of policies to change behaviour using taxes on e.g. sugar and fat, and the impact of planning laws on housing supply and prices.
2) Dynamics and Complexity. We are developing models and tools to understand influences on complex life planning choices made by people in the face of uncertainty and incomplete information. Applications are studying e.g. housing location and tenure choice.
3) Robust models. We are studying the utility and performance of incomplete models of complex economic and social process in which only theory-grounded restrictions on behaviour are employed. Applications consider e.g. models of auction bidding behaviour, determination of well-being and firm market entry and product choice.
4) Networks and interactions. We study properties of tools to estimate aspects of network structure and develop methods for estimating the impact of network structure on outcomes. Applications consider e.g. employer-employee matching processes and trade flows between countries.
5) Survey design and measurement. We research the optimal choice of scope, scale and measurement quality in investigations such as field experiments constructed to answer specific social science questions.

Planned Impact

This Project will benefit:
(A) Social science researchers, who use microdata to understand human behaviour and the influences on it.
(B) Statisticians, both practitioners and theorists, who develop statistical procedures and research their properties.
(C) Survey designers who gather data on individuals' circumstances, decisions and outcomes, and designers and analysts of policy evaluations and of social and economic lab and field experiments.
(D) Policy and decision makers in the public and private sectors who use microdata evidence to inform policy choices and analysts who process data to inform decisions.
(E) The general public.

(A) and (B) CeMMAP's research improves the tools used to extract information from microdata directly benefitting these groups. The Project's dissemination and training at basic and advanced levels will spread knowledge of new tools, raising the quality of empirical social science research.
(C) Designers of surveys, evaluations and experiments will benefit from the Project's activities that share new tools for understanding identification, big data and survey design. The conferences, training and blog will spread knowledge on what to measure, how to measure it, and other aspects of design, as well as information about new tools to analyse big data. The Project will help private sector survey designers use our tools for more efficient data collection, enabling better informed choices on prices, investment and product portfolios.
(D) Policy and decision makers will benefit from the Project's and activities by gaining better knowledge of how to conduct higher-quality analysis of microdata. The conferences, training and blog will enable these users to share their experiences and also to gain a more detailed understanding of new advances in the microdata revolution.
(E) The general public is often presented with statistics as if they give a direct insight into causal relationships. The Project's Econometric Insights, though primarily targeted at professional users will provide insights to the general public as well. Some insights will also be published as briefing notes for public consumption.
To ensure impacts are achieved, the activities are very tightly tied to collaboration with users. Users will be integrated in all aspects of the conferences, training courses, and masterclasses.
Benefits from the Project's work will range from immediate (e.g. improved skills from training) to long-term (e.g. accumulation of knowledge about behaviour). Intermediate benefits will also arise as novel methods and approaches deliver higher-quality evidence and better policy choices and networks of users with common interests grow.

Publications

10 25 50
 
Description Masterclass, Econometrics of Nonseparable Models (Rosa Matzkin, From: 8 May 2019 Until: 9 May 2019) Venue Friends House 173 Euston Road, London, NW1 2BJ Prices HE Delegates: £75 Charity or Government: £200 Other Delegates: £450 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Study participants or study members
Results and Impact When confronting an economic model with data, one usually encounters a situation where some important variables, such as tastes and productivity that appear in the model in nonadditive, nonseparable ways, are unobserved. Rather than transforming the model into one where the unobservables enter in separable ways, the nonseparable approach considers identification and estimation of the original model. The original model satisfies the economic restrictions of the model, which aid in identification and estimation. These restrictions are often lost in separable transformations.

This masterclass will cover identification and estimation methods for nonseparable models, with emphasis on nonparametric methods. First, some key econometric techniques used in nonseparable models will be presented. Next, it will be shown how these techniques have been used and extended to study particular econometric models.
Year(s) Of Engagement Activity 2019
 
Description Masterclass, Personalised Patient Care Under Uncertainty (Charles F. Manski, From: 28 March 2019 Until: 29 March 2019) Venue The IFS Prices HE Delegates: £75 Charity or Government: £200 Other Delegates: £450 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Study participants or study members
Results and Impact A masterclass given by Charles Manski (Northwestern) sponsored by CeMMAP and the Economics Research Initiative at Duke University, March 28-29 2019

This masterclass is the UK presentation of the Masterclass given at Duke University last September.
This masterclass will focus on evidence-based decision making for patient care under uncertainty, in which clinicians face only limited ability to predict patients' future illness and treatment response. To deal with this inherent uncertainty, partial identification analysis can be applied to make credible predictions for patient outcomes. This analysis motivates the use of decision criteria with well understood properties. Particular focus will be given to the minimax-regret criteria, which specifies a decision rule as uniformly close to the optimal decision rule as possible given the underlying uncertainty of patient outcomes.
Year(s) Of Engagement Activity 2019
 
Description Masterclass, Public Lectures on Structural Econometrics (Professor Robert Miller (Carnegie Mellon)) Venue Online only Prices Free 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Study participants or study members
Results and Impact The course analyzes the structural estimation and testing of nonlinear models. We explore relationships between economic theory, identification, estimation and econometric practice. It develops structural approaches for analyzing large cross sectional and longitudinal data sets, by exploiting restrictions derived from the equilibrium dynamic outcomes in individual discrete choice optimization problems and non- cooperative games. We investigate empirical content, characterize identification, evaluate alternative estimators and testing procedures, as well as consider counterfactuals. It has six segments:

The first segment gives a flavor of structural estimation, by show how some examples of economic models induce a data generating process that provides the basis for estimating the structure of the economic environment, critical for conducting counterfactual simulations. We analyze the estimation of preferences in a model of continuous choices in a competitive equilibrium with complete markets; we derive inequalities that are induced by an equilibrium in a limit order market; we quantify the importance of moral hazard in an optimal contracting model; and we introduce the estimation of dynamic discrete choice optimization models.
Then we profile many estimators that have been used to summarize data. They can be placed into four categories: estimators for linear data generating processes, parametric nonlinear processes, plus nonparametric and semiparametric estimators.
The rationale for the third segment of the course is that the exact distribution of most nonlinear estimators is intractable, explaining why we resort to large sample theory. We analyze several notions of convergence, present laws of large number and central limit theorems, derive the asymptotic distribution of several nonlinear estimators, and show how to conduct hypothesis tests.
The second half of this course integrates a discussion of the identification of primitives or deep parameters in economic models with data generating process of the equilibrium. The first segment focuses on various kinds of auctions and contracts.
Then we analyze dynamic discrete choice models in more depth; we derive a representation of the value function, prove identification under the conditional independence assumption, illustrate how various CCP estimators work, analyze the concept of finite dependence, as well as relax the conditional independence assumption.
Lastly we apply structural estimation methodology to lifecycle models of labor economics and product innovation.
Year(s) Of Engagement Activity 2020,2021
 
Description Training Course, Panel data methods (Jeffrey M. Wooldridge, From: 14 May 2020 Until: 16 May 2020) Venue Online only Prices HE Delegates: £144 Charity or Government: £264 Other Delegates: £570 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Study participants or study members
Results and Impact This course in panel data econometrics, presented from a microeconometrics perspective, will cover linear panel data models with unobserved heterogeneity, including discussions of the strengths and weaknesses of the various estimation methods.

Basic estimation methods include random effects, fixed effects, and first differencing. Instrumental variables estimation of models without strictly exogenous explanatory variables will also be covered, including those that contain contemporaneously endogenous variables and those that contain lagged dependent variables. Estimation of linear models with heterogeneous trends and heterogeneous slopes will also be covered. Special considerations with unbalanced panels will be discussed, including how to test for sample selection and attrition bias.

The statistical package Stata will be used to illustrate the methods during lectures and in obtaining hands-on experience during the practical work. Please note that this training course will be given in a microeconometrics laboratory where computers will be provided with the neccessary programmes.
Year(s) Of Engagement Activity 2020
 
Description Training Course, Panel data methods (Jeffrey M. Wooldridge, From: 18 May 2020 Until: 20 May 2020) Venue Online only Prices HE Delegates: £144 Charity or Government: £264 Other Delegates: £570 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Study participants or study members
Results and Impact This course in panel data econometrics, presented from a microeconometrics perspective, will cover linear panel data models with unobserved heterogeneity, including discussions of the strengths and weaknesses of the various estimation methods.

Basic estimation methods include random effects, fixed effects, and first differencing. Instrumental variables estimation of models without strictly exogenous explanatory variables will also be covered, including those that contain contemporaneously endogenous variables and those that contain lagged dependent variables. Estimation of linear models with heterogeneous trends and heterogeneous slopes will also be covered. Special considerations with unbalanced panels will be discussed, including how to test for sample selection and attrition bias.

The statistical package Stata will be used to illustrate the methods during lectures and in obtaining hands-on experience during the practical work. Please note that this training course will be given in a microeconometrics laboratory where computers will be provided with the neccessary programmes.
Year(s) Of Engagement Activity 2020
 
Description Training Course, Policy evaluation methods (Barbara Sianesi, From: 14 January 2020 Until: 17 January 2020) Venue UCL Economics Department Drayton House, 30 Gordon Street, London, WC1H OAX Prices HE Delegates: £420 Charity or Government: £770 Other Delegates: £1662 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Policymakers/politicians
Results and Impact Training Course, Policy evaluation methods (Barbara Sianesi, From: 14 January 2020 Until: 17 January 2020)
How can one evaluate whether a government labour market programme such as the Work Programme, or a subsidy to education such as the EMA is actually working? This course deals with the econometric and statistical tools that have been developed to estimate the causal impact on one or more outcomes of interest of any generic 'treatment' - from government programmes, policies or reforms, to the returns to education, the impact of unionism on wages, or of smoking on own and children's health.

After highlighting the 'evaluation problem' and the challenges it poses to the analyst, we focus on the main empirical methods to solve it. Specifically, in this 3 and a half-day course we cover:

Randomised social experiments
Naive non-experimental estimator
Natural experiments or instrumental variables
Regression Discontinuity Design
Regression analysis
Matching methods
Before-after
Difference-in-differences
Synthetic control methods
For each of these approaches, we give the basic intuition, discuss the assumptions needed for its validity, highlight the question it answers and formally show identification of the parameter of interest. The relative strengths and weaknesses of each approach are discussed in detail, drawing from example applications in the economics literature. Each method will be implemented 'hands-on' in practical Stata sessions.
Year(s) Of Engagement Activity 2020
 
Description Training Course, Remote Policy Evaluation Methods course (Barbara Sianesi, From: 16 November 2020 Until: 17 November 2020) Venue Online only Prices HE Delegates: £504 Charity or Government: £924 Other Delegates: £1994.40 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Study participants or study members
Results and Impact How can one evaluate whether a government labour market programme such as the Work Programme, or a subsidy to education such as the EMA is actually working? This course deals with the econometric and statistical tools that have been developed to estimate the causal impact on one or more outcomes of interest of any generic 'treatment' - from government programmes, policies or reforms, to the returns to education, the impact of unionism on wages, or of smoking on own and children's health.

After highlighting the 'evaluation problem' and the challenges it poses to the analyst, we focus on the main empirical methods to solve it. Specifically, in this 3 and a half-day course we cover:

Randomised social experiments
Naive non-experimental estimator
Natural experiments or instrumental variables
Regression Discontinuity Design
Regression analysis
Matching methods
Before-after
Difference-in-differences
Synthetic control methods
For each of these approaches, we give the basic intuition, discuss the assumptions needed for its validity, highlight the question it answers and formally show identification of the parameter of interest. The relative strengths and weaknesses of each approach are discussed in detail, drawing from example applications in the economics literature. Each method will be implemented 'hands-on' in practical Stata sessions.
Year(s) Of Engagement Activity 2020
 
Description Training Course, Remote Policy Evaluation methods course ( From: 25 January 2021 Until: 26 January 2021) Venue Online only Prices HE Delegates: £504 Charity or Government: £924 Other Delegates: £1994.40 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Study participants or study members
Results and Impact How can one evaluate whether a government labour market programme such as the Work Programme, or a subsidy to education such as the EMA is actually working? This course deals with the econometric and statistical tools that have been developed to estimate the causal impact on one or more outcomes of interest of any generic 'treatment' - from government programmes, policies or reforms, to the returns to education, the impact of unionism on wages, or of smoking on own and children's health.

After highlighting the 'evaluation problem' and the challenges it poses to the analyst, we focus on the main empirical methods to solve it. Specifically, in this 3 and a half-day course we cover:

Randomised social experiments
Naive non-experimental estimator
Natural experiments or instrumental variables
Regression Discontinuity Design
Regression analysis
Matching methods
Before-after
Difference-in-differences
Synthetic control methods
For each of these approaches, we give the basic intuition, discuss the assumptions needed for its validity, highlight the question it answers and formally show identification of the parameter of interest. The relative strengths and weaknesses of each approach are discussed in detail, drawing from example applications in the economics literature. Each method will be implemented 'hands-on' in practical Stata sessions.
Year(s) Of Engagement Activity 2021
 
Description Training course, Estimating causal parameters from a high dimensional model using the lasso in Stata (David Drukker, 21 October 2019) Venue Friends House 173 Euston Road, London, NW1 2BJ Prices HE Delegates: £50 Charity or Government: £50 Other Delegates: £50 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Study participants or study members
Results and Impact Training course, Estimating causal parameters from a high dimensional model using the lasso in Stata (David Drukker, 21 October 2019)
This six-hour course shows how to estimate causal parameters from a high-dimensional model using the inferential lasso commands in Stata.

It begins by introducing high-dimensional models and discusses which estimation methods work and which estimation methods do not work. This introduction shows how the lasso is used in these methods.

Next, the course provides an introduction to how the lasso is implemented in Stata and an overview of relevant theory. This part also discusses how to use Stata's lasso commands to solve prediction problems.

Finally, the course discusses the Stata implementation and the relevant theory for a series of commands that estimate causal parameters from high-dimensional models. These parts of the course discuss commands for linear models, logit models and Poisson models with exogenous variables. They also discuss commands for linear models with endogenous variables. Some extensions to average treatment effects for exogenous treatments are also discussed.
Year(s) Of Engagement Activity 2019