Calibrating Coercive Policy Transfer: Developing a Methodological Toolkit for Evaluating Policy Compatibility.
Lead Research Organisation:
University of Portsmouth
Department Name: Sch of Area Stud, Hist, Politics & Lit
Abstract
The proposed research will extend the methodological approach (Dynamic Patterns Synthesis - DPS) applied within my Ph.D. thesis by further developing the data visualization of results to allow for quicker, and easier, identification of patterns, relationships, and trends as well as advance the communication of findings while developing a track record of publications within this emerging quantitative mixed methods discourse.
The DPS method is situated within complexity theory and configurational methods (Haynes, 2017). It was included in recent ESRC CECAN advice on policy evaluation methods supplied to the UK civil service (Bicket, et al, 2020). It combines the complementary strengths and weaknesses of cluster analysis (CA) and aspects of configurative approaches like qualitative comparative analysis (QCA) in a specific and longitudinal policy research situation. In its application within my thesis, it became evident that the DPS method provided an avenue for assessing and interpreting aspects of country convergence, as well as the changing patterns of convergence that exist within and between cases. In instances where complexity is high, such DPS interpretations are also subject to the researchers' epistemological and ontological approaches. This could sometimes make the understanding inadequate and in need of further research and theorizing.
For this reason, this project aims to advanced the application of the DPS method by applying it in an RStudio software package (creating a Java program or 'do-file') and include the application of 'Natural Language Programming' (documentary analysis) to complement the method. This is aimed at providing an additional layer to the analysis of complexity across cases and assist in establishing "a history of policy and other processes in relation to each of the cases in a representative sample of areas" (Byrne & Uprichard 2007, p. 50). Here, users of this method would also be able to identify similarities and differences in policy documents and this could be used to further interpret DPS results. Through the ESRC funding, I would further develop this advanced methodological approach and improve the data visualization of results while developing a track record of publications within this emerging quantitative mixed methods discourse.
The DPS method is situated within complexity theory and configurational methods (Haynes, 2017). It was included in recent ESRC CECAN advice on policy evaluation methods supplied to the UK civil service (Bicket, et al, 2020). It combines the complementary strengths and weaknesses of cluster analysis (CA) and aspects of configurative approaches like qualitative comparative analysis (QCA) in a specific and longitudinal policy research situation. In its application within my thesis, it became evident that the DPS method provided an avenue for assessing and interpreting aspects of country convergence, as well as the changing patterns of convergence that exist within and between cases. In instances where complexity is high, such DPS interpretations are also subject to the researchers' epistemological and ontological approaches. This could sometimes make the understanding inadequate and in need of further research and theorizing.
For this reason, this project aims to advanced the application of the DPS method by applying it in an RStudio software package (creating a Java program or 'do-file') and include the application of 'Natural Language Programming' (documentary analysis) to complement the method. This is aimed at providing an additional layer to the analysis of complexity across cases and assist in establishing "a history of policy and other processes in relation to each of the cases in a representative sample of areas" (Byrne & Uprichard 2007, p. 50). Here, users of this method would also be able to identify similarities and differences in policy documents and this could be used to further interpret DPS results. Through the ESRC funding, I would further develop this advanced methodological approach and improve the data visualization of results while developing a track record of publications within this emerging quantitative mixed methods discourse.
Organisations
People |
ORCID iD |
Publications
Alemna D
(2022)
Monetary Convergence Across the Economic Community of West African States: Lessons for the Envisioned West African Monetary Union
in Complexity, Governance & Networks
Haynes P
(2022)
A Systematic Literature Review of the Impact of Complexity Theory on Applied Economics
in Economies
Haynes P
(2022)
From aggregations to multimethod case configurations. Case diversity in quantitative analysis when explaining COVID-19 fatalities
in International Journal of Social Research Methodology
| Description | This project sought to extend the application of Dynamic Patterns Synthesis (DPS), a research method that provides more realist comparative analytic techniques to social synthesis, by advancing its visual presentation and application with other methods. While this project did not intend to produce direct findings but rather advance the publicity and use of DPS, some achievements can be identified in the fulfillment of the project objectives. These are four-fold: First, as part of the Fellowship, the PI undertook specialist research methods training in computational social science methodologies (including training on advanced text analytics in R at the Australian National University; Machine Learning with R at Mind Project - London; amongst others). Secondly, the training has provided the PI with a specialist skillset and facilitated the publication of three peer-reviews publications relating to DPS, including a systematic review of the application of complexity theory in public policy research; on issues regarding monetary integration; and in comparison with other methods (see Haynes and Alemna, 2022; Alemna, 2022; Haynes and Alemna, 2022). There are also three solo publications in the pipeline that are an output of the fellowship, one applies DPS in assessing youth development, and the remaining two combine a synthesis of Text Analytics and DPS. The 'do-file' on the application of DPS in R would be available with upcoming publications. This would include a component on combining DPS with Text Analytics. In relation to simplifying the visualization of results and use of the DPS method, the PI and founder of the DPS (Prof. Philip Haynes) have recently published an online "how to do DPS" E-book on the application of DPS in Microsoft Excel. This effort seeks to simplify (and widen) the use of DPS. Thirdly, the fellow and training provided an opportunity for the PI to teach these methods and successfully advance from an Associate Fellow of Higher Education status to a Fellow of Higher Education. Lastly, in order to widen public awareness of DPS, the PI sort to disseminate the application of DPS through the presentation of research at two specialist workshops and three practitioner conferences. |
| Exploitation Route | Findings, from the systematic review understanding by the PI and Prof. Haynes, highlight some interesting trends in the application of complexity-related methods in public policy research. These include an observed growth in the use of complexity-appropriate methods. However, this growth is concentrated in areas such as economics and ecological studies. There is also limited growth in the use of case-based methods as compared to other simulative computational methods (eg., agent-based modeling) with the most active writing originating from Europe, America, or Asia. These findings suggest a need for more advances in the use of case-based methods, especially in relation to interdisciplinary applications, and widening usage amongst marginalized groups. |
| Sectors | Environment,Financial Services, and Management Consultancy,Government, Democracy and Justice,Other |
| Description | Panel/Committee Member for the "Being a Post Doc, Careers in and out of Academia" panel at the Reinvigorating Area Studies: A DTP Training Event for ESRC and non-ESRC Students. 2022. |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Postgraduate students |
| Results and Impact | Panel member for the "Being a Post Doc, Careers in and out of Academia" panel chaired by Dr Liz Wren-Owens (Cardiff University) at the Reinvigorating Area Studies: A DTP Training Event for ESRC and non-ESRC Students. Here, I shared my experience with post-doc applications and careers in and out of academia |
| Year(s) Of Engagement Activity | 2022 |
| Description | Panel/Committee Member for the "How did you get here?" panel for the South Coast Doctoral Training Partners discussion event |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Postgraduate students |
| Results and Impact | Chaired by Prof. Haynes (University of Brighton) this event was aimed at providing advise and support for students looking for careers in and out of academia. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Panel/Committee Member for the "Methods Café: Advances in Quantitative Methods" panel at the Reinvigorating Area Studies: A DTP Training Event for ESRC and non-ESRC Students. 2022 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Postgraduate students |
| Results and Impact | As part of the A DTP Training Event for ESRC and non-ESRC students, I was a discussant on the Methods Cafe panel with Dr Christian Arnold (Cardiff University) |
| Year(s) Of Engagement Activity | 2022 |
| Description | Presentation at the 27th Political Studies Association Annual Conference. Politics from the Margins, University of York, UK |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Presentation Title: Observing Longitudinal Patterns in the Forms of Marginalisation Facing Youth Overview: This presentation focused on a working paper that advances the use of macro-level quantitative data in assessing government responses to challenges facing the Youth. Abstract: Trends in the global development discourse have highlighted the importance of youth development in the world of today. This has been emphasized in the achievement of the Millennium Development Goals and the subsequent Agenda 2030 Global Development Goals (Sustainable Development Goals). While nations continuously attempt to ensure progress in the development of youth, the Covid-19 pandemic has uncovered existing inequalities that have marginalized the youth. Evidently, it is clear that government responses need to adapt swiftly taking into consideration factors such as technological advancements and the push for further involvement of the youth in the political space post-Covid-19. The current challenge faced by policymakers is the identification of priority areas to be strengthened in an effort to build a better future for the youth, especially in the post-Covid-19 era. In this paper, two quantitative case-based configurational methods (Cluster Analysis and Configurational tables) are applied to observe cross-national patterns of convergence and diversity in the forms of marginalization facing the youth. Cluster Analysis is used to provide a path for conceptualizing the relationship between countries and dimensional configurations. Configurational tables further offer suggestions on associated cluster variable trends for explaining youth progress. This provides an original and innovative approach to an assessment of the state of the art. The findings illustrate multi-finality in achieving youth progress. While most economies seem to be providing the youth with essential basic human needs, in less developed economies this is lacking. Nevertheless, more work needs to be done in ensuring the youth feel safe in their communities and environmental factors must be taken into consideration in government undertakings. These patterns signal areas for national and cross-national policy priorities. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Presentation at the Doctoral Training Partnership Annual Conference, Future of Area Studies, Cardiff University, UK |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | Presentation Title: Comparing IMF Interventions in Development Assistance Lending Overview: This presentation focused on the findings from a working paper that applies a mixed-method approach to evaluating IMF interventions. The presentation focused on highlighting some key findings and relevance for research within this area. Abstract: The International Monetary Fund (IMF) was created to serve as an international financier that loans to its member countries when they experience Balance of Payments problems. Alongside its lending, the IMF imposes conditionality on loans. This is claimed to have resulted in an overarching approach to policy conditions with disputed contextual relevance. Using Quantitative Analysis of Textual Data to examine the corpus of IMF policy interventions, this paper seeks to test the assumption that, through its institutional focus, the IMF has become a policy subsystem promoting certain economic ideologies via the use of conditionalities. The findings show that although emphasizing the tailored approach to IMF interventions, the analysis of the Letters of Intent suggests that the privatization of public sector entities, economic deregulation and, trade and financial liberalization across policy interventions still play a key role in IMF activities. This stresses the totalizing logic of the IMF as its fundamental principles fall back to an overarching neoliberal approach to problem-solving. It is further argued that, based on the findings, the IMF serves as a global stimulus for the transfer and diffusion of neoliberal ideologies. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Presentation at the Faculty of Humanities and Social Sciences Technology in Research Workshop, University of Portsmouth, UK |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Postgraduate students |
| Results and Impact | Presentation Title: Text Mining IMF Interventions in Sub-Saharan Africa and Latin America Overview: This presentation was based on a working paper that demonstrates the application of text-mining machine-learning techniques in analyzing policy documents. In this presentation, I provided a practical step-by-step approach to data collection, coding, and analysis of over 120 IMF policy documents implemented within Sub-Saharan Africa and Latin America. I also demonstrated findings of a convergent effect in IMF ideology on policy lending which supports academic findings within this field. Abstract: The International Monetary Fund (IMF) was created to serve as an international financier that loans to its member countries when they experience Balance of Payments problems. Alongside its lending, the IMF imposes conditionality on loans. This is claimed to have resulted in an overarching approach to policy conditions with disputed contextual relevance. Using Quantitative Analysis of Textual Data to examine the corpus of IMF policy interventions, this paper seeks to test the assumption that, through its institutional focus, the IMF has become a policy subsystem promoting certain economic ideologies via the use of conditionalities. The findings show that although emphasizing the tailored approach to IMF interventions, the analysis of the Letters of Intent suggests that the privatization of public sector entities, economic deregulation, and trade and financial liberalization across policy interventions still play a key role in IMF activities. This stresses the totalizing logic of the IMF as its fundamental principles fall back to an overarching neoliberal approach to problem-solving. It is further argued that, based on the findings, the IMF serves as a global stimulus for the transfer and diffusion of neoliberal ideologies. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Presentation at the Social Policy Association Annual Conference, Inter-generationality: Challenges and Prospects. Swansea University. UK |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Presentation Title: From aggregations to multimethod case configurations. Case diversity in quantitative analysis when explaining COVID-19 Fatalities Overview: This presentation was with regard to a coauthored publication by Prof. Haynes and Dr. Alemna. It sort to compare the strengths and weaknesses of three quantitative research methods (Regression; Dynamic Pattern Synthesis; and Qualitative Comparative Analysis) in understanding differences in Covid-19 fatality ratios across 33 OECD countries. Abstract: Three quantitative methods are compared for their ability to understand different COVID-19 fatality ratios in 33 OECD countries. Linear regression provides a limited overview without sensitivity to the diversity of cases. Cluster Analysis and Dynamic Patterns Synthesis (DPS) gives scrutiny to the granularity of case similarities and differences, and reveals case exceptions. Qualitative Comparative Analysis (QCA) develops causal theory about what conditions are sufficient for explaining outcomes by using robust and transparent conventions. Configurational case-based methods offer important advantages over inferential statistics when there is a need to focus on diversity in small n. These techniques can be combined as multi-methods. DPS and QCA can be used concurrently to aid research insights. These methods are also strengthened by additional qualitative evidence about the cases. |
| Year(s) Of Engagement Activity | 2022 |