National Centre for Research Methods 2020-2024

Lead Research Organisation: University of Southampton
Department Name: Sch of Economic, Social & Political Sci

Abstract

Social science transforms our understanding of the world, underpins the development and evaluation of public policy and business practice and informs national and international debate on matters of social, economic and political importance. While the quality of empirical research clearly depends on many factors, methodology and methods are critical. The ways in which social scientists collect, analyse and interpret social and economic data need to reflect rapid changes such as the emergence of new types and sources of data, the need for interdisciplinary working, qualitative data being produced and analysed at scale and the emergence of machine learning and Artificial Intelligence as new methodological frameworks. Such diversification and proliferation poses significant challenges to the robustness of our science, as does the pace of change. Hence, training social scientists to a high standard in using appropriate, and sometimes new, research methods is of vital and increasing importance.
The proposed National Centre for Research Methods (NCRM) will deliver a comprehensive programme of cutting-edge research methods training across the UK. It will advance methodological understanding and practice across all sectors, disciplines and career stages, and closely engage with key stakeholders, coordinating and enhancing research methods training and capacity building (TCB). The principal aim of the Centre is to increase research methods capability in the social sciences through TCB activities in the application of core and advanced research techniques. It will provide a vehicle for capturing methodological developments from the wider community, both within the social sciences and beyond, and drive them through to capacity building maturity.
The NCRM will be delivered by a partnership of three internationally leading centres of methodological excellence: the Universities of Southampton, Manchester and Edinburgh. This partnership will be significantly enhanced and expanded to include strategically selected Centre partner institutions who, through a nominated NCRM senior fellow, will deliver training locally to ensure geographical and subject expertise diversity. The nine institutions are UCL, NatCen, WISERD, Bristol, Exeter, Essex, Leeds, Liverpool and Glasgow. Effective engagement with key stakeholders will be at the core of all activities, including relevant ESRC investments, government departments, third-sector organisations and with business and industry. There will be several mechanisms for delivering this, e.g. to expand the data resource training network, cross-sectoral involvement in capacity building innovation fora, and through the Centre's Advisory Board. Opportunities for further funding streams and sustainable growth will be explored.
Building on the successful training programme developed and delivered by NCRM, the new NCRM will introduce innovative developments in the structure, content and delivery of training activities. It will comprise four core activities:
1. A programme of innovative face-to-face, blended and online learning in social science research methods delivered where it is needed.
2. A TCB innovation pipeline: a horizon scanning function which blends training needs analysis, systematic audit, knowledge creation and innovation fora to identify areas of methodological innovation with subsequent targeted training and resource development to build capacity in identified domains.
3. A new online learning portal including online learning content and information resources. The NCRM will provide a range of interactive content with a mixture of standalone instant access materials (including e-books, videos, interactive slide decks) through to whole modules based within a VLE.
4. National leadership in social science methods training; engaging, collaborating and co-ordinating activity and providing fora for other trainers.

Planned Impact

Who will benefit? A broad range of academic and non-academic users will benefit from the high quality training and resources in core and advanced social science research methods provided by NCRM. This includes researchers at all career stages, from disciplines within and beyond the social sciences, and from across all sectors, including those working in central and local government, the Office for National Statistics, public and third sector organisations, commercial and not-for-profit data collection agencies and in commercial and market research industries. The direct beneficiaries will be people conducting research, including doctoral researchers. Teachers and trainers of social science research methods will also benefit significantly from the coordination and support of their pedagogic development. Indirect beneficiaries include people planning or commissioning cutting-edge research to address social issues and policy-makers and stakeholders.
How will they benefit? Researchers will benefit by being able to: - Learn new research methods skills, keep up with the latest developments, and refresh and update understanding of core techniques through a flexible programme of face-to-face, blended and online training resources developed and delivered by internationally leading researchers, supported by pedagogic expertise. - Expand their methods competencies, which will help to deliver robust research projects. - Access training at affordable rates in accessible locations throughout the UK. - Access online resources and subscribe for up-to-date information on particular methods and events. - Participate in 'bring your own data' sessions and training using authentic data, available through UKDS, to resolve authentic research challenges.
Teachers and trainers of research methods will benefit from being part of a pedagogic community, accessing debates and materials and conducting teacher inquiry to enrich their practice. Students of research methods will benefit from the ways in which this pedagogic culture will make training increasingly engaging and effective. Given the need for lifelong learning and continuous skills development and training in the face of challenges arising from new technology and new sources of data, the training programme of the NCRM and the coordinated ESRC training investments will have direct impacts on the UK economy through improved employability skills and employment opportunities.
Finally, the NCRM will have an impact on the UK public and the wider economy, through helping to ensure that social science research is of high quality, making best use of methods so that research can lead to better quality evidence and better inform public policy.
What will be done to ensure that they have the opportunity to benefit? The NCRM team is exceptionally well placed to reach out to academic and non-academic users alike. Core members and partners are distributed strategically across expert areas, disciplines, sectors and geographic regions. We will utilise our strong links with key stakeholders across all sectors, built through longstanding research, training and consultancy partnerships. To optimise engagement we will: - Ensure the Advisory Board includes beneficiaries from all sectors who will have a voice in shaping the NCRM strategy and content of the training programme. - Conduct a series of training needs and impact analyses. We will particularly involve key stakeholders and gatekeepers representing the different sectors and disciplines in consultations to ensure that the planned training activities are aligned with user needs. - Operate a comprehensive communication strategy tailored to different groups. This will include mailing lists, e-bulletin, webinar series and social media. We will build on NCRM's current social media presence, while developing an entirely new branding style and identity for NCRM. A monthly e-Bulletin will promote key training activities, methodological developments and opportunities.
 
Description This NCRM Centre grant is primarily an infrastructure investment focussing on training and capacity building and, as a result, the grant does NOT include any core-funded research projects. The Centre therefore does not follow the typical structures of a research grant or research centre. We therefore do not produce research result with key findings. Also, our progress and outputs are regularly reported to the ESRC in detail in monthly and bi-annual progress meetings and progress reports are submitted. Here we summarise some highlights.
The Centre has 10 workstreams, all of which have progressed well and all KPIs have been achieved (data until end of 2023, the last time of reporting KPIs). NCRM includes a number of workstreams that have led to key findings and impact. We report here on the following areas: a.) achievements and impact by the strategic, high-quality training and capacity building programme; b.) the pedagogy workstream, c.) horizon scanning and innovation pipeline, and d.) engagement with key stakeholders (leading to TCB output). The following reports on each of these workstreams:
a.) NCRM has implemented a comprehensive, strategically informed, high-quality training and capacity building programme, covering a wide range of advanced and innovative training courses and events, although also covering core and intermediate training. The programme is strategically informed, including the activities below, such as work on pedagogy/how to deliver research methods training and teaching, engagement with key stakeholders and horizon scanning, 'Futures methods' identification and development and regular ongoing training needs investigations.
Between Jan 2020 and March 2024, we have run 366 training days (230 courses) with in total more than 4,660 course participants. Number of views and/or downloads of our online resources has continued to be buoyant and we have had more than 1M unique visitors to our website. In addition to course participants we also had more than 6,500 event participants (although the latter is more difficult to specify, example events include the annual lecture 2023, RMeF, DRTN and MSIG webinars, and the film festival and events run by recipients of awards via the research methods call).
We have continued to further expand provision and uptake, including for example another spring school, a second online research methods festival and a series of innovation fora in innovative and advanced methods, bringing together key experts and users across all sectors and from beyond the Social Sciences. The TCB workstream strand has created a very wide range of impact, following the Centre's comprehensive strategic impact framework.
Courses address topics in identified training needs and reflect our focal areas (e.g. our focal areas for engagement). Examples are specific topics in: Respondent Centred Design, Impact Evaluation, SEM, Causal Inference, Ethical Open-Source Investigation: A Deep Dive into Key Skills, Creative and Participatory Research Methods, Deliberative Research, Web Scraping, and The Bayesian modelling Summer School. We have also trialled innovative forms of training delivery, for example a 2-week online film festival "Screening the Social", showcases interdisciplinary, cutting-edge ways of using the moving image in social research.
The programme of the second Research Methods e-Festival in late 2023 consisted of 100 sessions (120 hours of content) and included training in a variety of formats and a keynote on evaluating impact. Demand was high with the 1,500-participant capacity reached within three months of opening bookings.
b.) The pedagogy workstream led by Nind, has produced wide ranging outputs, including most recently bitesize Lessons for Teaching Social Science Research Methods: 1. Active Learning; 2. Experiential Learning; 3. Learning from Learners; 4. Teaching Digital Methods to Students without Technology Backgrounds. The systematic review informed the creation of a pedagogy network which has continued to expand and develop (now more than 30 members). The Handbook of Teaching and Learning Social Research Methods (Edward Elgar, edited by Nind) has been published. The Handbook is an adaptation to the COVID situation in which methods teachers were reflecting on their pedagogic practice and often pausing from research practice. Nind's introductory chapter shows how the NCRM pedagogic research has been shaping the pedagogic culture in research methods education. A new creation is a Teatime for Trainers, launched in January 2024, targeting NCRM trainers, which forms a specific shared learning forum.
c.) The horizon scanning and innovation pipeline workstreams have looked at technological changes and how these may impact on research and the use of research methods. This workstream has led to a review of 'Futures Methods' and has produced, amongst other outputs, a range of innovation guides and Futures briefs (including on topics such as Transhumanism, Virtual Realities and Immersive Technologies, Human Social Genomics). Our focus is on technical development but with a broader socio-technical lens. The goal is not to produce a theory of the future but to reflect on how particular changes might impact on the way social science is conducted (its methods, approaches, how theories are developed). In addition, several innovation fora have been run. These events focus on bringing together a diverse group of experts (thereof addressing senior TCB needs) to share knowledge and best practice on a topic with a view to achieving clear objectives producing one or more outputs, e.g. producing a position paper and development of training or online resources. They aims to stimulate imaginative developments in methods and to be responsive to new methodologically-related needs and opportunities as they arise within the social science community and beyond. Two recent examples are "Using Film for Interdisciplinary Collaborations in Epidemic Preparedness and Response, Technology and Contemporary Classical Music: Methodologies in Practice-Based Research" and "Creative and Authentic Co-Production Methods: Podcasts, Poems and Zines". We have recently launched two new focal areas (addressing engagement and TCB outputs): A new focal area on AI and one on socio-environmental sciences.
d.) The NCRM engagement function includes liaison and collaboration with key ESRC investments, as well as cross-sector, non-academic researchers, with a current focus on the voluntary sector. The aim of the engagement is to produce collaborative TCB outputs in strategic areas and areas identified of need. As part of the engagement work, NCRM continues to lead the DTP training network (DTP-TN), and has made significant progress in supporting the DTP, including for example high-level coordination, the identification of training needs using a number of mechanisms for sharing national training and development needs analyses in order to better serve the PgR community (e.g. use of the one-question approach), as well as addressing some of the key training needs identified. NCRM also continues to offer the NCRM training database that enables access and availability of TCB activities from other organizations (including from the DTPs) on the portal. We have also continued with the Data Resource Training Network (DRTN) and have identified and inducted new members to the DRTN, which is adding value and capacity to the scale and impact of the individual services involved in the network. A series of online webinars have been produced, that were very well received and had high uptake. We have also been closely collaborating with ADR-UK to develop and facilitate the delivery of training on administrative data, focussing on UK flagship administrative datasets. A new 'health research and social sciences' MSIG has been established as an output of the co-production retreat hosted in May 2023 (led by two PHDs and an ECR and is producing an online series of six methods panels as well as an in-person meet-up. All of these activities, will continue to raise the profile of both the ESRC investments and NCRM, reduce silo working, will lead to increased capacity and will create a community of social science scholars. Particular emphasis is on the conversion of engagement and collaboration activities into a wide range of training and capacity building outputs.
Exploitation Route We are highlighting the following three workstreams: a.) the training programme offered, b.) pedagogy outputs, c.) outcomes from the Changing Research Practices project.
a.) The activities above all feed regularly into the strategy of the NCRM training programme and are therefore of direct benefit to NCRM and its TCB output. The training of a wide range of users will have very likely led and will continue to lead to an uptake in the methods learned. An analysis of data collected in 2022 by NCRM and again repeated very recently as part of an impact survey, which covered a range of questions around the likely use and impact after a training course/event, indicated, for example, around 70% of course participants to anticipate using the methods learned in their thesis, over 50% in a peer reviewed journal article, over 20% in research for the public domain, just under 20% in research, government or public policy, just above 10% in research for a non-academic organisation and just above 10% for the supervision of students. A further impact study evidenced actual uptake of a research method 1-2 years after a training activity took place.
b.) The Handbook of Teaching and Learning Social Research Methods (edited by Nind, for publication by Edward Elgar) and the results of the pedagogy systematic review will further build the pedagogic culture and good practice in the teaching of research methods in NCRM, nationally and internationally. They will each provide a useful reference point for methods teachers/trainers to appreciate both common and innovative practices that have a sound evidence or theory base. NCRM is exploring further new ways to communicate key messages from this work to those in a position to put them into practice. A network of interested methods teachers who research their own and others' pedagogy has been formed to expand the evidence base for others to use.
c.) The outcomes of the Changing Research Practices in the Context of Covid-19 project have led to recommendations on training priorities for NCRM and other organizations as well as the continued provision of workshop spaces for researchers to share knowledge and mutual support. The reports, publications and resources are all accessible via the NCRM website which provides a central access point for researchers in the climate of ongoing uncertainties. Already doctoral and other researchers have found what they need to be able to continue their studies and the material is informing doctoral education as evidenced in our impact assessment review.
Sectors Digital/Communication/Information Technologies (including Software)

Education

Government

Democracy and Justice
URL http://www.ncrm.ac.uk
 
Description NCRM has achieved a wide range of short- and long-term impacts. We outline evidence of impact from the the following selected workstreams: a.) the training programme, b.) pedagogy, c.) the Changing Research Practices project, d.) our networks and methodolocial special interest groups. a.) NCRM's strategy for the impact of its training and capacity building programme prioritises the generation of crosscutting, interconnected impacts, which include: (i) direct changes to knowledge and skills, practice, curricula, pedagogy and employment; (ii) multiplier capacity building effects with new skills being passed onto others in workplaces, educational settings or other communities, and (iii) outcomes and outputs that are of economic, societal, policy and cultural benefit. The overall aim is for the cumulative effect of these instrumental, conceptual, capacity building and academic impacts to demonstrate the reach, significance and indispensable nature of NCRM as a training and capacity building infrastructure, that is uniquely positioned to create synergies and impacts. Since 2020, NCRM has collected and analysed formal and informal evidence of impact through multiple methods. Quantitative and qualitative data have been collected from course and event evaluation forms, user follow-up surveys, social media, testimonials, case-study collation, in total two impact surveys, interviews and conversations with users and stakeholders. The focus has not just been on evidencing the achievements of the Centre's activities, but on how these impacts interconnect and build on each other and contribute to NCRM's methodological capacity. Our further impact evaluation continues to indicate that NCRM's training and capacity building have had immediate / short-term impacts, and longer-term, complex impacts with economic, societal, economic, policy and cultural benefits, which build incrementally over time. Since 2020, NCRM has experienced high demand for its courses, resources, and events, and has received consistently positive feedback from its stakeholders, with stakeholders indicating that they consider the Centre's activities to have been of high quality. Our evaluation forms ask participants a range of questions around the likely use and impact of learning acquired from a training course/event. Analysis of data indicate that a very high proportion have used or anticipate to use the methods they had learnt in different environment and for different types of output. In responses to a follow-up (impact) survey, we have seen evidence of how participants of NCRM events and courses are applying their new skills to their research and/or in their teaching and supervision of others. We have also heard from former NCRM grant holders, who have continued to be involved with the centre during our current phase. They have provided training on methods they have developed. We have also collected examples of longer-term impacts from their ongoing work with NCRM. NCRM has recently (March 2024) launched its 20th year anniversary impact price, specifically asking for evidence on long-term impact. We expect to be able to evidence more long-term impacts as a result of this impact prize and additional evidence gathering. b.) The pedagogy systematic review findings have already been used to design and deliver three Train the Trainer courses on using active, experiential and student-centred approaches when teaching methods online. Contributors to the Handbook have produced bite-size material and are further circulating key pedagogic messages to practitioners and have joined an 'in conversation' event at the 2023 Research Methods e-Festival. c.) We have experienced high levels of engagement with the Changing Research Practices project and researchers have told us that they are using the outputs, both directly, and in their support of their students and mentees. New informal support networks were formed during the workshops. We have used the findings to inform the work of research students and researchers through invitations to speak to staff groups, student groups, special interest groups, and at conferences. We have also developed a suite of resources based on the findings to help guide researchers in their search for relevant, high-quality material to guide their decision-making in response to the crisis. d.) our NCRM networks and methodological special interest groups continue to florish and expand and all have contributed to TCB activities. They have resulted in high-level coordination, sharing and exchange and learning from each other across a wide range of organisations and ESRC investments.
First Year Of Impact 2020
Sector Digital/Communication/Information Technologies (including Software),Education,Healthcare,Government, Democracy and Justice
Impact Types Societal

Policy & public services
 
Description Professor David Martin's membership of the UK Statistics Authority Methodological Advisory Panel - Census
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
URL https://uksa.statisticsauthority.gov.uk/the-authority-board/committees/national-statisticians-adviso...
 
Description Changing Research Practice: Undertaking social science research in the context of Covid-19
Amount £57,921 (GBP)
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 06/2020 
End 06/2022
 
Description The impact of Covid-19 on survey data collection methods in the social sciences : Survey Data Collection Network (SDC-Net)
Amount £64,133 (GBP)
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 11/2021 
End 10/2022
 
Description Data Analytics Partnership 
Organisation Innovate UK
Country United Kingdom 
Sector Public 
PI Contribution Innovate UK approached NCRM for support in developing their research methods skills. Undertaking a more engaged approach to understanding their training needs, we found that a simple training course was not needed. The 'problem' they had was more nuanced. Over several online sessions, Mark Elliot and Ali Hanbury facilitated action learning style conversations, asking curious questions and understanding their practice needs. This led to a co-designed approach to identifying, exploring and responding to a data problem. Subsequently Mark and Ali developed an offer for local authority data analysts, which brought into scope the needs identified by the Local Government Association which identified this group as a strategic priority to target, and they went on to work with two local authorities - Oldham and Cheshire West and Chester - who both responded to the bespoke call we created with the LGA. Over several online sessions we explored the 'data problem' and provided consultancy and expertise to ways that the data problem could be solved and improvements made.
Collaborator Contribution See above.
Impact See above. An internal report was drafted to review the success and limitations of this programme.
Start Year 2021
 
Description Data Analytics Partnership 
Organisation Local Government Association
Country United Kingdom 
Sector Public 
PI Contribution Innovate UK approached NCRM for support in developing their research methods skills. Undertaking a more engaged approach to understanding their training needs, we found that a simple training course was not needed. The 'problem' they had was more nuanced. Over several online sessions, Mark Elliot and Ali Hanbury facilitated action learning style conversations, asking curious questions and understanding their practice needs. This led to a co-designed approach to identifying, exploring and responding to a data problem. Subsequently Mark and Ali developed an offer for local authority data analysts, which brought into scope the needs identified by the Local Government Association which identified this group as a strategic priority to target, and they went on to work with two local authorities - Oldham and Cheshire West and Chester - who both responded to the bespoke call we created with the LGA. Over several online sessions we explored the 'data problem' and provided consultancy and expertise to ways that the data problem could be solved and improvements made.
Collaborator Contribution See above.
Impact See above. An internal report was drafted to review the success and limitations of this programme.
Start Year 2021
 
Description Contribution to the ESRC Festival of Social Sciences 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Sophie Woodward (and Sheehan, J) participated in the ESRC Festival of Social Sciences. Their contribution : 'I had one of those: objects based methods and memories' was delivered to an audience of 60 members of the Public at Chorlton Library in November, 2021.
Year(s) Of Engagement Activity 2021
 
Description Health and Social Sciences co-production retreat 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact This two-day event took place in May 2023. It was the result of a previous interface report produced by Robert Meckin and Mark Elliot. A mix of 24 collaborators attended, made up of final year PhD students and their supervisors. They were recruited form the ESRC DTP health and wellbeing pathways from across the UK. The group engaged with NCRM's Mark Elliot, Ali Hanbury and Dianna Smith.

The collaborators were requested to create a poster about their methods to kick off the event. They also attended an online information session before the in-person event. They engaged in various activities including a collaborative session on Ketso.

Towards the end of the retreat, collaborators were encouraged to identify possible future activity to develop methodological practice across the group. A small group of three collaborators subsequently applied to launch a new methodological special interest group which has been approved and is now being led by Dianna Smith. Activities are scheduled to begin in March 2024.
Year(s) Of Engagement Activity 2023
 
Description Panel presentation for Academy of International Business Research Methods SIG 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Melanie Nind contributed to a Panel presentation 'Doing Research During Covid-19: A meta view' for the Academy of International Business Research Methods SIG to an audience of approximately 250 people from 31 countries on 3rd December 2020.
Year(s) Of Engagement Activity 2020
 
Description Workshop at the World Conference on Qualitative Research (WCQR) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Sadhbh O'Dwyer and Cristina Azaola (with Sarah Lewthwaite and Melanie Nind) lead an in person workshop on 'Inquiry and Diary Circles: The power of the circle in qualitative research methods' at the 8th World Conference on Qualitative Research (WCQR) on 23rd January 2024.
Year(s) Of Engagement Activity 2024
 
Description Workshop at the World Conference on Qualitative Research (WCQR) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Robert Meckin lead a hybrid workshop on 'Exploring Uncertainty in Qualitative Research' at the 7th World Conference on Qualitative Research (WCQR) on 26th January 2023.
Year(s) Of Engagement Activity 2023
URL https://wcqr.ludomedia.org/wcqr-book-of-abstracts/
 
Description Workshop at the World Conference on Qualitative Research (WCQR) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Melanie Nind (with Sarah Lewthwaite and Abigail Croydon) lead an online workshop on 'Video Stimulated Recall Reflection and Dialogue Method' at the 8th World Conference on Qualitative Research (WCQR) on 24th January 2024.
Year(s) Of Engagement Activity 2024
 
Description Workshop for the World Conference on Qualitative Research (WCQR) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Melanie Nind (with Rob Meckin and Andrew Coverdale) lead an online workshop on 'Changing Research Practices for Covid-19' at the 5th World Conference on Qualitative Research (WCQR) on 21st January 2021.
Year(s) Of Engagement Activity 2021
URL https://wcqr.ludomedia.org/previous-editions/
 
Description Workshop for the World Conference on Qualitative Research (WCQR) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Melanie Nind (with Lewthwaite, S.) lead an online workshop on 'Teaching Qualitative Research in Covid-19 Times' at the 6th World Conference on Qualitative Research (WCQR) on 28th January 2022.
Year(s) Of Engagement Activity 2022
URL https://wcqr.ludomedia.org/