Patterns in Politics & Society: Promoting the Enrichment of Undergraduate Teaching with Quantitative Methods

Lead Research Organisation: University of Manchester
Department Name: Social Sciences

Abstract

The use of quantitative data is far too limited in social science teaching in the UK today. This project will help to fill this gap by providing a step-change in teaching methods for up to 12 Year 1-2 Undergraduate course units. We create then roll out innovations to a national level through a network, online resources and training, as well as training for trainers, in relation to these materials. The topics for the units are Sociology (e.g. how relationships form and what is classified as a marriage or civil registration) and Politics (e.g. egalitarian attitudes and how they change over time). We also support more advanced quantitative methods (QM) in Level 2-3 course units .

The creation of course unit elements has been initiated by our Lecturer partners, some of whom are named in the bid. Each element consists of one lecture (1-1.5 hours) and one tutorial class or formative exercise. We call this a 'course unit component' which might take up a week of study within a course (6 study hours). In developing these, we extend them in the area of innovative pedagogy with more explicit learning outcomes for quantitative methods. We argue that student roles vis-à-vis QM need to be built up via scaffolding (i.e. enriched gradually) from Reader through User to Maker of data; and finally also that the latest data sources and methods need to be disseminated in modernised ways. We aim to show teachers of undergraduate methods how to use a range of means of getting relevant data, revealing our sources and methods through footnotes and a brief resource pack for each course unit component.

We use peer review of our Manchester teachers to gather up expertise, leading to making video segments for online display and workshop use in promoting excellent teaching innovation in QM. We evaluate the project in terms of both student and staff impact.

The creators of the course unit elements initially are Univ. of Manchester lecturers, all named in the bid. Each gets two bespoke consultations and consults with a dedicated, pedagogically-experienced expert in secondary data analysis, who helps with refining a lecture or tutorial task that fits into a given syllabus. Next this material is tested in the classroom and evaluations are conducted. The resulting set of up to twelve topics' materials become the basis of a series of online web pages. A network of users is set up, incentivised by free access to a range of short courses for professional development. This network can include University and Higher Education social science and QM teachers in the UK. Among these users we expect to find a few trainers, i.e. those who provide Quantitative Methods Support services in various Higher Education settings in the UK. The whole network of teachers using QM has access to materials via online file management. The students in the local Univ. of Manchester classrooms also evaluate the materials and their learning experience.
The course materials also become the subject of four workshops:
1. Teaching undergraduates with quantitative data.
2. Making, using and interpreting enriched graphic images.
3. Making, using and interpreting tables: innovations for undergraduate social scientists.
4. Using secondary data the easy way: innovations for teachers of social science.

We will hold 16 workshops in all (some via web), developing the workshop topics to fit users' needs over 3 years.

Planned Impact

The immediate beneficiaries of the project are the social science undergraduate teachers who participate in our programme of activities and their students. For the teachers, the impact will primarily be in terms of the incorporation of a wider range of teaching materials, methodologies and supporting evidence in relation to their substantive module topic. A potential secondary impact on this group is an increased confidence in their ability to teach and draw upon quantitative methods, data sources and evidence, and also an increased likelihood in the use of such methods in their own research; as teaching quantitative methods should enable greater confidence in the use of such methodologies in research. For students, the project will develop their skills base by improving statistical literacy, and by increasing their ability to recognise difficulties of interpretation, based upon the underlying social and political complexity and social diversity. This is likely to have a large impact on their employability and future careers.

This project could also have a wider impact on the following groups:
1) The project will help the Royal Statistical Society's getstats program by increasing the statistical literacy of a large number of undergraduate social science students at Manchester (from year 1 onwards) and other universities which participate in the project.
2) Employers in the public and private sectors will benefit from a better skilled graduate workforce that is statistically literate, and familiar with quantitative analysis. One of the direct outcomes of the project is the students' ability to download and assess data from UK, EU and other international datasets, which could be of immense help in assessing evidence and making decisions for local authorities, government agencies, charities, and marketing organisations who constitute a large section of employers of Manchester social science undergraduates.
3) Directors of postgraduate taught and research programmes in the social sciences will benefit from a statistically literate graduate population interested in pursuing postgraduate education. The project could increase the range of methodologies adopted in Masters and PhD dissertations, and could also have an impact on the quality of Advanced Quantitative Methods studentship applications to the North-West ESRC Doctoral Training Centre.
4) The project will also have an impact on ESDS government and ESDS international (our partners in the delivery of the project) by making their materials and datasets more widely used. Wider dissemination of their activities and outputs is one of their key aims. At present, these groups have strong connections with social science postgraduates, but rather weaker links with social science undergraduates (with some notable exceptions among undergraduate economists who use ESDS international resources). The project embeds their resources, trainers and outputs directly into the social science undergraduate curriculum.
5) Members of learned Societies such as the British Sociological Association, Royal Statistical Society, and the Political Studies Association and participants in their annual conferences will be able to benefit from our presentations to these groups.
6) Legacy arrangements of training materials and courses with the Centre for Census and Survey Research (CCSR) will enable CCSR to be a world class trainer not only of quantitative methods but of quantitative methods teaching as well. Furthermore, such legacy arrangements with CCSR will provide ongoing training activities for future social science undergraduate teachers.

Publications

10 25 50
publication icon
Buckley J (2015) Embedding quantitative skills into the social science curriculum: case studies from Manchester in International Journal of Social Research Methodology

publication icon
Olsen W (2014) Comment The Usefulness of QCA under Realist Assumptions in Sociological Methodology

 
Description This grant is held jointly with a Curriculum Innovation grant. One publication summarises the pedagogical findings of our shared effort. This publication in International Journal of Sociological Research Methods (Buckley, et al., 2015) advises how "quantitative embedding" can successfully enhance teaching to social science. The second achievement is a data set combining student surveys and a questionnaire for teachers of sociology and politics. These questionnaire data are being deposited. The data have been analysed in relation to the obstacles students face in learning with quantitative data. We also have mixed methods data.

The second paper to be published from this project is specific to this grant and is targeted at the journal Sociology. Thus paper two, which is underway, presents the second set of findings from this project. Finally, we also produced online versions of our findings in the form of Briefing Papers numbered one and two. The first briefing paper on scaffolding relates closely to Paper One in IJSRM. The briefing paper was used in 11 workshops over three years to influence staff in their curriculum management. Briefing Paper two relates to Paper two (not yet submitted). This briefing paper on using data in dissertations covers all the teaching skills we had promised in our grant bid. It advertises NESSTAR and some even better alternatives for downloading data subsets. It explains the importance of dense graphics and it advocates interpretive complexity. It advocates that some dissertations could use data in a light fashion, and others in a heavier, statistical fashion. Our mailing list for the briefing papers and workshops consists of all participants as well as 350 lecturers in the north of England whom we particularly targeted. In conclusion, we have two papers, one data set and two Briefing Papers all aimed at training for trainers in sociology and politics.

Our website, http://www.socialsciences.manchester.ac.uk/research/research-centres-and-networks/essted// (accessed June 2015) gives access to the new teaching materials from our project. The briefing papers have been popular for over two years in their online PDF format. We created 8 videos of which 5 are of a professional standard. Each has a different world-class lecturer presenting advice on using data in classrooms. The topics range from employability to assessing students. Another three videos were provided in YouTube from 2013 onward, showing how to flip a lecture and how to innovate in sociology teaching. Lastly, the rich case studies of teaching material from the Curriculum Innovation part of our work are also provided in the website.

One barrier to holding workshops on pedagogy is that some busy academic staff feel they cannot afford the time for personal development or training. We find this short sighted. Our workshops included middle managers and early career researchers. We advised everyone to strategically take training in both online and face to face formats. In view of this we held workshops in London at the NatCen offices, in Edinburgh, and also twice in Cardiff University.
Exploitation Route Research on quantitative embedding in the undergraduate circle has been expanding. Our papers will contribute to an international discussion in which British undergraduate social scientists are perceived as noticeably behind American undergraduates in social science because of a quantitative deficit.

Besides researchers, teachers from universities across Britain have expressed interest in our findings. We attended the conferences of British Sociological Association and the Political Studies Association where we were received well. Wendy Olsen presented in plenary at the Research Methods festival and at Oxford University. Videos are still circulating of these events. Lastly, the Royal Statistical Society and British Academy are promoting our agenda. We also received a QStep grant (2015-2019).
Sectors Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Education,Financial Services, and Management Consultancy,Transport,Other

URL http://www.socialsciences.manchester.ac.uk/research/research-centres-and-networks/essted//
 
Description Students got more engaged with quantitative data in classrooms across six disciplines and three levels of undergraduate learning. We surveyed 165 students in year 1 and 130 in year 3 of the grant. Along with focus groups and interviews, the research data allowed us to feed back to academic staff in the University of Manchester about the student experience. Students experienced new teaching materials in 12 course units. Their teachers and teaching assistants Graduate Teaching Assistants widened their use of data and online evidence sources. Each year, methods of embedding were the subject of academic staff meetings focused on teaching quality. Specialised techniques were taught in a series of national workshops in Oxford, Cardiff, London, Edinburgh and Manchester. The academic publication in the IJSRM will reached both a national and international audience. With its online open access it already has 6 Scopus citations (see doi:10.1080/13645579.2015.1062624 ).
First Year Of Impact 2012
Sector Education
Impact Types Societal,Economic,Policy & public services

 
Description Methods Summer Schools: Influencing Content Internationally
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact My training elements have been exposed in a series of Youtube videos, workshops in UK and briefing papers online, after which the following impacts happened: 1. the content of the Berlin Summer School in Social Sciences was widened to include epistemological issues and qualitative comparative analysis, including a realist approach. I was invited for a funded stay in Berlin to manage a whole day of training in this premier international summer school of Social Science PhD students. 2. the content of the Sciences-Po Summer School on Comparative Social Science in 2013 included material from me on a funded visit to Biarritz where I stayed for 3 days presenting, with translation into French, on methodologies of mixed methods, a set of new ideas which widened the content of the school. The participants were from Social Sciences across over eight different country origins and were doing PhDs from top international universities, having won bursaries from the French government to attend this one-week event. 3. I was able to construct a new Summer School lasting one week in manchester under the Methods@Manchester umbrella in 2014. This summer school was focused on Integrated Mixed Methods Research and will now be an annual event. The training will be annual from now on. It attracted policy-engaged PhD students nearly finished with their PhDs coming from Britain and Netherlands.
URL http://www.methods.manchester.ac.uk/events/summerschool/
 
Description International Mobility Partnership: Innovation in Global Labour Research Using Deep Linkage and Mixed Methods
Amount £25,000 (GBP)
Funding ID PM140147 
Organisation The British Academy 
Sector Academic/University
Country United Kingdom
Start 09/2014 
End 11/2017
 
Title Facebook group on Integrated Mixed Methods Network 
Description The analysis of tensions in society is enhanced through mixed methods. Instead of seeing contradictory evidence as an epistemological problem, we see it as potentially showing real conflicts, ambiguity and/or cognitive clashes of view. This website allows a wide range of viewers and members to examine evidence of social tension around labour market social norms. See https://www.facebook.com/groups/438437119631157/ 
Type Of Material Model of mechanisms or symptoms - human 
Year Produced 2014 
Provided To Others? Yes  
Impact We are being partnered by several impressive people including consultants with experience of Resilience training in schools, British Council youth initiatives, and other partnerships between youth and older people. All aim to improve youth entry into the labour market. These partners want to work closely on an international project. 
URL https://www.facebook.com/groups/438437119631157/
 
Title DATASET ON STUDENT EVALUATION OF QUANTITATIVE METHODS IN TEACHING IN UG PROGRAMMES IN SOCIOLOGY AND POLITICS 
Description 1. Purpose and coverage of the survey The ESRC funded two projects, as shown, which enabled us to create this data set. The other materials produced during these projects are found at URL http://www.socialsciences.manchester.ac.uk/research/research-centres-and-networks/essted//, accessed Dec. 2015. 2012-2015 Economic and Social Research Council (ESRC), Researcher Development Initiative, "Patterns in Politics & Society: Promoting the Enrichment of Undergraduate Teaching with Quantitative Methods", Co-PIs: Dr Wendy Olsen, Dr Mark Brown, Dr Brian Heaphy, Professor Tarani Chandola, Professor Ian Plewis, Dr Jackie Carter, GBP 80K funded by ESRC, £100K FEC. Grant Number ES/J011665/1. 2012-2015 Economic and Social Research Council (ESRC), "Teaching quantitative methods in disciplinary context: integrating quantitative method and evidence into the Social Science undergraduate curriculum", Co-PIs: Dr Wendy Olsen, Dr Mark Brown, Dr Brian Heaphy, Professor Tarani Chandola, Professor Ian Plewis, Dr Jackie Carter, 80K funded by ESRC, £100K FEC, Grant Number ES/J011622/1. The users of the course unit elements initially were Univ. of Manchester lecturers, all named in the bid, and their students. Each lecturer participated in two bespoke consultations and was provided with a dedicated pedagogically experienced expert in secondary data analysis who helped with refining a lecture or tutorial task that fits into a given syllabus. Next this material was tested in the classroom and evaluations were conducted using mixed methods, with both staff and students as respondent. The students in the local Univ. of Manchester classrooms also evaluate the materials and their learning experience using a survey. The student survey was conducted on University of Manchester campus 2012-2013 with followup in classrooms in 2014/5. The survey of students initially drew upon a self-selecting set of respondents, aiming at all classroom attenders, from 1st year - 3rd year students, giving us about their views of quantitative data and methods, and answering a battery of questions that would give us a measure of quantitative skills. These surveys, known as 'evaluation' surveys internally, were applied in BA Economics and BA Social Sciences classrooms, but drew from a variety of degrees, whose names were not included in the data. For example, there are Honour and ordinary degree students, students with single honours and dual (or joint degree) honours, and students in accounting and finance as well as those whose specialist subjects were in the six Social Science disciplines in Manchester at the time: politics, sociology, philosophy, economics, anthropology (and Social Statistics, which existed at the University of Manchester since 2009, but did not have any major or dual major students at the time). The degree programmes represented are listed in Figure 1 (below). The students who took part in the 2015 survey were second and third year students taking part in either a politics or a sociology class. From 183 raw respondents in Year 1 there were 325 in Year 3, giving 508 respondents overall. In year 1, survey respondents were in course units on racism, comparative politics and policy making. In year 3, a wider range of courses were used; their names are not listed here. The participating students were primarily recruited from partner modules, ie those where the lecturer had agreed to meet twice with Project staff in either the Curriculum Innovation project or the Researcher Development Initiative project, to plan an embedding of quantitative methods into the classroom experience of students. 
Type Of Material Database/Collection of data 
Provided To Others? No  
Impact The Q-Step grant held at University of Manchester is strongly influenced by the findings of this project. THe article published by Jen Buckley et al. (2015) helped make public to other Q-Step projects the findings on hor learning is enhanced for those in the PIPS and CI projects at the University of Manchester. References: Buckley, Jen, M. Brown, S. Thompson, W. Olsen, J. Carter (2015), "Embedding Quantitative Skills into the Social Science Curriculum: Case Studies From Manchester", International Journal of Social Research Methodology. DOI | doi:10.1080/13645579.2015.1062624. Brown, M.B., Olsen, W., and the ESSTED team, 2013. How to Manage Undergraduate Dissertations Which Use Empirical Survey Data in Sociology and Politics, . Briefing Paper Number 2 (Online document), URL https://patternsinpoliticsandsociety.files.wordpress.com/2014/03/manging-dissertations.pdf, Feb. 2015. Olsen, W.K., and the ESSTED team, 2013. Scaffolding to Using Quantitative Data in Sociology and Politics Classroom: Building Bridges, University of Manchester, Briefing Paper Number 1, (Online document), URL https://patternsinpoliticsandsociety.files.wordpress.com/2014/03/scaffolding-and-bridging.pdf). 
 
Description Workshop for Teachers of Quantitative Methods in Sociology and Social Policy 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Talk sparked questions and there was a lively discussion afterwards of whether UK standards for Undergraduate social science are lower than in the USA and Canada.

We await more on this in feedback from our future workshops.
Year(s) Of Engagement Activity 2014
URL http://www.sociology.ox.ac.uk/component/com_eventbooking/Itemid,444/event_id,6/view,event/