ORA (Round 5) - Facilitating Self-Regulated Learning with Personalized Scaffolds on Student's own Regulation Activities

Lead Research Organisation: University of Edinburgh
Department Name: Sch of Informatics

Abstract

The focus of education is increasingly set on students' ability to regulate their own learning within technology-enhanced learning environments (TELs). Prior research has shown that self-regulated learning (SRL) leads to better learning performance but students often experience difficulties to adequately self-regulate their learning. Instructional scaffolds are a successful method to help learners and consequently improve learning outcomes. However, scaffolds are often standardized and do not adapt to the individual learning process. Learning analytics and machine learning offer an approach to better understand SRL-processes during learning. Yet, current approaches lack validity or require extensive analysis after the learning process. This research collaboration will research how to advance support given to students by i) improving unobtrusive data collection and machine learning techniques to gain better measurement and understanding of SRL-processes and ii) using these new insights to facilitate student's SRL by providing personalized scaffolds. We will reach this goal by investigating and improving trace data in exploratory studies (exploratory study1 and study 2) and using the insight gained from these studies to develop and test personalized scaffolds based on individual learning processes in laboratory (experimental study 3 and study 4) and a subsequent field study (field study 5). Our joint expertise in the fields of self-regulated learning and learning analytics provide superior opportunities to develop and test more powerful adaptive educational technologies.

Planned Impact

The results will be disseminated in both academic and non-academic avenues. The academic avenues will primarily include publications in major journals (e.g., Learning and Instruction, Journal of Educational Psychology, and Computers & Education) and conferences (EARLI, AREA, LAK). The impact for non-research beneficiaries will be made with the following stakeholders: teaching staff in the partner institutions, industry and practice-based communities, educational policy makers, and general public. The project team has an excellent track record in working non-academic stakeholders such policy makers (SURF, QAA, and Jisc), educational technology industry (Blackboard, Moodle, and Desire2Learn), and professional organizations (e.g., SoLAR and EARLI). The collaboration will ensure that impact is initiated from the outset through increased awareness of opportunities from the start of the research life-cycle, rather than waiting until the end of project when models have been developed. The project team will remain flexible in terms of approach and incorporate insights from stakeholders and end-users throughout the project phases. Specifically, direct impacts are expected in the following directions: i) Informing teaching practice in higher education and ii) Impact on learning analytics and online learning platforms. We shall organize a series of engagement events - one right after the project kickoff in the first month with key stakeholders, and after 12 months an education-wide and industry event will be organised to bring the various stakeholders together to share and co-construct initial research findings. Two follow-up knowledge exchange events will be held together in parallel with the partner meetings. The project will also have a website which will host the project results, policy briefs, relevant podcasts, and videos demonstrating how the project results be translated to practice and policy. All the resources will be available in English, Dutch, and German.

Publications

10 25 50
 
Description The project conducted four laboratory studies and two field studies involving both undergraduate and postgraduate students, to complete learning tasks with technology. The technology was designed in such what to collect digital footprints (i.e., trace data) about learning related activities of the studies. The previous research predominantly used data about clinks made by the students. Our study included also data about mouse movements, key strokes, and eye movement. The results that a combination of these data streams offered much more accurate accounts of the ways how students learn than the use of data about student clicks only.

We improved the measurement of self-regulated learning (SRL) by embedding instrumentation tools in a learning environment and validating the measurement of SRL with these instrumentation tools using think aloud.

We proposed a novel validation approach to evaluating and investigating the validity of trace-based SRL measurement protocols. In this validation approach includes a theory-driven perspective and a data-driven perspective, using both empirical evidence from think aloud data and rationale from our theoretical framework of SRL to construct an improved process library. More importantly, our results showed that measuring and interpreting SRL from trace data is a very promising method which deserves more attention and practical application.

We proposed an analytic method for detecting SRL strategies from theoretically supported SRL processes and applied the method to a dataset collected from a multi-source writing task. The results of the study related to learning outcomes and effects of internal and external conditions on strategy use have a potential to inform teaching and learning practice.

We collected trace data that multi-source writers generated in the experimental writing session, and analysed how measures extracted from those data relate to essay performance. Path model results supported our hypotheses, i.e., engagement in metacognitive monitoring and control benefits the quality of a written product. Moreover, our results indicate that the essay scoring algorithm applied in our project can be used to estimate essay scores with considerable reliability and thus identify writers at risk of producing low quality drafts, i.e., writers who may need to increase their metacognitive processing to succeed in a multi-source writing task.
Exploitation Route The results have direct implications for educational technology industry. The results can be used to inform next-generation of digital learning environments that can collect more granular data about students' learning process, and thus, enhance student learning experience and learning performance by custom-made and personalised support for each and every learner.
Sectors Digital/Communication/Information Technologies (including Software),Education

 
Description FLoRA team 
Organisation Radboud University Nijmegen
Country Netherlands 
Sector Academic/University 
PI Contribution - Developed a methodology for analysis of multichannel data about self-regulated learning including log data, keystrokes and mouse moves, and eye tracking data; - Performed analysis of the data collected in Studies 1 and 2 of project; - Created a prototype of learning environment that is used by the partners for conducting laboratory studies planned in year 2 of the project; and - Hosted the kick-off meeting of the project in Edinburgh in Feb 2019.
Collaborator Contribution - Designed and conducted studies 1 and 2 of the project that include collection of about self-regulated learning including log data, keystrokes and mouse moves, eye tracking data, student performance, and think aloud data; - Analyzed the think aloud data and the data about student performance; and - Coordinated the collaboration between the partners
Impact This is an inter-disciplinary collaboration involving our partners with the background in educational psychology and our team with the background in computer science. The publications resulted from this collaboration: Joep van der Graaf, Inge Molenaar, Kia Puay Lim, Yizhou Fan, Katharina Engelmann, Dragan Gaševic and Maria Bannert (2020). Facilitating Self-Regulated Learning with Personalized Scaffolds on Student's own Regulation Activities. Companion Proceedings of the 10th International Conference on Learning Analytics and Knowledge Kia Puay Lim, Joep van der Graaf, Yizhou Fan, Katharina Engelmann, Inge Molenaar, Maria Bannert, Dragan Gaševic, and Johanna Moore (2020). Triangulating Multimodal Data to Measure Self-Regulated Learning. Companion Proceedings of the 10th International Conference on Learning Analytics and Knowledge Yizhou Fan, Kia Puay Lim, Joep van der Graaf, Katharina Engelmann, Inge Molenaar, Maria Bannert, Johanna Moore, Dragan Gaševic (2020). Measuring Micro-Level Self-Regulated Learning Processes with Enhanced Log Data and Eye Tracking Data. Companion Proceedings of the 10th International Conference on Learning Analytics and Knowledge
Start Year 2019
 
Description FLoRA team 
Organisation Technical University of Munich
Country Germany 
Sector Academic/University 
PI Contribution - Developed a methodology for analysis of multichannel data about self-regulated learning including log data, keystrokes and mouse moves, and eye tracking data; - Performed analysis of the data collected in Studies 1 and 2 of project; - Created a prototype of learning environment that is used by the partners for conducting laboratory studies planned in year 2 of the project; and - Hosted the kick-off meeting of the project in Edinburgh in Feb 2019.
Collaborator Contribution - Designed and conducted studies 1 and 2 of the project that include collection of about self-regulated learning including log data, keystrokes and mouse moves, eye tracking data, student performance, and think aloud data; - Analyzed the think aloud data and the data about student performance; and - Coordinated the collaboration between the partners
Impact This is an inter-disciplinary collaboration involving our partners with the background in educational psychology and our team with the background in computer science. The publications resulted from this collaboration: Joep van der Graaf, Inge Molenaar, Kia Puay Lim, Yizhou Fan, Katharina Engelmann, Dragan Gaševic and Maria Bannert (2020). Facilitating Self-Regulated Learning with Personalized Scaffolds on Student's own Regulation Activities. Companion Proceedings of the 10th International Conference on Learning Analytics and Knowledge Kia Puay Lim, Joep van der Graaf, Yizhou Fan, Katharina Engelmann, Inge Molenaar, Maria Bannert, Dragan Gaševic, and Johanna Moore (2020). Triangulating Multimodal Data to Measure Self-Regulated Learning. Companion Proceedings of the 10th International Conference on Learning Analytics and Knowledge Yizhou Fan, Kia Puay Lim, Joep van der Graaf, Katharina Engelmann, Inge Molenaar, Maria Bannert, Johanna Moore, Dragan Gaševic (2020). Measuring Micro-Level Self-Regulated Learning Processes with Enhanced Log Data and Eye Tracking Data. Companion Proceedings of the 10th International Conference on Learning Analytics and Knowledge
Start Year 2019