Objective-based Iterative Learning Control for Robotics and Rehabilitation

Lead Research Organisation: University of Southampton
Department Name: Electronics and Computer Science

Abstract

The purpose of this project is to develop and significantly enhance an exciting new technology that has been shown to help stroke patients regain arm function. This technology is based on using Electrical Stimulation to assist subjects in performing arm movements which they cannot otherwise manage. By making muscles work, Electrical Stimulation solves the problem that people have when they try to re-learn skills after having a stroke. This problem is that re-learning skills takes practice, and this requires feedback which you can't get when you are unable to move your arm at all. The way in which people re-learn skills after a stroke is exactly the same process as you do when you learn to play tennis. You become better at it, because new nerve connections have been made within your brain and spinal cord. Not only do you need to practice, but you also need feedback of your performance so that you can correct your movement.When Electrical Stimulation is applied to muscles, electrical impulses travel along the nerves in much the same way as the electrical impulses from your brain. If stimulation is carefully controlled, a useful movement can be made. The re-learning process works better if the person is attempting the movement themselves. Recent innovative research has exploited this fact by combining a person's own effort with just enough Electrical Stimulation to achieve the intended movement. This research involved subjects performing horizontal reaching movements with Electrical Stimulation applied to their triceps. Their task was to track a spot of light as it moved slowly in front of them, and a technique called 'Iterative Learning' was used to decide what stimulation to apply to help them perform the tracking task accurately. By carefully varying this stimulation to best assist the subject, this research has established that Electrical Stimulation is able to help people re-learn movement after a stroke.The aim of this project is to maximise the therapeutic benefit of Electrical Stimulation during treatment. Foundations will also be laid that are necessary for the development of an inexpensive system that can be used in patient's own homes to increase access to this innovative technology. The way in which this will be done is to develop 'Iterative Learning' into a much more powerful and flexible tool for governing the stimulation. It will then be used to help people perform far more natural movements such as picking up a bottle and pouring from it, pressing a series of buttons, and turning a handle. Since these sort of tasks are important for day to day living, the technology then directly helps patients re-learn the tasks that are most useful to them. Another advantage is that the 'Iterative Learning' will also be able to respond to the wishes of the physiotherapist who supervises the treatment. If they decide that the movement the subject is trying to do is not ideal, they will have the freedom to change the way in which the stimulation helps the subject perform the movement. Furthermore, the simple way in which the tasks are presented to people means that very little equipment is needed, as there is no trajectory to display, no form of constraint to the movement, nor any robotic assistance used. This therefore removes a substantial obstacle preventing the technology from moving from the lab and into patient's homes.Although stroke rehabilitation is the application focussed on, the added flexibility given to 'Iterative Learning' will also benefit many processes that are found in industry. Examples of these include robots which perform the same operation over and over again in production lines. This flexibility will give the way in which the repeated task is performed the freedom to vary in order to maximise efficiency, respond to changes in the task, and satisfy desired constraints. This will all be achieved whilst still maintaining a high degree of accuracy.

Publications

10 25 50
publication icon
Freeman C (2011) Model and experience-based initial input construction for iterative learning control in International Journal of Adaptive Control and Signal Processing

publication icon
Freeman C (2011) Phase-lead iterative learning control algorithms for functional electrical stimulation-based stroke rehabilitation in Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering

publication icon
Freeman C (2011) Iterative Learning Control for Multiple Point-to-Point Tracking Application in IEEE Transactions on Control Systems Technology

 
Description The purpose of this work was to develop and test novel control algorithms which broaden the scope and effectiveness of upper arm FES-based stroke rehabilitation, as well as being suitable for a broad range of other applications. The project has succeeded in these aims, producing 21 journal papers (13 published/in press, and 8 under review in top quality academic journals), as well as 28 conference publications in top quality international conferences (25 published/accepted, 3 under review). The short time-scale means that many of the core publications are still under review, and also that more publications are still being written up for submission. A summary of these papers (see list below) is:



- Novel Iterative learning control theory (point-to-point, hard and soft constraints): [3,8,13,14,19]

- Overview of ILC development for FES and robotics rehabilitation [5,11]

- Development of FES ILC controllers: [6,15,16]

- Clinical Results: [17,20,21]

- Other ILC/ repetitive control theory that supports current/future rehabilitation work: [1,3,7,9,12,18]

- Muscle identification: [4,10]



A key change in project goals was, due to the technical and hardware side of the project progressing so well, the proposed validation tests were expanded into a full clinical trial. This has a much larger impact, especially on the medical/rehabilitation community, and also is a significant step in obtaining follow on funding for ultimate commercialisation and transference into patients' homes. The extra workload of a clinical trial (involving 5 participants each receiving 18 treatment sessions) was undertaken by the clinical RF (supported by Dr Freeman's clinical colleagues at Southampton). The results of the study were extremely encouraging and have been submitted to leading journals and conferences (a no-cost project extension was requested and obtained to write up this work).



In terms of the training of students, the grant has supported four PhD Students, as well as a great number of undergraduate students who have had projects involving the system. For example, four intern students, two supported by Nuffield Foundation bursaries (URB/38332 & URB/38483), one from Eindhoven under the Erasmus agreement, and another receiving money from the School of Electronics and Computer science at Southampton. Two of these have gone on to pursue PhDs, and a third is considering this option. The four PhD students have directly worked on the project, and hence have benefited from project resources, with their work feeding into the project deliverables. Two of these, Fengmin Le and Muhammad Alsubaie, obtained their PhDs in early 2011. Fengmin Le worked on identification of muscle models which were used in the clinical trials (producing 2 journal papers on this topic). Muhammad Alsubaie worked on ILC theory which will be used to broaden the tasks trained (producing 3 journal papers). Daisy Tong commenced her PhD prior to the grant starting and is due to submit in October 2011. Her studies focused on developing control methods for application of stimulation, and fed directly into the clinical trials. Anna Soska commenced her PhD in October 2010 and is studying stimulation of the hand and wrist, building on the outcomes of the project and using technology developed during it.

This first grant has been instrumental in establishing the career of Dr Freeman as an independent researcher capable of world-leading research of high impact.



Journal Papers



Published:

[1] Wang, L., Freeman, C. T., Rogers, E. and Owens, D. H. (2010). Experimentally Validated Continuous-time Repetitive Control of Non- minimum Phase Plants with a Prescribed Degree of Stability. Control Engineering Practice, 18 (10). pp. 1158-1165.

[2] Freeman, C.T., Tan., Y. (2011). Introduction to special issue: Special Issue on Iterative Learning Control & Repetitive Control. International Journal of Control. 84 (7), pp. 1193-1195.

[3] Tan, Y., Xu, J. X. Norrlof, M and Freeman, C. T. (2011). On reference governor in iterative learning control for dynamic systems with input saturation. Automatica. doi:10.1016/j.automatica.2011.08.024

[4] Le, F., Markovsky, I., Freeman, C. T. and Rogers, E. (2010). Identification of Electrically Stimulated Muscle after Stroke. Control Engineering Practice, 18. pp. 396-407.

[5] Rogers, E., Owens, D., Werner, H., Freeman, C. T., Lewin, P., Kirchhoff, S., Lichtenberg, G. and Schmit, C. (2010). Norm Optimal Iterative Learning Control with Application to Problems in Accelerator based Free Electron Lasers and Rehabilitation Robotics. European Journal of Control, 16 (5). pp. 497-524.

[6] Freeman, C. T., Tong, D., Meadmore, K., Cai, Z., Rogers, E., Hughes, A. M. and Burridge, J. H. (2011). Phase-lead Iterative Learning Control Algorithms for Functional Electrical Stimulation based Stroke Rehabilitation. Proceedings of the Institution of Mechanical Engineers - Part I: Journal of Systems & Control Engineering. 225 (6) pp. 850-859.

[7] Freeman, C., Alsubaie, M., Cai, Z., Rogers, E. and Lewin, P. (2010). Model and Experience-based initial input construction for iterative learning control. International Journal of Adaptive Control and Signal Processing , 25 (5). pp. 430-447.

[8] Freeman, C., Cai, Z., Rogers, E. and Lewin, P. (2010). Iterative Learning Control for Multiple Point-to-Point Tracking Application. IEEE Transactions on Control Systems Technology, 19 (3). pp. 590-600. ISSN 1063-6536.

[9] Freeman, C. T., Alsubaie, M., Cai, Z., Lewin, P. and Rogers, E. (2011). Initial Input Selection for Iterative Learning Control. ASME Journal of Dynamic Systems, Measurement and Control. 133 (5). 054504.



In Press:

[10] Le, F., Markovsky, I., Freeman, C. T. and Rogers, E. (2011) Recursive Identification of Hammerstein Systems with application to Electrically Stimulated Muscle. Control Engineering Practice. In press.

[11] Freeman, C. T., Rogers, E., Hughes, A.-M., Burridge, J. H. and K. L. Meadmore. (2011). Iterative Learning Control in Healthcare: Electrical Stimulation and Robotic-assisted Upper Limb Stroke Rehabilitation. IEEE Control Systems Magazine. In press.

[12] Wang, L., Chai, S., Rogers, E., and C. T. Freeman. (2011). Multivariable Repetitive-predictive Controllers using Frequency Decomposition. IEEE Transactions on Control Systems Technology. In press.

Submitted journal papers:

[13] Owens, D. H., Freeman, C. T. and T. V. Dinh. (2011). Norm Optimal Iterative Learning Control with Intermediate Point Weighting - Theory, Algorithms and Experimental Evaluation. Automatica. (Submitted)

[14] Freeman, C. T. and Y. Tan. (2011). Iterative Learning Control with Mixed Constraints for Point-to-Point Tracking. IEEE Transactions on Control Systems Technology. (Submitted).

[15] Freeman, C., Tong, D., Meadmore, K. L., Hughes, A. M., Rogers, E. and Burridge, J. H. (2011). Iterative learning control of arm and shoulder movements for FES-based rehabilitation. Control Engineering Practice. (Submitted)

[16] Freeman, C. T., Tong, D., Meadmore, K. L., Hughes, A. M., Rogers, E. and Burridge, J. H. (2011). FES based Rehabilitation of the Upper Limb Using Input/Output Linearization and ILC. IEEE Transactions on Control Systems Technology. (Submitted)

[17] Meadmore, K. L., Hughes, A. M., Freeman, C. T., Z. Cai, D. Tong, Burridge, J. H. and Rogers, E. (2011) Iterative learning mediated FES and 3D robotics improves motor control in chronic stroke. Journal of NeuroEngineering and Rehabilitation. (Submitted)

[18] Freeman, C. T., Muhammad, A., Cai, Z., Lewin, P. L. and Rogers, E. (2011). A Common Setting for the Design of Iterative Learning and Repetitive Controllers with Experimental Verification. International Journal of Adaptive Control and Signal Processing. (Submitted)

[19] Freeman, C. T. (2011) Constrained Point-to-Point Iterative Learning Control with Experimental Verification. Control Engineering Practice. (Submitted)

[20] Hughes, A.M., Donovan-Hall, M., Burridge, J., Freeman, C. T., Chappell, P., Dibb, B. (2010) Stroke participants' perceptions of robotic therapy. Disability and Rehabilitation: Assistive Technology. (Submitted).

Professional Journal papers:

[21] Meadmore, K., Cai, Z., Tong, D., Hughes, A. M., Freeman, C., Rogers, E. and Burridge, J. (2011) SAIL: A 3D rehabilitation system to improve arm function following stroke. Progress in Neurology and Psychiatry, 15 (2). pp. 6-10.



Conference Publications

Published:

Leyland, L-A., Liversedge, S., Meadmore, K., Hughes, A. M., Freeman, C., Burridge, J. and Rogers, E. (2011). Patterns of Eye Movements during Cancellation Tasks in Stroke Patients Exhibiting Hemispatial Neglect. European Conference on Eye Movements.

Meadmore, K., Hughes, A. M., Freeman, C. T., Tong, D., Rogers, E. and Burridge, J. (2011). Iterative Learning Mediated FES in Stroke Rehabilitation. 16th Annual International FES Society Conference. São Paulo, Brazil.

K. L. Meadmore, A.-M. Hughes, C. T. Freeman, V. Benson, D. Tong, J. H. Burridge, E. Rogers. User feedback driving change in the design of new technologies. 9th Annual SoNG Meeting, September 2011, Southampton.

Cai, Z., Tong, D., Freeman, C. and Rogers, E. (2011). Application of Newton-method Based ILC to 3D Stroke Rehabilitation. In: 18th IFAC World Congress, August 28 - September 2, Milano, Italy.

Cai, Z., Tong, D., Meadmore, K., Freeman, C. T., Hughes, A. M., Rogers, E. and Burridge, J. (2011). Design & Control of a 3D Stroke Rehabilitation Platform. In: International Conference on Rehabilitation Robotics, 29th June, Zurich, Switzerland.

Meadmore, K., Cai, Z., Tong, D., Hughes, A. M., Freeman, C. T., Rogers, E. and Burridge, J. (2011). Upper Limb Stroke Rehabilitation: The Effectiveness of Stimulation Assistance through Iterative Learning (SAIL). In: International Conference on Rehabilitation Robotics, 29th June, Zurich, Switzerland.

Burridge, J. H., Hughes, A. M., Freeman, C. T., Rogers, E. and Tedesco-Triccas, L. (2011). Using technology to optimize recovery in upper limb stroke rehabilitation. In: Festival of International Conferences on Caregiving, Disability, Aging and Technology - FICCDAT 2011, June 5-8, Toronto, Canada.

Le, F., Markovsky, I., Freeman, C. T. and Rogers, E. (2011) Online Identification of Electrically Stimulated Muscle Models. IEEE American Control Conference, June 29 - July 1, 2011, San Francisco, California, USA.

Le, F., Markovsky, I., Freeman, C. and Rogers, E. (2011) Recursive Identification of Hammerstein Structure. 18th World Congress of International Federation of Automatic Control, Milano, Italy, August 28 - September 2, 2011.

Freeman, C. T, Tan, Y. (2011) Point-to-Point Iterative Learning Control with Mixed Constraints. IEEE American Control Conference, June 29 - July 1, 2011, San Francisco, California, USA.

Freeman, C. T. (2011) Constrained Point-to-Point Iterative Learning Control. 18th World Congress of International Federation of Automatic Control, Milano, Italy, August 28 - September 2, 2011.

Meadmore, K. L., Hughes, A.-M., Cai, Z., Tong, D., Freeman, C. T., Burridge, J. H., Rogers, E. (2010). 3D Stoke Rehabilitation using Learning Control Mediated by FES. 8th Annual SoNG Meeting, September 2010, Southampton.

Freeman, C., Lewin, P., Rogers, E. and Owens, D. (2010). Iterative Learning Control of the Redundant Upper Limb for Rehabilitation. In: 2010 IEEE American Control Conference, June 30 - July 2, 2010, Baltimore, Maryland.

Cai, Z., Tong, D., Freeman, C., Rogers, E., Hughes, A. M. and Burridge, J. (2010). ILC for 3D Rehabilitation Robotics. In: UK-Japan Network EPSRC Postgraduate Workshop on Human Adaptive Mechatronics.

Alsubaie, M., Freeman, C. T., Cai, Z., Rogers, E. and Lewin, P. (2010). Iterative Learning and Repetitive Controller Design Via Duality with Experimental Verification. In: 49th IEEE Conference on Decision and Control, December 15-17, 2010, Atlanta, Georgia, USA.

Tropea, P., Freeman, C. T., Hughes, A. M., Burridge, J. and Micera, S. (2010). Modifications of upper limb muscle synergies in post-stroke patients during rehabilitation based on functional electrical stimulation. In: The XVIII Congress of the International Society of Electrophysiology and Kinesiology, 16-19th June, Aalborg, Denmark.

Burridge, J., Freeman, C. T., Hughes, A. M., Rogers, E., Lewin, P. and Chappell, P. (2010). Relationship between changes in tracking performance and timing and amplitude of biceps and triceps EMG following training in a planar arm robot in a sample of people with post-stroke hemiplegia. In: The XVIII Congress of the International Society of Electrophysiology and Kinesiology, 16-19th June, Aalborg, Denmark.

Alsubaie, M. A., Cai, Z., Freeman, C. T., Lewin, P. and Rogers, E. (2010). Selecting the Initial Input for Iterative Learning Control: Algorithms with Experimental Verification. UKACC International conference on CONTROL 2010. Coventry University, Coventry, UK.

Freeman, C. T., Cai, Z., Lewin, P. and Rogers, E. (2009). Iterative Learning Control For Multiple Point-to-Point Tracking. In: 48th IEEE Conference on Decision and Control, December 2009, Shanghai, China.

Hughes, A. M., Freeman, C., Burridge, J., Chappell, P., Lewin, P. and Rogers, E. (2009). Clinical effectiveness and patient perceptions of an ILC mediated by ES system using a robotic workstation. In: XVIII European Stroke Conference, 26-29 May 2009, Stockholm, Sweden.

Alsubaie, M. A., Freeman, C., Cai, Z., Lewin, P. and Rogers, E. (2009). ILC Initial Input Selection with Experimental Verification. In: Symposium on Learning Control at IEEE CDC 2009, December 14-15, 2009, Shanghai.

Le, F., Markovsky, I., Freeman, C. T. and E. Rogers. (2009). Identification of Electrically Stimulated Muscle after Stroke. In: European Control Conference 2009 - ECC'09, 23-26 August, 2009, Budapest, Hungary.

Freeman, C. T., Hughes, A. M., Burridge, J., Chappell, P., Lewin, P. and Rogers, E. (2009). An Upper Limb Model Using FES for Stroke Rehabilitation. In: European Control Conference 2009 - ECC'09, 23-26 August, 2009, Budapest, Hungary.

Freeman, C.T., Cai, Z., Lewin, P. L. and E. Rogers. (2009). Objective-Driven ILC for Point-to-Point Movement Tasks, IEEE American Control Conference.

Wang, L., Freeman, C. T., Chai, S. and Rogers, E. (2011). Multivariable Repetitive-predictive Control of a Robot Arm with Experimental Results. In: 18th IFAC World Congress, Milano, Italy, August 28 - September 2, 2011.

Wang, L., Chai, S., Freeman, C. T. and Rogers, E. (2010). Structure Selection for Multivariable Repetitive-predictive Controllers. In: 49th IEEE Conference on Decision and Control. pp. 6973-6978.

Submitted conference papers:

Freeman, C. T., Tong, D., Meadmore, K. L., Hughes, A.-M., Rogers, E. and J. Burridge. (2012). FES based Rehabilitation of the Upper Limb using Input/Output Linearization and ILC. American Control Conference. (Submitted).

Dinh, T. V. Freeman, C. T. and P. L. Lewin. (2012). Development of a multivariable test facility for the evaluation of iterative learning controllers. American Control Conference. (Submitted).

Freeman, C. T. and T. D. Van. (2012). Convergence and Robustness of a Point-to-Point Iterative Learning Control Algorithm. IEEE Conference on Decision and Control. (Submitted).

Book Section

Le, F., Freeman, C. T., Markovsky, I. and Rogers, E. (2011) Progress and open questions in the identification of electrically stimulated human muscle for stroke rehabilitation. In: System Identification, Environment Modelling and Control System Design L .Wang and H. Garnier eds , Springer. (In Press)
Exploitation Route The research is currently being further developed, and three further clinical trials have been run (each with 5 patients over 18 treatment sessions each). The goal is to produce an effective home-based stroke technology system within the next 5 years. A major success for this project has been to receive follow-on funding in the form of an EPSRC grant (EP/I01909X/1) which will use the FES controllers, test platforms and theory to develop a platform for rehabilitation of the whole upper limb (including the hand and wrist), during realistic functional tasks (using no explicit reference trajectory) . This is an extremely ambitious step which heavily relies on the algorithm development, and supporting experimental and clinical tests, undertaken in this grant. At the same time, a Wessex Medical Research Innovation Fund grant of £20k ('Open Hand for Stroke') is enabling more clinical tests to be undertaken using the same equipment to gain fuller clinical evidence for patient functional improvement.
Sectors Digital/Communication/Information Technologies (including Software),Healthcare

URL http://ctfreeman.com/
 
Description The purpose of this work was to develop and test novel control algorithms which broaden the scope and effectiveness of upper arm FES-based stroke rehabilitation, as well as being suitable for a broad range of other applications. The project has succeeded in these aims, producing 21 journal papers (13 published/in press, and 8 under review in top quality academic journals), as well as 28 conference publications in top quality international conferences (25 published/accepted, 3 under review). The short time-scale means that many of the core publications are still under review, and also that more publications are still being written up for submission. A summary of these papers (see list below) is: • Novel Iterative learning control theory (point-to-point, hard and soft constraints): [3,8,13,14,19] • Overview of ILC development for FES and robotics rehabilitation [5,11] • Development of FES ILC controllers: [6,15,16] • Clinical Results: [17,20,21] • Other ILC/ repetitive control theory that supports current/future rehabilitation work: [1,3,7,9,12,18] • Muscle identification: [4,10] A key change in project goals was, due to the technical and hardware side of the project progressing so well, the proposed validation tests were expanded into a full clinical trial. This has a much larger impact, especially on the medical/rehabilitation community, and also is a significant step in obtaining follow on funding for ultimate commercialisation and transference into patients' homes. The extra workload of a clinical trial (involving 5 participants each receiving 18 treatment sessions) was undertaken by the clinical RF (supported by Dr Freeman's clinical colleagues at Southampton). The results of the study were extremely encouraging and have been submitted to leading journals and conferences (a no-cost project extension was requested and obtained to write up this work). A major success for this project has been to receive follow-on funding in the form of a £464k EPSRC grant (EP/I01909X/1) which will use the FES controllers, test platforms and theory to develop a platform for rehabilitation of the whole upper limb (including the hand and wrist), during realistic functional tasks (using no explicit reference trajectory) . This is an extremely ambitious step which heavily relies on the algorithm development, and supporting experimental and clinical tests, undertaken in this grant. At the same time, a Wessex Medical Research Innovation Fund grant of £20k ('Open Hand for Stroke') is enabling more clinical tests to be undertaken using the same equipment to gain fuller clinical evidence for patient functional improvement. In terms of the training of students, the grant has supported four PhD Students, as well as a great number of undergraduate students who have had projects involving the system. For example, four intern students, two supported by Nuffield Foundation bursaries (URB/38332 & URB/38483), one from Eindhoven under the Erasmus agreement, and another receiving money from the School of Electronics and Computer science at Southampton. Two of these have gone on to pursue PhDs, and a third is considering this option. The four PhD students have directly worked on the project, and hence have benefited from project resources, with their work feeding into the project deliverables. Two of these, Fengmin Le and Muhammad Alsubaie, obtained their PhDs in early 2011. Fengmin Le worked on identification of muscle models which were used in the clinical trials (producing 2 journal papers on this topic). Muhammad Alsubaie worked on ILC theory which will be used to broaden the tasks trained (producing 3 journal papers). Daisy Tong commenced her PhD prior to the grant starting and is due to submit in October 2011. Her studies focused on developing control methods for application of stimulation, and fed directly into the clinical trials. Anna Soska commenced her PhD in October 2010 and is studying stimulation of the hand and wrist, building on the outcomes of the project and using technology developed during it. This first grant has been instrumental in establishing the career of Dr Freeman as an independent researcher capable of world-leading research of high impact.
First Year Of Impact 2008
Sector Digital/Communication/Information Technologies (including Software),Healthcare,Manufacturing, including Industrial Biotechology
Impact Types Cultural,Societal

 
Description EPSRC
Amount £464,231 (GBP)
Funding ID EP/I01909X/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 03/2011 
End 04/2014
 
Description EPSRC 'Bridging the Gap' - Pilot Scheme Award
Amount £20,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 09/2012 
End 03/2013
 
Description Wessex Medical Research
Amount £20,000 (GBP)
Organisation Wessex Medical Research 
Sector Charity/Non Profit
Country United Kingdom
Start 06/2012 
End 07/2013