Autonomous vehicles for wheelchair use
Lead Research Organisation:
Imperial College London
Department Name: Computing
Abstract
To integrate current methods for autonomous vehicles into an indoor-use wheelchair for paraplegic users. This base system is going to be controlled by a gaze-based interface that decodes high-level action intentions (where a user would like to go) to take him there.
Organisations
Publications
Balamurugan Sadaiappan
(2022)
Applications of machine learning in chemical and biological Oceanography
Gas P
(2020)
In Silico Study on Tumor-Size-Dependent Thermal Profiles inside an Anthropomorphic Female Breast Phantom Subjected to Multi-Dipole Antenna Array.
in International journal of molecular sciences
James Wells
(2021)
Challenges and recommendations for magnetic hyperthermia characterization measurements
in International Journal of Hyperthermia
Mahendran Subramanian
(2021)
Gaze-contingent decoding of human navigation intention on an autonomous wheelchair platform
Mahendran Subramanian
(2022)
Eye Tracking Background, Methods, and Applications
Mahendran Subramanian
(2022)
Zero-UI control of an autonomous wheelchair through cognitive level intention decoding
Mahendran Subramanian
(2019)
Cognitive Wheelchair: A Personal Mobility Platform.
Miaskowski A
(2019)
Numerical Model for Magnetic Fluid Hyperthermia in a Realistic Breast Phantom: Calorimetric Calibration and Treatment Planning.
in International journal of molecular sciences
Miaskowski A
(2017)
Specific absorption rate parameter model in magnetic hyperthermia
Miaskowski A
(2019)
An in vivo coil setup for AC magnetic field-mediated magnetic nanoparticle heating experiments.
in Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
Raymond L
(2018)
Natural Gaze Data-Driven Wheelchair
Sadaiappan B
(2020)
Metagenomic 16S rDNA amplicon data of microbial diversity and its predicted metabolic functions in the Southern Ocean (Antarctic).
in Data in brief
Sadaiappan B
(2021)
Meta-analysis cum machine learning approaches address the structure and biogeochemical potential of marine copepod associated bacteriobiomes.
in Scientific reports
Subramanian K
(2020)
Bioconversion of chitin and concomitant production of chitinase and N-acetylglucosamine by novel Achromobacter xylosoxidans isolated from shrimp waste disposal area.
in Scientific reports
Subramanian M
(2019)
A.Eye Drive: Gaze-based semi-autonomous wheelchair interface.
in Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
Subramanian M
(2019)
Practical bioinstrumentation developments for AC magnetic field-mediated magnetic nanoparticle heating applications
in Applied Physics A
Subramanian M
(2017)
Oscillating Magnet Array-Based Nanomagnetic Gene Transfection: A Valuable Tool for Molecular Neurobiology Studies.
in Nanomaterials (Basel, Switzerland)
Subramanian M
(2019)
Remote manipulation of magnetic nanoparticles using magnetic field gradient to promote cancer cell death
in Applied Physics A
Ságová I
(2020)
Heart failure as a manifestation of acromegaly.
in Vnitrni lekarstvi
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/N509486/1 | 01/10/2016 | 31/03/2022 | |||
| 1979819 | Studentship | EP/N509486/1 | 01/10/2017 | 31/07/2021 | Mahendran Subramanian |
| Description | People with severe lower-limb movement impairments rely heavily on devices such as wheelchairs for their daily activities. Electric wheelchair controllers such as joysticks, sip and puff, or head-mount gyroscope-based devices are effective when giving full control of the wheelchair to the user, but require continuous attention, and engage them on a low-level of control. We propose the use of natural gaze patterns as an implicit interface for high-level wheelchair command, while low- level control, including navigation and obstacle avoidance is done autonomously. In this manner, the system allows subjects to navigate their wheelchairs in dynamic environments, without the need to interact with a controller, engaging them instead on a higher cognitive level. Wearable eye-trackers are used to record eye-movements, with online object detection and labelling, to identify the object of interest, and regions of gaze interest within the object boundaries. Resulting data from contextual gaze fixation duration and gaze length are used to build an intention decoding classifier. This, combined with autonomous navigation algorithms on the wheelchair platform, allow users to accomplish their navigation requirements, as indicated naturally by their gaze, in an autonomous manner. |
| Exploitation Route | Cognitive technologies towards smart vehicles are evolving, but specific to wheelchairs are often overlooked. High- level gaze informatics can be complemented with context-aware algorithms for natural interaction with the environment through robotic devices or autonomous vehicles. Herein, we harvest the eye movements to enhance the cognitive abilities of an autonomous wheelchair platform as eye movements are correlated with motor intention and act as a precursor to movement. First, we developed tools to estimate 3D gaze point and to recognise the object at the wheelchair user's 'Area Of Interest' for high-level intention decoding. This 3D eye-tracking tool with practical accuracy levels to analyse natural human behaviour during physical interaction with the environment was used to obtain human intention in autonomous systems for natural, user-centric interaction with the environment. With high-level intention decoding capability, such cognitive wheelchair systems can not only perform autonomous mobility tasks but also function in a contextual, semantic and continuous spectrum of tasks. |
| Sectors | Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software),Education,Electronics,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Manufacturing, including Industrial Biotechology,Transport |
| Description | Nokia corporation is interested in commercializing our following inventions. They have already paid for the patent applications, and we have signed agreements with them to take these technologies forward 1. Mahendran Subramanian, Aldo Faisal (2017). Breath detection extraction from motion data. (Nokia Corporation - Imperial Business School - School of Bioengineering Project) EP18213990.7 2. Mahendran Subramanian, Aldo Faisal (2017). Movement indication for a scene based on video data. (Nokia Corporation - Imperial Business School - School of Bioengineering Project) EP18213981.6 Numerous investors have also shown interest in investing and commercializing our 3D gaze-based wheelchair technology - Apart from EPSRC my PhD topic also received some external funding over the course based on the outcome from commercial companies like Toyota, BMW, Nokia, which shows the real-world impact of the topic. We have already started the process of developing our technology into a commercial project. |
| First Year Of Impact | 2017 |
| Sector | Digital/Communication/Information Technologies (including Software),Healthcare |
| Impact Types | Societal,Economic,Policy & public services |
| Description | Demo of my AI wheelchair at the Imperial Global Science Policy Forum - AI Showcase |
| Geographic Reach | Multiple continents/international |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Impact | I performed a Demo on Autonomous (AI) Wheelchair (resulting from my PhD- 3d gaze based semi-autonoous wheelchair) Autonomous Urban Mobility for the elderly and the severly disabled. The demo was perfromed to politicians/ policymakers from throughout the globe including UK politicians/ policymakers. |
| Description | Bay France - BMW - TUM grant to attend Summer School on Intelligent Vehicles |
| Amount | £600 (GBP) |
| Organisation | Bayerische Motoren Werke (BMW) |
| Sector | Academic/University |
| Country | Germany |
| Start | 06/2018 |
| End | 08/2018 |
| Description | Faculty of Engineering - Travel grant |
| Amount | £1,000 (GBP) |
| Organisation | Imperial College London |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 06/2018 |
| End | 08/2018 |
| Description | Nanotherics Research and Development Fund |
| Amount | £150,000 (GBP) |
| Organisation | nanoTherics Ltd |
| Sector | Private |
| Country | United Kingdom |
| Start | 11/2010 |
| End | 09/2019 |
| Description | Nokia Corporation - Imperial Business School initiative |
| Amount | £40,000 (GBP) |
| Organisation | Nokia |
| Sector | Private |
| Country | Global |
| Start | 07/2017 |
| End | 05/2021 |
| Description | Tech 4 Good - Inclusive Design Award (Finalist) |
| Amount | £710 (GBP) |
| Organisation | BT Group |
| Sector | Private |
| Country | United Kingdom |
| Start | 07/2019 |
| Description | Toyota - Mobility Unlimited Challenge Discovery Award |
| Amount | $50,000 (USD) |
| Organisation | Toyota Motor Corporation |
| Sector | Private |
| Country | Japan |
| Start | 04/2018 |
| End | 09/2018 |
| Title | Alternating magnetic field mediated drug release testing setup |
| Description | Alternating magnetic field mediated drug release testing setup (finished, published in Applied Physics A and undergone external evaluation in IIT, Italy): Novelty - Controlled drug delivery for magnetic fluid cancer hyperthermia experiments. Heat dissipation during magnetization reversal processes in magnetic nanoparticles (MNP), upon exposure to alternating magnetic fields (AMF), has been extensively studied in relation to applications in magnetic fluid hyperthermia (MFH). The paper demonstra |
| IP Reference | |
| Protection | Trade Mark |
| Year Protection Granted | 2017 |
| Licensed | Commercial In Confidence |
| Impact | Device used for in vivo testing of controlled drug delivery for magnetic fluid cancer hyperthermia experiments. Currently sold by nanotherics Ltd (UK based company that funded this research) and bought globally (£ 3500-4000 GBP). |
| Title | Method and apparatus to enable real time non-contact in vitro temperature measurement for nanoparticle calorimetry in time varying magnetic field |
| Description | An apparatus for maintaining organic cells in vitro whilst simultaneously applying a time-varying radio frequency magnetic field and taking non-contact, real-time temperature or calorimetry measurements. The organic cells are cultured with magnetically susceptible particles or nanoparticles so that hyperthermia takes place within the magnetic field. The apparatus comprises: an incubation chamber; a magnetic field source, e.g. a coil or gapped core, connected to an alternating magnetic field generator; a cell culture vessel preferably with a biocompatible base of thickness 50µm or less; temperature sensors, preferably either a fibre optic or an infrared sensor, whose tips are attached/focussed to the underside of the cell culture vessel respectively. The magnetic field source may be cooled e.g. using liquid nitrogen or water. The magnetically susceptible particles may comprise gold. The temperature sensor may be of single channel or multichannel, may contain a computer interface and software to measure and record the real-time temperature change. |
| IP Reference | GB2551743 |
| Protection | Patent application published |
| Year Protection Granted | 2018 |
| Licensed | Commercial In Confidence |
| Impact | Nanomagnet actuating air core inductor with time lapse imaging setup (finished, published in International Journal of Hyperthermia and currently patent pending): Novelty - Controlled drug delivery, magnetic fluid cancer hyperthermia experiments can be monitored real time, when the cells/ tissue samples are exposed to alternating magnetic field. Currently sold by nanotherics Ltd (UK based company that funded this research) and bought globally (£ 3500-4000 GBP), including the top 20 (world rank) U |
| Title | Movement detection from a scene based on video data. |
| Description | Mahendran Subramanian, Aldo Faisal (2020). Movement detection from a scene based on video data. (Nokia Corporation - Imperial Business School - School of Bioengineering Project) EP 3 671 630 A1. |
| IP Reference | WO2020126459; EP 3 671 630 A1; US20220028082A1 |
| Protection | Patent application published |
| Year Protection Granted | 2020 |
| Licensed | Commercial In Confidence |
| Impact | The research was funded by Nokia Corporation |
| Title | Movement indication for a scene . |
| Description | Mahendran Subramanian, Aldo Faisal (2020). Movement indication for a scene (Nokia Corporation - Imperial Business School - School of Bioengineering Project) EP 3 671 631 A1. |
| IP Reference | WO2020126452; EP 3 671 631 A1; US20220036567A1 |
| Protection | Patent application published |
| Year Protection Granted | 2020 |
| Licensed | Commercial In Confidence |
| Impact | The research was funded by the Nokia Corporation |
| Title | Self-driving AI wheelchair system with gaze based intention decoding capability |
| Description | Self-driving AI wheelchair system (PhD project): Novelty - Eye gaze-controlled, artificially intelligent semi-autonomous wheelchair capable of decoding human intentions from their eye movements. Won Discovery award - $50,000 USD. The prize was announced as part of Mobility Unlimited Challenge Discovery Award, which is funded by Toyota Mobility Foundation and run by Nesta's Challenge Prize Centre. Featured on the Imperial College London homepage. Have built an Autonomous Wheelchair with a Novel eye gaze interface and the submission came on the Top 10 for Top 5 - $500,000 USD prize - Toyota Mobility Final Challenge: Finalist Tech4Good award (Inclusive design award) - BT, UK. |
| Type | Therapeutic Intervention - Medical Devices |
| Current Stage Of Development | Initial development |
| Year Development Stage Completed | 2019 |
| Development Status | Under active development/distribution |
| Impact | Design and development were published in IEEE EMBC 2019 and IEEE IROS 2019 - Towards Cognitive Vehicles Workshop This technology will help improve independent mobility and quality of life for the severely disabled. |
| Company Name | FARADAY-FLEMING LABORATORY LTD |
| Description | We have set up this company in order to commercialize our Biomedical Engineering products. Currently, we have attracted £710, i.e. the prize money we got for being a Tech4good UK - Inclusive design award finalist for my (Mahendran Subramanian) PhD topic - 3D gaze-based semi-autonomous wheelchair. We are currently working towards attracting more money from external funding sources, including venture capital investments. |
| Year Established | 2017 |
| Impact | We intend to commercialize the final product that I have developed based on my PhD research: 3D gaze-based semi-autonomous wheelchair through this company. The technology is also capable of gaze-based intention decoding for goal definition. |
| Description | 2 conference articles accepted in IEEE EMBS Conference on Neural Engineering 2021 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | 2 conference articles accepted in IEEE EMBS Conference on Neural Engineering 2021. 1. Gaze-contingent decoding of human navigation intention on an autonomous wheelchair platform (https://arxiv.org/abs/2103.03072) 2. Neuromechanics-based Deep Reinforcement Learning of Neurostimulation Control in FES cycling (https://arxiv.org/abs/2103.03057) |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://arxiv.org/abs/2103.03072 |
| Description | Article about my invention in CNN World |
| Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | I was interviewed by CNN and my invention 3D gaze based semi-autonomous wheelchair with intention decoding capability featured in an article discussing 'life-changing tech for people with disabilities' as well as 'High-tech innovations for people with disabilities'. https://edition.cnn.com/2020/11/18/world/gallery/cybathlon-2020-spc-intl/index.html |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://edition.cnn.com/2020/11/18/world/cybathlon-2020-tech-disabilities-spc-intl/index.html |
| Description | Attended IEEE International Conference on Robotics and Automation (ICRA) 2020 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Attended IEEE International Conference on Robotics and Automation (ICRA) 2020 |
| Year(s) Of Engagement Activity | 2020 |
| Description | BMW Summer School - Emotional Aware Assistance for Intelligent Vehicles. Technical University Munich, Germany |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Industry/Business |
| Results and Impact | Mahendran Subramanian, Noyan Songur, Darrell Adjei, Aldo Faisal (2018). A.Eye-Drive: Autonomous navigation for the ageing and physically impaired. BMW Summer School - Emotional Aware Assistance for Intelligent Vehicles. Technical University Munich, Germany. Noyan Songur+, Mahendran Subramanian+, Aldo Faisal (2018). A.Eye-Drive: Gaze-based intention decoding for human-in-the-loop autonomy. BMW Summer School - Emotional Aware Assistance for Intelligent Vehicles. Technical University Munich, Germany. [Outcome: Best Poster Award ] |
| Year(s) Of Engagement Activity | 2018 |
| Description | Conference - Understanding images in biological and computer vision. Royal Society of London, UK. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Mahendran Subramanian, Lou-Ann Raymond, Margherita Piccini, Pavel Orlov, Aldo Faisal (2018) Decoding natural eye movements during driving to steer a wheelchair. Conference - Understanding images in biological and computer vision. Royal Society of London, UK. |
| Year(s) Of Engagement Activity | 2017 |
| Description | Demonstration of our 3D gaze based AI wheelchair at The AI Summit, TechXLR8 2019, London, UK |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | TechXLR8 encompasses 7 of the biggest technology conferences under one roof including The AI Summit, Blockchain for Business, IoT World Europe, Cloud & DevOps World, Startup Elevate, and AR & VR World. TechXLR8 is London Tech Week's flagship event, Europe's leading tech festival that welcomes 55,000 attendees across a week filled with creativity, talent and innovation. This 2- day tech festival ignites transformation through connecting 15,000 attendees with industry leaders, iconic brands and innovative products. I demonstrated my invention, i.e. 3D gaze-based semi-autonomous wheelchair to more than 100 event participants who were interested in our technology. We also did a lot of Networking and Interacting directly to the whole community. I got the opportunity to be amongst the latest cutting-edge technology, hear from 600+ tech guru speakers and experience impressive demo zones which will help elevate my AI knowledge first hand. I was also interviewed by TV channels at this event. |
| Year(s) Of Engagement Activity | 2019 |
| Description | Done numerous demonstrations of our 3D gaze based AI wheelchair |
| Form Of Engagement Activity | Participation in an open day or visit at my research institution |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Done numerous demonstrations of our 3D gaze based AI wheelchair to visitors at our research facility, TV channels incl. BBC, ITV, Channel 4, and on conferences held within Imperial College London on robotics and AI |
| Year(s) Of Engagement Activity | 2017,2018,2019,2020 |
| Description | Google Poster Competition |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Postgraduate students |
| Results and Impact | Mahendran Subramanian, Darrell Adjei, Aldo Faisal (2018). 3D Gaze based Semi-Autonomous Wheelchair. Google Poster Competition. Department of Computing, Imperial College, UK. [Outcome - Best Poster Award]. |
| Year(s) Of Engagement Activity | 2018 |
| Description | Interview for BBC |
| Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | I was interviewed by BBC for my invention 3D gaze based semi-autonomous with intention decoding capability. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.youtube.com/watch?v=Yw0TOVmx8tw&list=PL63lwGZ_8vsnmWuxFBTL6NulPaYZXj0zv&t=480s |
| Description | Interview for Sports n Spikes |
| Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | I was interviewed by sports n spokes magazine on my invention 3D gaze based semi-autonomous wheelchair with intention decoding capability. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://vimeo.com/479307767 |
| Description | Patient house visit for Demonstration and evaluation of my invention 3D gaze based semi-autonomous wheelchair - Co creation |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Patients, carers and/or patient groups |
| Results and Impact | Drove my invention (3D gaze-based semi-autonomous wheelchair) to severely disabled patient homes (Peterborough and Oxfordshire) to perform demonstration and evaluation of our technology along with their participation, i.e. Co-creation. This helped me to understand more of their needs and how our technology can be modified or improved to well suit its purpose. Their enthusiasm and results ensured that the technology is capable of improving patient quality of life as well as provide independency to the severely disabled and the elderly. |
| Year(s) Of Engagement Activity | 2018 |
| Description | Poster presentation IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019); Workshop - Towards Cognitive Vehicles Workshop. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | Mahendran Subramanian, Suhyung Park, Pavel Orlov, Aldo Faisal (2019). Cognitive Wheelchair: A Personal Mobility Platform. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019); Workshop - Towards Cognitive Vehicles Workshop. |
| Year(s) Of Engagement Activity | 2019 |
| Description | Poster presentation on Conference - Interactive eye gaze |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Mahendran Subramanian, Lou-Ann Raymond, Margherita Piccini, Pavel Orlov, Aldo Faisal (2018) Natural Gaze data driven wheelchair. Conference - Interactive eye gaze. University College London, UK. |
| Year(s) Of Engagement Activity | 2017 |