From theory to practice: putting HCI frameworks to work

Lead Research Organisation: University of Nottingham
Department Name: School of Computer Science

Abstract

This fellowship will investigate and bridge the gaps between Human-Computer Interaction (HCI) theory and the practical realities of designing and building interactive technologies. It will produce new ways to weave HCI theory into practice, feed experiences from practice back to theory, and enhance methods of working together across disciplines and sectors.

Innovating, designing and studying interactions with digital technologies across a diverse and ever-widening range of activities and settings is a signature challenge of the field of Human-Computer Interaction (HCI). Much HCI seeks to address how best to design interactivity for a range of purposes and technologies, from improving usability, productivity and efficiency in user interface components, to developing collaborative systems for workplaces, to creating new art, cultural and leisure experiences with mobile devices, interactive displays and novel tangible user interfaces.

However, HCI has an uneasy relationship between this strong practical focus on the applied aspects of designing and building interactive technologies, and its theoretical innovations. Ill-defined, the wealth of HCI 'theory' that has developed since HCI's inception assumes a wide range of different forms: from 'big-T' theories such as Activity Theory, and models such as Fitt's Law, to frameworks and taxonomies, to broad 'sensitising concepts', recommendations, guidelines, principles, heuristics and rules-of-thumb. And yet, in spite of this proliferation of theoretical work, little is known about how such theoretical knowledge is practically manifest 'in the wild'; i.e., how it is actually applied in practice both within HCI research (beyond limited terms such as citation), as well as across industries developing interactive digital technologies which encounter HCI-related issues.

Of this theoretical HCI knowledge, it is frameworks-a way of packaging up HCI theory into disseminable and reusable parcels in order to inform subsequent design and application-which have arguably been most successful, at least within the HCI research community. However, like the wider problem affecting theory and its relation to practice, little is known about how these frameworks are actually applied in practice even within research communities. This fellowship will develop ways to bring this wealth of knowledge into practice and application more readily, easily, and accessibly for those working within HCI research, and in areas outside research that are impacted by HCI issues. Through exploring HCI frameworks in this way, the fellowship will address the wider issue of theory-practice connection.

This fellowship seeks to perform three key duties:

1. Deliver, via ethnographic work, detailed empirical understandings of the current role that HCI frameworks play in the design, development and evaluation of interactive experiences within research and industry. As part of this it will cross a range of domains in which interactive technology design work takes place, including digital technologies in culture and media (e.g., digital arts); the home (e.g., smart homes, energy monitoring); and workplaces (e.g., interaction / user experience design consultancy work).

2. Develop tools and techniques to support the practical application of HCI's theoretical products such that they can be operationalised for research and industry domains which increasingly encounter HCI-related issues, as well as within HCI research itself.

3. Support and strengthen a nascent interdisciplinary community concerned with the broader issue of bridging theory and practice. This will deliver long-term impact maintaining future research to study and refine the tools and techniques for applying HCI frameworks to a range of practices.

Planned Impact

Issues related to the design of user interfaces are increasingly seen as a key differentiator for many technology products and services generated by the global ICT sector (e.g., Apple's USP is largely presented as being the importance they place on user experience design in their products). Human-Computer Interaction (HCI) research investigates innovating, designing and studying user experience and interactions with digital technologies across a diverse and ever-widening range of activities and settings. However, in spite of the ever increasing industry recognition of user interface issues being core to the success of a range of digital technologies, coupled with the huge relevance of HCI research in this endeavour, a recent EPSRC review of research in the UK highlighted the need for greater exploitation and impact of HCI research. By bringing theory and practice together, the fellowship charts a path of delivery of broad impact to these core concerns for EPSRC. Given the importance placed by global industry players in the role of the user interface, and its relationship to the success of putting digital technologies to market, the potential for economic impact of the fellowship's work in bringing industry and academia design and research practices together is considerable, both at a national and international level.

Key to the fellowship's mechanisms of transfer is the dissemination of tools and techniques, and their integration into community building in the form of best practices. Through developing these tools and techniques for bridging theory and practical work, and via the fostering of a research-practice community concerned with sustaining and developing these best practices and tools and techniques, the fellowship can deliver long term impacts that extend beyond the lifetime of the funding period.

The fellowship addresses practical social economic purposes, through its dual engagement with research and practice, adding value to the existing innovative research agendas of HCI and maximising their future impact, producing benefit for academics, industry and funders. Specifically the fellowship's outputs can impact user experience / interaction design practices as found in industry and research. In these ways the fellowship is structured to deliver societal impact across sectors, but also can make a difference to the everyday user experiences of members of society in their interactions with technology by grounding interaction / user experience design practices in HCI theory. In this way the work will have economic and societal benefits, improving the design of interactive experiences across a range of target domains including technology for the home, workplace and cultural applications, and reducing the costs of research through gaining greater value through reuse.

The fellowship will also deliver impact through engaging with academic and industry partners. This will help foster the transfer of knowledge between practitioners and researchers through empirical study, practical intervention, community building and the development of tools and techniques to support these activities.

Finally, the fellowship will develop impact in terms of fostering research capability through the experience of the Fellow, postdoctoral student position, and building / strengthening collaborative links between host institutions and partners.

Publications

10 25 50
publication icon
Bødker S (2016) Nine questions for HCI researchers in the making in Interactions

publication icon
Fischer J (2015) Building a Birds Eye View

publication icon
Hornbæk K (2015) What to Study in HCI?

publication icon
Höök K (2015) Framing IxD knowledge in Interactions

publication icon
Reeves S (2015) Human-computer interaction as science in Aarhus Series on Human Centered Computing

publication icon
Reeves S (2017) Commentary: Usability in Vivo in Human-Computer Interaction

publication icon
Reeves S (2016) The Future as a Design Problem in Design Issues

publication icon
Reeves S (2015) I'd Hide You

publication icon
Reeves S (2019) How UX Practitioners Produce Findings in Usability Testing in ACM Transactions on Computer-Human Interaction

publication icon
Reeves S (2019) Talking about interaction* in International Journal of Human-Computer Studies

publication icon
Reeves S (2015) The Challenges of Using Biodata in Promotional Filmmaking in ACM Transactions on Computer-Human Interaction

publication icon
Reeves S (2015) Locating the 'big hole' in HCI research in Interactions

publication icon
Smith T (2019) "Off the beaten map": Navigating with digital maps on moorland in Transactions of the Institute of British Geographers

publication icon
Tekin B (2017) Ways of Spectating

publication icon
Velt R (2020) Translations and Boundaries in the Gap Between HCI Theory and Design Practice in ACM Transactions on Computer-Human Interaction

 
Description - Greater understanding of the peculiar disciplinary makeup of human-computer interaction as a field of research - what it can do and what it can't do, and what the role of theory, method, and practice is within academia and industry.
- Empirical understandings of what UX practitioners actually do in order to accomplish usability testing as an everyday activity that impacts product and service outcomes. This has opened up a significant set of future work expanding into broader investigations of testing practices in UX.
- Prototyped and trialled ways of practically connecting academic HCI research with UX and design practice.
- Supervised PhD research on Internet of Things design practice in industry which emerged from Fellowship research.
- Discovered how difficult it is to establish and maintain meaningful links between academic research in HCI and industry practice, and how to develop a win-win as part of this.
Exploitation Route Within HCI research, others need to take up the challenge of doing fundamental empirical work on unpacking UX practices and sharing this information with HCI communities who largely tend to have a simplistic picture.
Within UX and design practice, there are individual ways to proceed, including disseminating my own research in effective ways (which I've learned a lot about) but also considering how our undergraduate and masters courses relate to UX industry practices, particularly whether there are integrated courses to build that train 'bridge-building' graduates.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software)

URL http://www.cs.nott.ac.uk/~pszsr/#fellowship
 
Description This is reported elsewhere in the portfolio, individually attached to individual outputs. One of the main themes across this has been impact on practitioners from this research---specifically the dissemination and reported interest in outputs from our work. Update 2022: Since the pandemic impact has slowed, because of a reduced focus on external engagements and more on just getting basic academic duties done. However, I am starting to do practitioner-facing talks again and getting invitations etc. since the pandemic hit. I'll be reporting this next round of researchfish submission.
Sector Digital/Communication/Information Technologies (including Software)
Impact Types Economic

 
Description EPSRC CASE PhD funding
Amount £84,900 (GBP)
Funding ID Voucher number 18000109 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 10/2018 
End 09/2022
 
Description PhD funding from Microsoft Research PhD Scholarship 2018 Programme (minimum contribution)
Amount £28,300 (GBP)
Funding ID Voucher number 18000109 
Organisation Microsoft Research 
Sector Private
Country Global
Start 10/2018 
End 09/2022
 
Description Article in UXmatters online magazine for UX professionals 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I published a short article on the online UX professionals magazine / website "UXmatters". This article was part of an attempt to publicise a short survey for UX practitioners that I had constructed. It was also an attempt to start engaging with the UX community. The article was accepted and copyedited for the website (see link below).

The online publication resulted in a few new contacts who are interested in a similar area to my Fellowship. This is helping me build up a network of contacts / community that is interested in my Fellowship topic. The article also got shared on social media (e.g., Twitter, LinkedIn) a reasonable amount, generating further awareness of my Fellowship topic.
Year(s) Of Engagement Activity 2014
URL http://uxmatters.com/mt/archives/2014/08/what-is-the-relationship-between-hci-research-and-ux-practi...
 
Description Evaluating the outdoor learning experience 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Third sector organisations
Results and Impact This event was put on for outdoor education practitioners, curators, and cultural heritage practitioners. The intention was to provide them with a range of different methods and approaches they could use to evaluate their practices and the efficacy of them (according to a range of metrics). I was invited to present some introductory material on using video analysis techniques to study interaction with / around technology, since many of these practitioners use mobile devices and other technologies increasingly in their work and wanted to see how they might go about addressing the evaluation of them.

The workshop organisers distributed a workshop-related toolkit document as an output of the day: http://www.cardiff.ac.uk/__data/assets/pdf_file/0008/475676/Evaluating_The_Outdoor_Learning_Experience_Doc_EMAIL.pdf
Year(s) Of Engagement Activity 2009,2016
 
Description HCI-UX connections symposium (Nottingham) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This was a 2-day symposium / meeting I organised for November 2015, drawing together a combination of academic researchers who were connected in some way to the topic of my Fellowship along with a selection of industry UX practitioners (e.g., independent consultants, working with UX teams, etc.) who were familiar (to varying degrees) with academic HCI research and bridging the research-practice 'gap'. The purpose was to kick-start a network of people interested in the topic. This has led to:
- planning for a future meeting together, probably elsewhere in Europe this year;
- planning joint research funding (e.g., network grants, specifically we are looking into COST Actions) to enable collaborative working across the group;
- engagement activities with academia and industry through publishing joint magazine articles etc. (i.e., 'outreach' and communicating our collective aims / interests to others);
- sharing / connecting PhD proposals;
- connecting one of my PhD students with potential users of his thesis work.
Year(s) Of Engagement Activity 2015
 
Description Internal talk delivered to Bunnyfoot staff 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Industry/Business
Results and Impact We conducted a 'Brainfood' internal talk at Bunnyfoot (UX / design agency), in London, Nov 2017, titled: "Talking with Machines? Voice UI and conversation design". This involved about 10 staff working at Bunnyfoot. Led to our research being incorporated in a blogpost on their website: http://bunnyfoot.com/blog/2018/01/user-perception-vs-reality-voice-controlled-user-interfaces/
Year(s) Of Engagement Activity 2017
URL http://bunnyfoot.com/blog/2018/01/user-perception-vs-reality-voice-controlled-user-interfaces/
 
Description Internal talk delivered to Hallam Internet 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Industry/Business
Results and Impact On the basis of my Design Exchange talk in Nottingham, I was invited to give a further talk on a different topic to a local business, Hallam Internet (title: "Producing findings in user testing"). This talk described further research I have been doing beyond what was presented at the Design Exchange.

Currently discussing further involvement with Hallam as participants in further research on usability testing but also how they can be involved in the university etc.
Year(s) Of Engagement Activity 2018
 
Description Invited Studio Day at BBC Salford 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact We (myself, Joel Fischer, Martin Porcheon) conducted a BBC Studio Day at BBC UX&D (1 hr workshop, inc. 20 minutes talk), at BBC Salford, Aug 2017, title: "Talking with devices: Developing guidelines for conversational design". The workshop involved about 10-15 designers some of whom are actively engaging in voice user interface related projects at the BBC. This has led to further discussions with the BBC Design Research team about further results from our research
Year(s) Of Engagement Activity 2017
 
Description Invited talk at Service Lab London 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Invited talk at Service Lab London, a technology meetup group in London based around service design and UX design (talk title: "'Alexa This is not what we wanted': Talking with and around Voice agents").

I was invited based on my Interaction18 talk earlier in the year.
Year(s) Of Engagement Activity 2018
URL https://www.meetup.com/Service-Lab-London/events/255433024/
 
Description Invited talk at The Design Exchange 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Invited to do a talk at The Design Exchange, which is a technology design meetup group based in Nottingham (title: "'This is not what we wanted': Talking with voice assistants").

This invitation to speak was based on someone seeing my Interaction18 conference talk.

This talk led to a current engagement with Hallam Internet (local business) which we are currently exploring.
Year(s) Of Engagement Activity 2018
URL https://www.meetup.com/Design-Exchange-Nottingham-DxN/events/253128786/
 
Description Invited talk to The Research Thing meetup 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Invited to deliver talk at The Research Thing, a London-based UX design practitioner meetup group (title: "Talking with machines?-Voice UI and conversation design", meetup theme: "User research and voice").

I was asked to give the talk on the basis of my Interaction18 talk earlier.
Year(s) Of Engagement Activity 2018
URL https://www.meetup.com/researchthing/events/249294744/
 
Description Keynote speaker at "Video, Mobility, Technology and Multimodal Interaction: One-Day Seminar", Institute for Advanced Study, Loughborough University, Jan 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact Around 40-50 postgrad students, along with academics and postdoc researchers attended the one day seminar. Talk title "Dealing with trouble in a performative control room". Much discussion during all the sessions including mine. Informal feedback suggested that the talk was well received and of value to postgrads e.g., in their methodological development etc. Other speakers on the programme are all internationally recognised experts in the area of interaction studies.
Year(s) Of Engagement Activity 2018
URL http://www.lboro.ac.uk/research/ias/events/videomobilitytechnologyandmultimodalinteraction/
 
Description Medium post on voice interface design practitioners' approaches to guidelines 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Medium post on how voice interface practitioners use guidelines, see:

https://medium.com/voice-interfaces-and-hci/what-do-voice-interface-experts-think-about-design-guidelines-c885ea60cb12
Year(s) Of Engagement Activity 2021
 
Description Presentation at Tech Nottingham meetup ("I'd Hide You: Playing with Live Video in Public") 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact Presentation to a non-academic (largely industry) audience. I presented findings from a recent CHI paper; talk entitled: "I'd Hide You: Playing with Live Video in Public". This was part of a regular evening event (TechNotts), meaning that we were able to bring research findings to the regional Nottinghamshire technology community.
Year(s) Of Engagement Activity 2016
URL http://www.technottingham.com/events/2016/10/3/tech-nottingham-october-2016-the-mixed-reality-lab-ta...
 
Description Talk at Interaction18 conference (UX + design practitioners), Lyon, Feb 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Talk delivered at Interaction18 conference in Lyon, the main yearly conference for the IXDA professional association (UX, interaction designers, etc.). This is one of the major interaction design conferences, and has international reach. It generated a lot of questions and discussion with practitioners at the conference, as well as initial discussions with a book publisher (Rosenfeld media) about the possibility of developing a book for practitioners on the topic of the talk. The data connected with the talk has been requested by several practitioners for access etc. I also posted a Medium post version of my talk (https://t.co/NooXh3xtgv) which got 22 retweets and 60 likes on Twitter, along with 1.8k+ "views" and 460+ "reads" on Medium as well as being linked to by other blog posts. The talk itself has also been mentioned in various conference reports. Feedback suggested practitioners found the talk and blog post to be of value to them in their design practice.
Year(s) Of Engagement Activity 2018
URL https://interaction18.ixda.org/program/6_tuesday/
 
Description Talk delivered to Cambridge Usability Group 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Talk at Cambridge Usability Group, hosted by Cambridge Consultants, Aug 2017: "How to talk with machines: Voice UI and conversation", discussion afterwards and recommendation to propose similar talks at Interaction18 and UX in the City: Manchester. Led to discussions with Cambridge Consultants about possible partnership on funding bids on this topic.
Year(s) Of Engagement Activity 2017
URL https://www.eventbrite.co.uk/e/how-to-talk-with-machines-voice-ui-and-conversation-tickets-366027949...
 
Description Talk given at UX practitioner day conference (HCID Open Day, City University) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact Each year the HCID centre at City University has an open day which is for UX and design practitioners in local industry. I gave a talk based on my research about the history of HCI (the theme for the Open Day was "Past, Present and Futures").
Year(s) Of Engagement Activity 2016
URL http://hcidopenday.co.uk