A Population Approach to Ubicomp System Design
Lead Research Organisation:
University of Glasgow
Department Name: School of Computing Science
Abstract
Adding and removing modules (also called plug-ins, add-ins and other names) without technical support or supervision is becoming the norm for everyday software applications such as web browsers, image editors and email tools, e.g. Firefox, Photoshop and Outlook. This same approach is becoming important in mobile phone software, where 'in-app purchase' of modules is becoming more and more popular, and a huge money-spinner for developers.
The consequences are not all good: users often do not know which modules to use or change, to suit their goals, or whether a program will crash after such changes. As a result of modules being from different developers, their combination may never have been tested prior to public use. Evaluators and developers struggle to help, because established approaches to software definition, design and analysis are based on the structure of a program being the same wherever and whenever it is used. In constrast, one would be hard put to define a single software structure that accurately describes what a program like Firefox is. Use is similarly hard to pin down, as individuals make systems fit with their own uses and contexts, and share their innovations with others.
As a modular program becomes complex, the result is often a 'plug-in hell' of broken software dependencies, functions, uses and expectations. If, instead, software structure is kept simple, then design opportunities are lost as developers avoid the difficulty and cost of implementing innovative programs. More generally, software theory and engineering have fallen behind practice. We lack ways to reason predictively in a principled way about real world structure and use. We lack tools for designers, evaluators and users that support adaptation, and we lack principles and techniques that can deal with the scale of human and software interaction.
Our primary objective is to deliver a new science of software structures, with design, theory and tools that reflect software in real world use, and able to tackle the complex problem of how to design to support change and appropriation. The key concept is the 'software population': a statistical model of the variety we see when we look at how the same initial program has been used and adapted by its users. A population model is kept up to date by each instance of a program logging how it is used and changed. The population idea affords a common design vocabulary for users' understanding and adaptation of programs, for evaluators' analysis of programs in use, and for developers' making informed changes to the modules available to users.
As a result, users will have programs that may vary but are more comprehensible, robust and adaptable than is the case today. We will enable each individual user to make a practical decision that only he/she is qualified to make: how to balance the changed robustness and functionality of one's system with changes to the system's support for individual and social interactions. One will have tools built into one's program that makes clear what it consists of, and how its structure relates to the programs and experiences of other users. One can find out about other modules that are compatible with one's program, how it will work after adding in one or more new modules, and therefore which configurations of modules will and will not work.
In order to test whether the approach works at the scale of, for example, typical iPhone applications, we will build and deploy programs among large numbers of users, for weeks or months-mobile games and social networking applications. We will work with industrial partners including the Edinburgh Festivals, as well as using popular sports and events, such as soccer (e.g. the 2014 FIFA World Cup Brazil), athletics (e.g. the London 2012 Olympics and the Glasgow 2014 Commonwealth Games), and popular TV programmes. Overall, We plan for 500,000-1,000,000 users of our systems in the course of the programme.
The consequences are not all good: users often do not know which modules to use or change, to suit their goals, or whether a program will crash after such changes. As a result of modules being from different developers, their combination may never have been tested prior to public use. Evaluators and developers struggle to help, because established approaches to software definition, design and analysis are based on the structure of a program being the same wherever and whenever it is used. In constrast, one would be hard put to define a single software structure that accurately describes what a program like Firefox is. Use is similarly hard to pin down, as individuals make systems fit with their own uses and contexts, and share their innovations with others.
As a modular program becomes complex, the result is often a 'plug-in hell' of broken software dependencies, functions, uses and expectations. If, instead, software structure is kept simple, then design opportunities are lost as developers avoid the difficulty and cost of implementing innovative programs. More generally, software theory and engineering have fallen behind practice. We lack ways to reason predictively in a principled way about real world structure and use. We lack tools for designers, evaluators and users that support adaptation, and we lack principles and techniques that can deal with the scale of human and software interaction.
Our primary objective is to deliver a new science of software structures, with design, theory and tools that reflect software in real world use, and able to tackle the complex problem of how to design to support change and appropriation. The key concept is the 'software population': a statistical model of the variety we see when we look at how the same initial program has been used and adapted by its users. A population model is kept up to date by each instance of a program logging how it is used and changed. The population idea affords a common design vocabulary for users' understanding and adaptation of programs, for evaluators' analysis of programs in use, and for developers' making informed changes to the modules available to users.
As a result, users will have programs that may vary but are more comprehensible, robust and adaptable than is the case today. We will enable each individual user to make a practical decision that only he/she is qualified to make: how to balance the changed robustness and functionality of one's system with changes to the system's support for individual and social interactions. One will have tools built into one's program that makes clear what it consists of, and how its structure relates to the programs and experiences of other users. One can find out about other modules that are compatible with one's program, how it will work after adding in one or more new modules, and therefore which configurations of modules will and will not work.
In order to test whether the approach works at the scale of, for example, typical iPhone applications, we will build and deploy programs among large numbers of users, for weeks or months-mobile games and social networking applications. We will work with industrial partners including the Edinburgh Festivals, as well as using popular sports and events, such as soccer (e.g. the 2014 FIFA World Cup Brazil), athletics (e.g. the London 2012 Olympics and the Glasgow 2014 Commonwealth Games), and popular TV programmes. Overall, We plan for 500,000-1,000,000 users of our systems in the course of the programme.
Planned Impact
This programme involves advances in knowledge of direct relevance to those involved in the design, construction and use of systems based on modules, plug-ins and similar software infrastructures. These advances will include a new approach to fundamental computer science concepts, in particular 'class', as well as practically oriented results such as tools and infrastructures for large-scale public deployments of applications. Key aspects and examples of our approach will be packaged as high-level guidelines and case studies, open source code with tutorial demonstrator examples, and the broader evidential record of code and data that will be released in its final form at the end of the programme. These resources will collectively form empirical evidence of the theoretical concepts' validity and significance, and the technical effectiveness and social acceptability of system construction based on these concepts.
We will promote the new approach within our research communities, by maintaining our strong publication record in premier journals and conferences, organizing workshops and tutorials at premier conferences, and presenting our work at leading research institutions. The programme will have a significant impact in increasing the UK's capacity to undertake interdisciplinary research that spans between academic and industrial concerns. We will help develop the next generation of research leaders by exposing the RAs/RFs and interns to a world-leading interdisciplinary environment, in which research is expected to be both pioneering in terms of academic quality and yet grounded in terms of relevance and utility in large-scale real world deployments.
As a result of our work, the general public will have programs that may vary but are more comprehensible, robust and adaptable than is the case today. The general public and the education sector can benefit from learning about, and experimenting with, population-based application design and associated analysis tools, so as to understand their everyday implications. We will pursue opportunities for outreach and advocacy among the general public, industry and the wider scientific community, e.g. demonstrations and displays at science festivals, trade fairs and industrial outreach events, and debate through learned societies. Part of this outreach is inherent in our large-scale deployments, which will engage with 500,000-1,000,000 users over the lifetime of the programme. We will continue to use press coverage to inform the public and industry of our work. Benefits will start to accrue during the project and many of the demonstrators will focus on activities that aim to engage with the general public. Taken together, these points make for high impact research carried out in a very public way, which is one of the distinctive features of this research endeavour.
Policy makers will benefit from considering new avenues for introducing new socially acceptable policies, using knowledge about people's reaction to public deployments and case studies. Benefits are likely to accrue starting late on in the programme, in a 4-6 year timescale, drawing upon the programme's case studies and evidential record.
Industrial beneficiaries will gain from exposure to the development and deployment of technologies that have demonstrated their potential for exploitation in products for mass markets. The benefits are likely to accrue as collaborative research and spin off projects within the programme, and independent development within 1-2 years of the end of the programme. Collaborating partners will benefit from access to the leading researchers in this new area, the ability to shape the ongoing research as it happens, the exchange of staff through inward and outward secondments, and the ability to demonstrate and evaluate the work in deployments aligned with their commercial areas of interest.
We will promote the new approach within our research communities, by maintaining our strong publication record in premier journals and conferences, organizing workshops and tutorials at premier conferences, and presenting our work at leading research institutions. The programme will have a significant impact in increasing the UK's capacity to undertake interdisciplinary research that spans between academic and industrial concerns. We will help develop the next generation of research leaders by exposing the RAs/RFs and interns to a world-leading interdisciplinary environment, in which research is expected to be both pioneering in terms of academic quality and yet grounded in terms of relevance and utility in large-scale real world deployments.
As a result of our work, the general public will have programs that may vary but are more comprehensible, robust and adaptable than is the case today. The general public and the education sector can benefit from learning about, and experimenting with, population-based application design and associated analysis tools, so as to understand their everyday implications. We will pursue opportunities for outreach and advocacy among the general public, industry and the wider scientific community, e.g. demonstrations and displays at science festivals, trade fairs and industrial outreach events, and debate through learned societies. Part of this outreach is inherent in our large-scale deployments, which will engage with 500,000-1,000,000 users over the lifetime of the programme. We will continue to use press coverage to inform the public and industry of our work. Benefits will start to accrue during the project and many of the demonstrators will focus on activities that aim to engage with the general public. Taken together, these points make for high impact research carried out in a very public way, which is one of the distinctive features of this research endeavour.
Policy makers will benefit from considering new avenues for introducing new socially acceptable policies, using knowledge about people's reaction to public deployments and case studies. Benefits are likely to accrue starting late on in the programme, in a 4-6 year timescale, drawing upon the programme's case studies and evidential record.
Industrial beneficiaries will gain from exposure to the development and deployment of technologies that have demonstrated their potential for exploitation in products for mass markets. The benefits are likely to accrue as collaborative research and spin off projects within the programme, and independent development within 1-2 years of the end of the programme. Collaborating partners will benefit from access to the leading researchers in this new area, the ability to shape the ongoing research as it happens, the exchange of staff through inward and outward secondments, and the ability to demonstrate and evaluate the work in deployments aligned with their commercial areas of interest.
Publications
Andrei O
(2016)
Integrated Formal Methods
Andrei O
(2014)
Quantitative Evaluation of Systems
Andrei O
(2012)
Transactions on Computational Systems Biology XIV
Andrei O
(2019)
Interpreting Computational Models of Interactive Software Usage
Andrei O
(2018)
Data-driven modelling and probabilistic analysis of interactive software usage
in Journal of Logical and Algebraic Methods in Programming
Andrei O
(2018)
Software Engineering and Formal Methods
Asadzadeh P
(2016)
Investigating how users engage with a pedometer app
Bell M
(2013)
Experiences in Logging Everyday App Use
in Proc DE2013 - Open Digital, Fourth RCUK All Hands Digital Economy Conference
Benford S
(2016)
On Lions, Impala, and Bigraphs Modelling Interactions in Physical/Virtual Spaces
in ACM Transactions on Computer-Human Interaction
Buls D
(2017)
Technology for Self-Management of Rosacea
Elsden C
(2017)
Quantified Data & Social Relationships
Higgs M
(2013)
Analysing user behaviour through dynamic population models
Keen J
(2013)
Big data + politics = open data: The case of health care data in England
in Policy & Internet
McCallum C
(2019)
An N-of-1 Evaluation Framework for Behaviour Change Applications
McMillan D
(2013)
Categorised ethical guidelines for large scale mobile HCI
Morrison A
(2014)
Logging Phone Usage to Understand Health and Wellbeing
in ACM CHI 2014 'Beyond the Quantified Self' Workshop
Morrison A
(2012)
A hybrid mass participation approach to mobile software trials
Morrison A
(2017)
Stickers for Steps A Study of an Activity Tracking System with Face-to-Face Social Engagement
in Proceedings of the ACM on Human-Computer Interaction
Morrison A
(2018)
A Large-Scale Study of iPhone App Launch Behaviour
Morrison, A
(2018)
A Large-Scale Study of iPhone App Launch Behaviour
Podlubny M
(2017)
Synchronous Text Messaging A Field Trial of Curtains Messenger
in Proceedings of the ACM on Human-Computer Interaction
Description | The biggest development in the programme was the shift from modelling a 'snapshot' of all the instances of an app that we deployed, i.e. patterns drawn from data on all users at a single given time, to modelling temporal patterns of app use over time, i.e. patterns (such as 'strategies' and 'habits') of use common among groups of users. These patterns often change as people get used to apps, and with updated app versions, and so it's good to do this kind of analysis for multiple different 'windows' of time. Our nicest example of this was probably in our AppTracker app, in which we found a pattern of repeated movement between disparate parts of the app so as to compare information. We redesigned the app so as to make this comparison easier and quicker to do, and observed people using the updated version more readily and comfortably. We realised that the first 'snapshot' approach simply wasn't giving us insight into the meaning and value of apps (and of features within apps), and so we had to do a radical retooling in terms of experimental method and timescale, data collection approaches, and analytic methods. As mentioned below, we were also pushed towards studying use more than software structure (as per the case) because of technical constraints of Apple phones, but this turned out to be beneficial in terms of deeper insight. As part of this, there was a core methodological advance, in that we bridged from statistical inference methods (as evinced in the programme by the work of Prof. Mark Girolami and RAs) to the formal modelling methods (evinced by the work of Prof. Muffy Calder and RAs). We have developed ways to go from the fuzzy statistics of who did what when with our apps, which are very hard to ask questions about, to more 'crisp' and logical models of groups of users and their patterns of app use. This kind of technical combination did not exist before, and is (admittedly) still quite hard to do, but it brings remarkable insight into the operations of software systems in use. Also, we have developed methods and experience in formal modelling that challenge the norms there. Instead of using such mathematical models to verify a hypothesis about what apps do, or to test what the ideal value for a system structure is, we have advanced work in which one can explore different models and structures... and then use that to gain a broader knowledge of app use, that in turn can feed back into app design. A challenge (for a future project perhaps) is to find ways to speed up or automate that formal modelling, so that it fits with the pace of modern software development... which is fast and getting faster. Also, I would highlight some of the recent statistical work that has broadened the range of data types used in modelling apps, e.g. combining our logs of what apps people use on their phones with App Store descriptions and ratings for those apps. We have been trying to model and understand how apps get used, and this broadening makes our models more accurate. I point this out because most analysis in this area uses a very narrow set of features in its modelling, but in this programme we have explored and demonstrated that there are many more useful choices available. More generally speaking, we have shown that there are choices not only with regard to the breadth of features analysed, but also to the use of temporal patterns in those features, as well as the combination of formal and statistical modelling techniques. |
Exploitation Route | We have published and are publishing analytic methods for software (and related systems) that take account of temporal patterns, and which extend to formal modelling. These offer app developers greater insight into app use than was previously feasible. At the moment, such formal modelling approaches are far less well known than statistical ones, (as mentioned above) it would be of benefit if they could be given technical support so as to make them quicker/easier to use. The statistical methods we used (e.g. using side information) can and will be used by many researchers and analysts working on understanding software and its use/execution. They offer greater accuracy and insight in processes such as the comparison and recommendation of software. |
Sectors | Digital/Communication/Information Technologies (including Software) |
URL | http://www.softwarepopulations.com |
Description | Two of my RAs (Marek Bell and Scott Sherwood) were instrumental in the tech infrastructure of Glasgow City Council's TSB-funded future cities demonstrator. They applied our approach to systems infrastructure and innovation there, with regard to basic infrastructure but also app user experience. They found this fruitful enough to concentrate on it in their own company, Dynamically Loaded, later pulling in another of my RAs: Matthew Higgs. There is also an ongoing feeding in of ideas from the programme into Glasgow University's £1B campus development programme, in the 'smart campus' work described elsewhere and in the more general use/collection/analysis of student data. One of the most significant presentations in this regard was on the potential for advanced analytics in the smart campus, to Jo Johnson (Minister for Universities and Science), at U. Glasgow on October 26th 2015. The context was the minister's interest in the potential of technology to improve educational and student satisfaction outcomes. |
First Year Of Impact | 2013 |
Sector | Digital/Communication/Information Technologies (including Software),Education,Government, Democracy and Justice |
Impact Types | Economic Policy & public services |
Description | Smart Campus project at U. Glasgow |
Geographic Reach | Local/Municipal/Regional |
Policy Influence Type | Participation in a guidance/advisory committee |
Impact | As my university was beginning to consider 'smart campus' developments, and new forms of analysis of student data more generally. It was therefore considering ramping up the collection of analysis of data on students' life on campus, the Populations programme created a demonstrator to make apparent to senior management some of the pros, cons and particularities of this kind of work -- aiming to inform them of potential directions that might alleviate concerns about 'big brother' surveillance. To this end, we created two apps for iOS phones and tablets, that collected data on the user's location and movements, and then allowed to user to selectively pass on that data to our servers, where we could apply our analytic methods to determine patterns of movement, occupancy, etc. Each app also was designed so that there was a direct benefit to the user (in reflecting on their activity and in selectively sharing it with friends) instead of their data just being siphoned off for institutional benefits. Our qualitative study of the users showed that this direct/individual benefit, along with the opt-in nature of data sharing, meant that users were much happier about taking part in large scale data collection/analysis exercises. As a result, Frank Coton (the VP leading these programmes, shifted his priorities so as to use opting in to data collection, making data collection anonymous if at all possible, and in finding ways to give benefits back to students. Chalmers (the PI on the grant) thereafter joined the university's project boards focusing on the Ethical Use of Student Data, large scale software procurement, and on GDPR. |
Description | Digital Economy NetworkPlus |
Amount | £1,039,048 (GBP) |
Funding ID | EP/R045178/1 |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 06/2018 |
End | 06/2021 |
Description | EU FP7 |
Amount | € 5,957,158 (EUR) |
Funding ID | 602170 |
Organisation | European Commission |
Sector | Public |
Country | European Union (EU) |
Start | 11/2013 |
End | 10/2018 |
Description | Grant award (NICOP) |
Amount | $93,333 (USD) |
Funding ID | N62909-18-1-2076 |
Organisation | US Navy |
Department | US Office of Naval Research Global |
Sector | Academic/University |
Country | United States |
Start | 02/2018 |
End | 04/2019 |
Description | Multi-Perspective Design of IoT Cybersecurity in Ground and Aerial Vehicles (MAGIC) |
Amount | £193,921 (GBP) |
Funding ID | MAGIC |
Organisation | PETRAS National Centre of Excellence |
Sector | Academic/University |
Country | United Kingdom |
Start | 05/2020 |
End | 11/2022 |
Description | SICSA Short Term Research Theme |
Amount | £4,000 (GBP) |
Organisation | SICSA Scottish Informatics and Computer Science Alliance |
Sector | Academic/University |
Country | United Kingdom |
Start | 07/2013 |
End | 07/2014 |
Description | programme grants |
Amount | £4,183,694 (GBP) |
Funding ID | EP/N007565/1 |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 01/2016 |
End | 12/2020 |
Title | User trial of the ScreenLife personal tracking application |
Description | |
Type Of Material | Database/Collection of data |
Year Produced | 2016 |
Provided To Others? | Yes |
Description | Arup Group Ltd |
Organisation | Arup Group |
Country | United Kingdom |
Sector | Private |
PI Contribution | The stadium project discussed its ideas with Arup, to seek their advice as to industrial settings and to feed back into their design work |
Collaborator Contribution | The time of Arup staff members |
Impact | We had informal workshops in which Arup staff such as Duncan Wilson participated (before he left for Intel). |
Start Year | 2007 |
Description | Campus positioning system |
Organisation | Dynamically Loaded Ltd |
Country | United Kingdom |
Sector | Private |
PI Contribution | We set up a collaboration with this local SME (founded by ex-postdocs in my group) in which DL developed a positioning system that bridges between indoor and outdoor locations, so as to create part of a new app or apps for our campus. They did the positioning subsystem but we did the app development. |
Collaborator Contribution | Skills in positioning systems, and also a large database of beacon (e.g. wifi) locations. |
Impact | Demonstrators for 'Smart Campus' -- part of 'other outputs'. |
Start Year | 2015 |
Description | Microsoft Research Ltd |
Organisation | Microsoft Research |
Country | Global |
Sector | Private |
PI Contribution | MSR staff such as Alex Taylor took part in discussions and workshops about the project, advising us re methodological issues in particular. |
Collaborator Contribution | Methodological advice |
Impact | The sociological work fed into the overall project's approach to engaging with study participants. |
Start Year | 2007 |