Exascale Agent-based Modelling for PoLicy Evaluation in Real-time (ExAMPLER)
Lead Research Organisation:
James Hutton Institute
Department Name: Information & Computational Sciences
Abstract
ExAMPLER will bring the Social Sciences into the conversation about the exciting potential exascale computing offers to enable empirical agent-based models (sometimes referred to as agent-based social simulations) to be built and experimented with in a one-day transdisciplinary workshop.
During the COVID crisis, computer modelling played a significant role in evaluating difficult policy options. In particular, a form of computer modelling known as empirical agent-based modelling, which explicitly represents people and businesses interacting with each other in a simulated geographical space, can be used to see how and whether proposed policy interventions yield their intended societal effect, while checking for disproportionate consequences in different regions, or by gender or ethnicity. These models are expensive to build and use, because they have many potential parameters to adjust, and are capable of very different outcomes even given the same parameters and initial conditions. Experimentation with such models to explore possible interventions can therefore use months of computing time. Exascale computing means that this time can be reduced to a few seconds, creating the potential that evaluation of scenarios can be done as part of a creative, exploratory discussion.
Although exascale computing means that experimenting with an existing model can be done much more quickly, there is more fundamental uncertainty in the social sciences over the structure of such models in the first place - due to the diversity of ways in which social systems can be conceived and theorized. The dramatic shortening of computing time needed to experiment with a model at exascale also means that model development and data assembly and integration - which also take months of time - are now much more significant bottlenecks. Exascale computing therefore creates a context in which we need a deeper review of the computing environments in which agent-based modelling is done.
The ExAMPLER project will undertake a series of activities aimed at engaging with the diverse, interdisciplinary and international community of researchers working on empirical agent-based modelling, to discuss what exascale computing means for the community. We will undertake a structured literature review to assess the state-of-the-art in high-performance computing to support empirical agent-based modelling, and use this as a basis to create Key Performance Indicators of 'exascale readiness' in a community, so that we can benchmark ourselves against other communities. We will co-construct a vision of the future of empirical agent-based modelling supported by exascale computing, and a roadmap of the software and hardware, institutions and training needed to bring that vision into effect. We will engage with other ExCALIBUR communities considering the potential of exascale computing, and use existing conferences as a basis for meeting with our own community.
During the COVID crisis, computer modelling played a significant role in evaluating difficult policy options. In particular, a form of computer modelling known as empirical agent-based modelling, which explicitly represents people and businesses interacting with each other in a simulated geographical space, can be used to see how and whether proposed policy interventions yield their intended societal effect, while checking for disproportionate consequences in different regions, or by gender or ethnicity. These models are expensive to build and use, because they have many potential parameters to adjust, and are capable of very different outcomes even given the same parameters and initial conditions. Experimentation with such models to explore possible interventions can therefore use months of computing time. Exascale computing means that this time can be reduced to a few seconds, creating the potential that evaluation of scenarios can be done as part of a creative, exploratory discussion.
Although exascale computing means that experimenting with an existing model can be done much more quickly, there is more fundamental uncertainty in the social sciences over the structure of such models in the first place - due to the diversity of ways in which social systems can be conceived and theorized. The dramatic shortening of computing time needed to experiment with a model at exascale also means that model development and data assembly and integration - which also take months of time - are now much more significant bottlenecks. Exascale computing therefore creates a context in which we need a deeper review of the computing environments in which agent-based modelling is done.
The ExAMPLER project will undertake a series of activities aimed at engaging with the diverse, interdisciplinary and international community of researchers working on empirical agent-based modelling, to discuss what exascale computing means for the community. We will undertake a structured literature review to assess the state-of-the-art in high-performance computing to support empirical agent-based modelling, and use this as a basis to create Key Performance Indicators of 'exascale readiness' in a community, so that we can benchmark ourselves against other communities. We will co-construct a vision of the future of empirical agent-based modelling supported by exascale computing, and a roadmap of the software and hardware, institutions and training needed to bring that vision into effect. We will engage with other ExCALIBUR communities considering the potential of exascale computing, and use existing conferences as a basis for meeting with our own community.
Publications
Polhill G.
(2023)
Exascale computing and 'next generation' agent-based modelling
in Review of Artificial Societies and Social Simulation
| Description | Agent-based modelling is a computer simulation method used primarily in the social and ecological sciences. The attributes and behaviours of each individual in a complex system of interest are explicitly represented, with attention given to how people (or in ecology, plants and non-human animals) affect each other. These computer simulations are useful for exploring intervention scenarios - including most recently in assessing the effectiveness of measures to manage the spread of Covid. However, by representing complex systems in such detail, they need more computing power than other kinds of model. The ExAMPLER project investigated how ready the community of agent-based modellers in the social sciences is to take advantage of the latest exascale computing environments using a gap analysis. Using a systematic literature review, we have found that the vast majority of agent-based modelling work is still being done on personal computer devices (laptops and desktops), and that details of the computing environment are under-reported. We also found that there is considerable variation and uncertainty in the language used by the interdisciplinary community of agent-based modellers to describe the ways in which they have addressed the challenges posed by complexity in their agent-based model - to the extent that the term 'agent-based model' carries relatively little information about a model's features. We also consulted with the community of agent-based modellers in the social sciences in the UK and internationally over a series of workshops we organized. These workshops outlined visions of what exascale computing could bring to agent-based modelling, and a roadmap to achieving those visions. High-Performance Computing environments in the UK have primarily focused on the needs of the natural sciences, where project teams have more resources (ESRC's budget is a fraction of that of EPSRC's) and can afford to pay for Research Software Engineers who can specialize in writing software dedicated even to specific hardware. By contrast, the majority of agent-based modelling software used in the social sciences is based on Java - though there is increasing use of Python and Julia by some researchers. We tried, and failed, to apply for computing time on ARCHER-2, with no explanation for the rejection. This experience rings true for colleagues attempting to access institutional HPC environments. The bureaucracy over access to HPC infrastructure together with the technical obstacles, combine to make HPC access prohibitively onerous for our community. A critical finding, then, for the social sciences, is to make HPC (and any future exascale) computing environments much more accessible. Finally, we engaged with other projects in the ExCALIBUR Programme over a series of events, and by arranging to meet with specific teams. We have met with Brunel to implement a NetLogo component to their fabsim3 software, which goes some way to hiding some of the technical issues with accessing HPC. We also had productive conversations with Exeter about applying their Uncertainty Quantification methods to agent-based rather than more traditional kinds of model. We met with the developers of the FLAME GPU architecture, which, with respect to providing an agent-based modelling environment for GPUs - essential to moving to exascale - is the only such system available to our community. |
| Exploitation Route | We will be able to make a strong case that, at least for some publication outlets (and most notably, the Journal of Artificial Societies and Social Simulation), details of the computing environment used to run agent-based models should be included in the authors' description of the methods. The European Social Simulation Association and other relevant international bodies can explore the possibility of including HPC literacy in future education programmes aimed as Ph.D. students and post-docs wanting to enhance their agent-based modelling skills. The Road Map document makes five recommendations that can be taken forward in the future. One is to explore what agent-based modelling can be achieved running on graph networks, and evaluating what dynamics and emergent features are 'lost' by this simplification. The second recommendation is to encourage greater use of peta-scale (HPC) computation in the community, the literature review having revealed its rarity. The third is to develop systems that support better code reuse and reduce the 'time to science', which is non-trivial with ABMs because issues with semantic heterogeneity tend to lead to bespoke models being developed for each scenario. Fourth is to develop proxy apps providing optimized code that can be deployed where needed. Fifth is to explore the use of metamodels trained on ABM behaviour. Research would be needed to pursue each of these recommendations further. Pragmatically, having GPUs available to model developers as part of standard IT provision by academic institutions is also a rather obvious missing step. Increased GPU literacy among those developing ABMs, and a growing expectation that GPU-based frameworks will be used for ABM is also a potential significant outcome from ExAMPLER. Equally, the expertise for this may need to be recognized as a specialism, and greater emphasis put on teamwork and the use of Research Software Engineers for ABM development could become more typical. ExAMPLER has created another layer of community linking computing and social sciences, that is working on a joint article about visions for exascale computing and ABM. |
| Sectors | Aerospace Defence and Marine Agriculture Food and Drink Communities and Social Services/Policy Digital/Communication/Information Technologies (including Software) Environment Financial Services and Management Consultancy Government Democracy and Justice Transport |
| URL | https://exascale.hutton.ac.uk/ |
| Description | Agent-based modelling has been included in a benchmarking project led by Prof. Garth Wells and funded by the EPSRC to investigate hardware needed to deliver to a broad range of requirements at HPC level. Findings from the Glasgow visions workshop (mainly involving social scientists) were shared and discussed in the London workshop (mainly involving computer scientists), enabling knowledge exchange across two communities. |
| First Year Of Impact | 2024 |
| Sector | Digital/Communication/Information Technologies (including Software) |
| Impact Types | Policy & public services |
| Description | Modelling Uncertainty |
| Organisation | Improbable |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | This work looks at quantifying uncertainty through the use of probabilistic programming for improving the forecasting accuracy of simulation tools. This directly feeds into the work undertaken by the Fellowship and links to other funded work. This work has resulted in a publication in 2025 in The Proceedings of the Royal Society on spatial validation of agent-based modelling outputs, a highly novel piece of work. The fundamentals of this work have inspired international collaboration and continued work within this area. The recent published work will have value for any agent-based modelling work whose outputs require spatially validating. |
| Collaborator Contribution | They have funded two intern projects and a PhD studentship as well as giving full access to their probabilistic framework (Keanu). |
| Impact | So far this work has resulted in the developed of two conference papers and software development. A paper has recently been published (see full portfolio). |
| Start Year | 2018 |
| Description | Agent-based modelling in the interface between economics and behavioral psychology (Student Course at NTNU) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Five students from across Norway (including a specialist working on fraud detection at a bank in Oslo) attended a training course I was asked to give on agent-based modelling. The course was described as "This course will give an overview of agent-based modelling (ABM). It will start exploring the need for this type of models, its complexity and predictability. The course will address social networks and socio ecological systems, including a discussion about modelling human behavior. The connection with behavior economics will be presented. Finally, the challenges in using ABM and the ethics of using artificial populations will be highlighted." |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://www.ntnu.edu/studies/courses/I%C3%988814#tab=omEmnet |
| Description | Chair of an international conference |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Co-Chair of the GIScience conference, one of the largest computational geography conferences in the field. 450 delegates attended. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://giscience2023.github.io |
| Description | Conference talk, International Microsimulation Association, University of Vienna |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Challenges of using microsimulation for simulating the impact of interventions on populations. IMS. University of Vienna, Austria. Jan 2024. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://ima-2024.wifo.ac.at |
| Description | High Performance Computing for Agent-Based Modelling Training Course |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | We organized a training course on running NetLogo agent-based models on High-Performance Computing using SLURM for practitioners in the field on 29 February and 1 March 2024. There were 9 in-person attendees, and 5 online, including from India, Colombia, the Netherlands and Canada. Feedback on the 'draft' course has been positive and constructive, and it is clear that this is a gap in training in agent-based modelling. Almost all the attendees who completed the evaluation reported feeling at least somewhat more confident about using HPC as a result of attending the course, with more than three quarters of them saying they would try to access local HPC facilities within the next three months. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://exascale.hutton.ac.uk/hpc-abm |
| Description | Invited keynote Expos Utrecht |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Keynote presentation - Challenges and opportunities of using ABMs for simulating social system. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://exposome.nl/news-events/news/highlights-from-the-exposome-nl-conference-2023.html |
| Description | Invited talk at a symposium at the University of Zurich |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Invited talk at University of Zurich. Challenges and opportunities of using ABMs for simulating social systems. |
| Year(s) Of Engagement Activity | 2023 |
| Description | Presentation on "Towards automated provenance collection for experimental runs of agent-based models" at ESSA 2023 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | About 30 people attended a presentation on the conference paper: "Towards automated provenance collection for experimental runs of agent-based models". This was at the European Social Simulation Association annual conference and one of a series of talks |
| Year(s) Of Engagement Activity | 2023 |
| Description | Roadmap to Agent-Based Modelling at Exascale |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | This workshop follows on from visioning workshops that took place at the Social Simulation Conference (September 2023) and UCL (November 2023) to outline what needs to be done to bring those visions into reality. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://exascale.hutton.ac.uk/roadmap-glasgow |
| Description | Scientific Chair of AGILE conference |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Scientific chair of the AGILE conference, an applied geography conference. Responsible for the scientific theme of the conference. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://agile-gi.eu/conference-2024 |
| Description | Talk at GIScience conference |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Talk called Exascale agent-based modelling for policy evaluation in real-time |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://giscience2023.github.io |
| Description | The transformative potential of exascale computing for agent-based modelling |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | As part of the project, we organized a workshop at the Social Simulation Conference on 4 September 2023, with a view to coconstructing with the practitioners in agent-based modelling there various visions of the impact exascale computing could have on the way they work. Approximately thirty people attended and the workshop stimulated discussion and ideas on use-cases for exascale computing that went beyond what we had conceived when putting the project together. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://ssc23-sphsu.online/programme/ |