Understanding and improving risk assessment on domestic abuse cases

Lead Research Organisation: University of Manchester
Department Name: Law

Abstract

Domestic violence is one of the most common forms of crime coming to the attention of the police. It can have severe and long-standing consequences. Yet our understanding of how to respond to domestic violence, despite various decades of research is limited. It is, thus, important to develop evidence that helps to improve our responses.

A particular response that became popular during the 1990s was to develop tools trying to identify those cases coming to the attention of the police that could result in more severe incidents in the future. As much as a third of domestic abuse cases may result in new calls to the police, but only a smaller percentage will result in more severe injuries. Given the volume of cases and the increasingly limited resources available, police and victim services organisations have used these predictive tools to focus their attention in those cases that have a higher probability of future harm. These predictions have very real consequences for they shape the level of responses.

In the UK the tool that most organisations and police forces use is called DASH. DASH was developed only by means of (1) reviewing the literature on intimate partner violence (IPV) characteristics and (2) by examining the features of a small non-random sample of homicides and near misses. There are more appropriate methods for developing and testing predictive models. These methods have been developed by statisticians and computer scientists and they are increasingly been used to improve the quality of decisions made in criminal justice. Our project aims to use these methods to evaluate the quality of the predictions made with DASH. We suspect DASH may work better in cases of IPV than in other cases in which it is applied, because of the way it was originally developed. We also suspect (based on existing research) that simply counting the presence of particular risk factors is not optimal -some factors may be more important or may particularly elevate risk when appearing in conjunction with others. Finally, we suspect there may be other characteristics of these cases that would lead to better predictions - given all we have learnt about domestic abuse since DASH was created. Our project will use data from the police and other organisations to explore if we can develop better predictive models that could result in tools to subsequently be piloted. We will also investigate the challenges the police would face to implement these new models. We know that it is possible to develop national risk assessment systems like the one we envisage (as in Spain), but we also know that implementing these changes would benefit from understanding the challenges associated with it.

Our second objective is to investigate if we can identify types of IPV. Various scholars have argued that IPV can be grouped in various types associated with different causes, evolution, and treatment needs. Although various researchers have been making this point for over 20 years now, we have little understanding of whether we can actually identify these types using police data. If we could identify particular profiles when they first come to the attention of the police, then potentially we could start experimenting with more tailored responses to the needs of each of these profiles.

Finally, our project is also trying to explore whether current responses to domestic abuse work. Those victims classified as high risk with DASH are referred to a MARAC: a group of professionals aiming to produce a set of responses that can help victims to cope with their situation and improve their safety. Our data will allow us to study whether these referrals reduce the probability of future victim harm. The official evaluation of the MARAC's concluded that we do not know whether they are having a measurable impact. We will use a method called regression discontinuity design that is particularly well suited in situations in which you cannot do a classic experiment.

Planned Impact

Domestic violence is presented as a high priority for public services (WHO 2013; HM Government 2013). It is widely accepted we require better evidence on how to assess and respond to it, particularly with cuts to public spending (Towers & Walby, 2012). Risk assessment in domestic violence presents a number of complex problems for the police and victim support organisations. Decisions about the level of risk shape the level of intervention and support provided to victims (i.e., referral to independent domestic violence advisors) and perpetrators (i.e., police bail). Our research aims to improve practice in this area.
The University of Manchester has a strong tradition of using research to make a positive impact on real world challenges. In recent years the University has been awarded more than £15million to support impact and KE activities, helping us to drive improvements in a wide range of external partners. Our research team has a track record of impact in this area through the development of risk assessment tools for police and probation, the publication of an article in a professional journal, and the involvement in the College of Policing Working Group tasked to improve the current system. Beneficiaries will be:
-Police. The central aim of the police is to protect the public. A risk assessment framework that allows them to better identify those at high risk of future harm and to better understand their needs are instrumental to that goal. This project fits with calls for stronger evidence-base policing (Sherman, 2013). Our work also seeks to improve the efficiency of service delivery by developing evidence that could lead to a systems design approach to risk assessment.
-Other criminal justice actors. Today, despite the emphasis on multiagency collaborations, the input from other criminal justice agencies (i.e. probation) in the early stages of risk assessment is limited. Our work will aim to illustrate the difference that an earlier involvement in risk assessment these agencies could make and pave the way for more creative ways of thinking about responding to offenders.
-Public health services. Violence is increasingly seen as a public health issue. There are concerted efforts for better screening of domestic violence in healthcare settings, some of which use the risk assessment tool (DASH) we plan to evaluate. A better understanding of properties of this tool will benefit screening programs.
-Third Sector. Likewise most organisations working with victims or abusive men (i.e., Respect) also use DASH. Our work will provide evidence that could guide this practice.
-Policy makers. Domestic abuse has become a political priority for all major political parties and government. Our research can help political actors to inform their action plans in this area.
-General public. The 2012/13 Crime Survey For England and Wales (ONS 2014) showed that 16% of men and 30% of women reported experiencing any partner abuse since 16 and 4% of men and 7% of women in the last 12 months. Given that these figures are likely to be an underestimate, it is obvious that our research has the potential of being relevant to a non-trivial proportion of the population. Several organisations have piloted the deployment of these tools for self-assessment. We will explore the technical and ethical issues associated with doing this as part of our project. But more generally it could be argued that a better informed risk assessment tool could be of benefit to victims insofar as it would allow better responses tailored to their specific needs.
-Commercial sector. There is a flourishing sector of companies producing user-friendly data analytic solutions to the police. Although one could question whether the added-value justifies the cost of some of these, it is also clear that these companies' expertise can provide an invaluable service. The publication of our programming code will facilitate re-utilisation by the actors trying to provide these service

Publications

10 25 50
 
Description The work so far has established that:
(1) DASH as it is currently used has very poor classification accuracy (not better than random);
(2) This is likely due to measurement error, that is, the quality of the information that officers gather is of insufficient quality;
(3) The use of regularised logistic regression model or gradient boosting models to score this tool provides better, if still poor, predictive accuracy that the scoring done by officers;
(4) Only a subset of items in the tool provide predictive information and it was clear that some officers were better at capturing useful (predictive) information than others, although lack of data prevented us from identifying the features of the most successful officers (clearly this area needs more work) ;
(5) As hypothesized the tool works marginally better for intimate partner violence rather than for other scenarios of domestic abuse.
(6) The use of additional information sitting in police databases (instead of DASH) provides more decent and usable intelligence about the risk of recidivism, the predictive models using this type of information offers better predictive validity
(7) The best model was a logistic regression with regularisation with a ROC around .75. The top ten features included information about the history of abuse, the previous criminal antecedents, and some basic demographic data (e.g., gender, type of relationship).
(8) When examining the fairness of the classification (defined as equal TPR, FPR, or PPV across protected characteristics) the model performed better than the use of DASH as it is currently being deployed
(9) We also managed to explore how other police assessment tools used elsewhere work. We focused specifically data from VIOGEN (the Spanish risk assessment tool and data) and obtain fairly acceptable predictive classification with this tool. Given there is some overlap in terms of questionnaires items between VIOGEN and DASH (though VIOGEN is much broader) we suspect the whole infrastructure of support and training possible makes a difference in terms of the results obtained with this tool in Spain.
We are still in the process of establishing whether using data beyond DASH but available can improve the risk assessment tool.
(10) We demonstrated that although it is hard to beat with machine learning methods the current scoring rules of VIOGEN to predict new instances of abuse, using machine learning methods could help to improve the prediction of more serious forms of recidivism.
(11) We also conducted more descriptive analysis of types of abuse, with a focus on intimate partner violence.
(11a) Using data from the Spanish police we managed to document six different configurations of abuse that have close relationship with existing typologies of domestic abuse developed by authors such as Michael Johnson.
(11b) Using British data we managed to identify various typological trajectories. Most instances (61%) of dyadic intimate partner abuse only have one incident in the period we observed (2 years). Focusing on frequency of incidents, the most common pattern is de-escalation (51%) through the entire period. Focusing on harmfulness, we observe the somehow opposite effect, the most common pattern is escalation of the severity (46%). We are still working on analysing this data using different algorithms to classify trajectories and to study factors that may be associated with each of them.
(12) The qualitative component of the study suggested the need for:
-More training for officers at all level
-The need for some form triage for incidents classed at the various levels of risk (this is what happens in more successful models like the Spanish one, what the different levels of risk classification do is that they expedite the triage for the more severe cases)
-There is a general view among practitioners that it would be convenient for all triage to take place within the same safeguarding hub
-There is also a general view that there has been perhaps too much focus and discussion around risk assessment tools and too little policy action, discussion, and programmatic service delivery for opportunities for abusers to accept their responsibility and to undertake behaviour-change work.
Exploitation Route There is sufficient evidence to support the ongoing efforts by the College of Policing and others to shorten the length of DASH. Our work also sugges that these tools could be used in conjunction with modelling of existing police data in order to prioritise what cases require more urgent and dedicated response. Our findings with relation to the Spanish data suggest there are elements of that model that require further study by British police authorities, particularly its dynamic nature, its storage in a national warehouse that can be accessed by all police forces (and other relevant stakeholders), its continuous and ongoing work on developing the tools they use (rather than keeping them unchanged for over a decade as it has happened here).

We are still conducting analysis of the data to firm up some of our conclusions, particularly around the typological work. But we are also taking the data in new directions, to explore the actual impact of the police measures adopted on the back of the risk classifictions that result from these models.
Sectors Communities and Social Services/Policy,Government, Democracy and Justice

 
Description We are in the process of consultation with the Spanish Ministry of Interior for modifying the scoring algorithm of their risk assessment model. We also know a number of British police forces are looking at our results in order to rethink their risk assessment processes.
First Year Of Impact 2020
Sector Government, Democracy and Justice
Impact Types Policy & public services

 
Description Collaboration with Universitat Autonoma de Barcelona 
Organisation Autonomous University of Barcelona (UAB)
Department Department of Mathematics
Country Spain 
Sector Academic/University 
PI Contribution We managed to establish a collaborations at this institution to obtain access to data from the Spanish police. We are in the process of developing a protocol to collaborate in the analysis. A senior researcher from this institution will supervise, at no cost to our project, one of our research assistants work analysing this data.
Collaborator Contribution See above.
Impact The collaboration in the end came to no fruition.
Start Year 2018
 
Description Collaboration with the Ministry of Interior (Spain) 
Organisation Ministry of Interior, Spain
Country Spain 
Sector Public 
PI Contribution We incorporated data from the Spanish police into the project.
Collaborator Contribution They provided access to the large sensitive dataset on intimate partner abuse in Spain. The two members of the Spanish Ministry of Interior team responsible for the system are collaborating in the co-authorship of two research articles. There are also plans for further analysis that go beyond the scope of the original funding,
Impact We are in the process of completing two peer review articles for which all the analysis has been completed. It is likely the lessons will be used by the Spanish Ministry of Interior to readjust the scoring rules of their risk assessment tool.
Start Year 2018