Robust approaches for the analysis of agreement between clinical measurements: development of guidance and software tools for researchers

Agreement studies are concerned with assessing how closely two different methods or devices agree in their measurements of the same variable. For example, an agreement study may wish to compare two devices for measuring blood pressure, and if the measurements from the two devices are sufficiently close to each other, we can conclude that the two devices agree. There are many different statistical approaches that can be used to assess agreement, but not all of them are valid in every context, and some are not valid in any context. In this project, we wish to review the recent literature to investigate the methods that have been used by researchers for assessing agreement in practice. We will also develop a new method for assessing agreement that we expect will be suitable for when data does not follow the unusual pattern. We will thoroughly test this new method against standard approaches, as well as evaluate other newly developed agreement methods that have been put forward by other researchers in the last few years. In addition, we will develop software tools so that researchers can easily use the new approaches, as well as standard methods. Finally, we will publicise all of our research findings so that they are readily accessible by the research community. One of the main aims of this project is to promote the use of valid methods to assess agreement, and prevent any wrong applications of agreement methods, which could lead to inferior devices, medical procedures, or data collection methods being used in patients.

In this project, we will perform a systematic scoping review of the agreement literature to determine which methods of analysing agreement are currently being used, and whether these are valid. We will also develop a new agreement method on the basis of robust linear mixed models, which does not rely on normality assumptions and can be used in the context of repeated measurements. We will then perform a thorough simulation study, covering a wide range of real life scenarios, to evaluate the performance of recently proposed novel methods of agreement as well as the new method we have developed, particularly focussing on repeated measures agreement. This simulation study will involve benchmarking each of the methods against each other as well as against standard methods. We will develop software tools to make it easier for researchers to apply the methods. In particular, we will develop an R software package, which will include multiple methodologies and facilitate practical application of the agreement methods. We will also develop a shiny app implementing several methods to provide an interactive web app for practitioners. We will ensure that comprehensive documentation accompanies the R package and Shiny app so that users know how to use the software properly. Finally, we will disseminate our results to end users of agreement.

An independently chaired research study steering group (RSG) of end users, consisting of clinical researchers and statisticians from industry and academia, as well as Patient and Public Involvement (PPI) representatives, will be set up. The RSG will ensure that we are focussed on doing methodological research that really matters to researchers, clinicians and biostatisticians, and that patients are the ultimate beneficiaries. The RSG will also advise on appropriate dissemination strategies.