The International Cognitive Ability Resource

After more than a century of active research on cognitive abilities, there now exist dozens of assessment tools of tremendous variety. Many of these are no longer widely used on account of being outdated or poorly maintained. Most of the remainder - those tests which are actively used and well maintained - can generally be organized into two broad types: (i) proprietary measures owned and administered by profitable commercial entities or (ii) idiosyncratic measures which are usually developed and used by isolated research groups for narrow purposes. This circumstance is problematic for the international community of cognitive ability researchers because it inhibits the generalization of findings across sub-fields and, more generally, slows the pace at which knowledge can advance. The tangible consequences of this circumstance on the societal level are manifested through limited utility of cognitive ability assessment in clinical settings (as used for diagnoses of cognitive impairment) and selection contexts (as used for personnel recruitment or academic admissions).

These consequences will be addressed by the development of the International Cognitive Ability Resource (ICAR), a public-domain and open-source tool which will incorporate modern item generation and adaptive testing techniques. At its essence, ICAR is a framework for international collaboration. This framework will facilitate the development of a large and dynamic bank of cognitive ability measures for use in a wide variety of applications. By encouraging the use, revision, and ongoing development of these measures among qualified research groups around the world, the ICAR will further understanding about the structure of cognitive abilities as well as the nature of associations between cognitive ability constructs and the life outcomes they predict.

This framework is not without precedent; a similar project has been implemented for use in psychology. The International Personality Item Pool (IPIP) is a large collection of publicly available personality items which was introduced to resolve similar problems with measurement standards. This project has met with considerable success; data collected with IPIP measures have contributed to more than 500 journal articles and book chapters over the last decade. The proposed ICAR is not entirely analogous, mainly because cognitive ability items are fundamentally different from typical, self-report personality items. While the latter are measures of typical behaviour, most intelligence items are intended to assess maximal performance. This distinction makes the development of ICAR considerably more challenging, though it has already been demonstrated that such development is feasible.

The development project proposed herein will result in a resource which contains many types of "item generators". These will be distributed as functions in psychometrically informed, easily implemented, open-source software. Research
groups will be able to use these item generators by downloading the software through the widely-used, open-source statistical software platform, R. As needed, the revision of existing item generators and the development of additional types will be encouraged among qualified international researchers. These item generators will be well suited for large scale, international data collection as might be conducted over the internet.

The primary aim of this research is to integrate recent innovations in test construction, large-scale data collection, and statistical methodology for the purpose of improving the measurement of cognitive ability. This presents a formidable challenge, though one which is well-matched with the collective expertise of the collaborators. The most obvious outcome, of course, will be the creation of an open-source, public-domain tool which will be iteratively improved upon for many years to come. Additional benefits to society-at-large include:
1. better understanding of the ways in which cognitive abilities relate to one another (their internal structure);
2. broader inclusion of cognitive ability assessment in social sciences research generally, including several sub-domains of psychology where ability assessment is often omitted due to the practical limitations posed by extant proprietary
measures;
3. following from the above benefit, more empirical data collection which objectively relates cognitive abilities to a wide variety of life-outcomes and societal goals. Examples include:
(a) broader STEM (science, technology, engineering, and maths) involvement among females and historically under-represented minorities based on the earlier identification of spatial abilities and more simple assessments of the efficacy of STEM interventions;
(b) more tailored treatment of physical and mental health issues based on cognitive ability assessments designed to gauge patients' capacity for self-care (recent research suggests that cognitive abilities/impairments are strong predictors of health outcomes for a variety of chronic diseases and unhealthy behaviours);
(c) better understanding of the ways in which cognitive abilities change over the lifespan, including study of the factors which exacerbate or protect against cognitive ageing;
(d) enhanced well-being resulting from knowledge about the ways in which an individual can be matched to academic and occupational pursuits that are likely to bring personal fulfilment;
4. Most importantly, this research program will lead to the development of an international "collaboratory" platform without precedent in cognitive ability research. The reach of this platform will extend well beyond the collaborators named in this proposal as many research teams have already expressed a strong interest in using the item types and/or developing their own. Given this breadth, it is likely that the most meaningful outcomes will stem from the international cross-fertilization of ideas, the more efficient use of resources across universities and nations, and ultimately the more rapid advancement of knowledge in this field.