Perception of Material and Texture in Vision and Action

There is a vast amount of research on how the visual system processes form and shape. However, much less is known about how object properties, such as surface texture, that indicate important material properties (eg its roughness or softness), are processed by the perceptual system. This is surprising because our perceptual systems recognise these properties with ease, but also surprising because many of our behavioural decisions, how we grasp or interact with an object, depend on the accurate perception of material properties.

Human imaging studies have shown that specific areas within the ventral stream, that are distinct from those processing shape, form and colour, are responsible for texture discrimination (Cant, Arnott, & Goodale, 2009). Yet, it is unknown which visual information is used to estimate the roughness or softness of a material. Recently, Giesel and Zaidi (2013) suggested that the visual system estimates roughness in pictures using monocular/image based cues which are related to the 3D structure of the material.

Taken together, this raises several interesting questions:

1) Are there any differences in how the visual system determines material properties for 2D and 3D stimuli;

2) How is the perceptual estimation of material properties used to inform the visuomotor system and vice versa? and

3) How does the visuomotor system use material properties to adapt its actions accordingly?

To address these questions, we will use a range of techniques, including behavioural reaching and grasping studies, perceptual ratings, computational analysis of material properties and neuropsychology. In the perceptual experiments, parameters that are known to relate to the perceived roughness of a texture (ie height, density, sharpness and pointedness of the bumps) will be varied independently and systematically in both 2D and 3D stimuli and the effects on the perceptual system will be measured in psychophysical experiments. In the visuomotor experiments, the visual and haptic information about texture will be dissociated using a mirror setup such that participants grasp an object (placed behind a mirror) that feels different (eg rougher or smoother) than the object they see (placed in front of a mirror). Perceptual estimates of object properties will be assessed prior to and after object interactions. This will allow us to determine whether material perceptions change depending on the haptic experience. This is important as currently little is known about if and to what extent vision is affected by tactile experience or vice versa. In addition, a neuropsychological study is planned to investigate if a patient with ventral stream damage that is known to be unable to perceive shape and form but has no problems discriminating between different textures and colours (Cavina-Pratesi, Kentridge, Heywood, & Milner, 2009), is able to determine material properties such as rough and soft textures. If these estimations, as suggested, rely on inferring 3D structure (relating to form) from pictorial cues, the patient should be unable to identify specific material properties such as soft or rough for unfamiliar stimuli even when texture discrimination is still possible.

Together these experiments will provide an increased understanding of which cues the perceptual and visuomotor system use to estimate material properties and how this information is processed in the human brain.