Poetry Beyond Text: Vision, Text and Cognition
Lead Research Organisation:
University of Dundee
Department Name: English
Abstract
Poetry Beyond Text: Vision, Text and Cognition
This research project uses psychological, critical and creative methods to study how readers respond to the visual aspects of poetry. It involves specialists in English and Comparative Literature, Fine Art and Psychology. These include the shape of visual or concrete poetry (where words are arranged spatially in particular patterns on the page), the combination of poetry with images (in artists' books and prints), and the moving words and images found in digital poetry (a relatively new form of poetry which is usually web-based and often interactive).
Psychologists have established typical patterns of eye-movements for reading text and looking at pictures, as well as models of the cognitive processes reflected by such looking behaviour. We are interested in finding out what happens when readers are presented with art works which combine text and images. If the text and image are separate elements, do we look at text or image first? Or do we move rapidly from one to the other? If textual and visual elements are fully integrated, do we adopt a 'reading' or a 'viewing' approach to the text? For example, do we read from top to bottom and left to right, as if we were looking at a more conventionally-arranged poem? Or do we treat the poem as a form of image?
Arising from these questions are more general ones. How are our visual exploration strategies affected by the presentation of the art work and the previous knowledge of the reader? How do different reading strategies influence our assessment of the interest, meaning and aesthetic value of the work? What determines whether the visual aspect enriches the meaning of the words (and vice versa), or whether each element limits the force of the other? For example, sometimes the shape of a poem, or an accompanying image, can seem to determine the subject of the poem, reducing the possibilities of interpretation. In other cases, the two elements can each prompt further interpretative possibilities.
To study these questions, we plan to use a combination of methods from literary criticism, psychology, and creative practice. Literary criticism has developed theories of reader response, including the idea of a 'horizon of expectation' (a set of expectations which conditions how we approach the reading of a text), as well as many accounts of interpretation and the assessment of aesthetic value. Psychological methods include eye-tracking (which measures eye-movement with great precision) and pupil dilation (which relates to the complexity of cognitive processes). We see creative practice, involving both the creation of new works of art, and individual spoken responses to works, as another way of thinking about and understanding these questions. So we will be using discursive methods based on argument, experimental methods based on analysis, and creative methods based on imaginative engagement.. We will assess how reading strategies affect memory, interpretation and perceived aesthetic value, using both quantitative measures and reader-response theories.
We have developed, on the basis of previous work in this area, a strategy of a 'reflective feedback loop', in which participants in experiments are regarded as co-researchers. Their cognitive processes will be assessed, using various experimental methods, while they are reading the various types of poetry, in some cases with modifications of layout. Crucially, these results will also be presented to participants, who will be asked to write their own responses, allowing us to explore their aesthetic experience and interpretation of the poems, before and after receiving such feedback. Furthermore, participants on appropriate degree programmes, together with poets and artists will be invited to create works in response to the investigations. Art works and materials from the project willappear in exhibitions and an on-line gallery, and results will be discussed in journal articles.
This research project uses psychological, critical and creative methods to study how readers respond to the visual aspects of poetry. It involves specialists in English and Comparative Literature, Fine Art and Psychology. These include the shape of visual or concrete poetry (where words are arranged spatially in particular patterns on the page), the combination of poetry with images (in artists' books and prints), and the moving words and images found in digital poetry (a relatively new form of poetry which is usually web-based and often interactive).
Psychologists have established typical patterns of eye-movements for reading text and looking at pictures, as well as models of the cognitive processes reflected by such looking behaviour. We are interested in finding out what happens when readers are presented with art works which combine text and images. If the text and image are separate elements, do we look at text or image first? Or do we move rapidly from one to the other? If textual and visual elements are fully integrated, do we adopt a 'reading' or a 'viewing' approach to the text? For example, do we read from top to bottom and left to right, as if we were looking at a more conventionally-arranged poem? Or do we treat the poem as a form of image?
Arising from these questions are more general ones. How are our visual exploration strategies affected by the presentation of the art work and the previous knowledge of the reader? How do different reading strategies influence our assessment of the interest, meaning and aesthetic value of the work? What determines whether the visual aspect enriches the meaning of the words (and vice versa), or whether each element limits the force of the other? For example, sometimes the shape of a poem, or an accompanying image, can seem to determine the subject of the poem, reducing the possibilities of interpretation. In other cases, the two elements can each prompt further interpretative possibilities.
To study these questions, we plan to use a combination of methods from literary criticism, psychology, and creative practice. Literary criticism has developed theories of reader response, including the idea of a 'horizon of expectation' (a set of expectations which conditions how we approach the reading of a text), as well as many accounts of interpretation and the assessment of aesthetic value. Psychological methods include eye-tracking (which measures eye-movement with great precision) and pupil dilation (which relates to the complexity of cognitive processes). We see creative practice, involving both the creation of new works of art, and individual spoken responses to works, as another way of thinking about and understanding these questions. So we will be using discursive methods based on argument, experimental methods based on analysis, and creative methods based on imaginative engagement.. We will assess how reading strategies affect memory, interpretation and perceived aesthetic value, using both quantitative measures and reader-response theories.
We have developed, on the basis of previous work in this area, a strategy of a 'reflective feedback loop', in which participants in experiments are regarded as co-researchers. Their cognitive processes will be assessed, using various experimental methods, while they are reading the various types of poetry, in some cases with modifications of layout. Crucially, these results will also be presented to participants, who will be asked to write their own responses, allowing us to explore their aesthetic experience and interpretation of the poems, before and after receiving such feedback. Furthermore, participants on appropriate degree programmes, together with poets and artists will be invited to create works in response to the investigations. Art works and materials from the project willappear in exhibitions and an on-line gallery, and results will be discussed in journal articles.
Publications
Otty L
(2012)
'Dim-conceived glories of the brain': On Ambiguity in Literature and Science
in Culture, Theory and Critique
Otty L, Roberts A M
(2012)
'Creative Practice and Experimental Method in Electronic Literature and Human Experimental Psychology'
in dichtung-digital
Roberts A
(2013)
Space and Pattern in Linear and Postlinear Poetry Empirical and theoretical approaches
in European Journal of English Studies
Shaki S
(2012)
Multiple spatial mappings in numerical cognition.
in Journal of experimental psychology. Human perception and performance
Scheepers C
(2013)
Listening to Limericks: a pupillometry investigation of perceivers' expectancy.
in PloS one
Knowles K, Schaffner, A K, Weger U, Roberts AM
(2012)
'Reading Space in Visual Poetry'
in Writing Technologies
Roberts, A.M.
(2017)
'Poetry Objects from the Poetry Beyond Text Project: Text, Form and Texture'
Roberts, A.M.
(2017)
Poetry, Mediatization and New Sensibilities
Title | Big Upon Little |
Description | Installation by artist / poet David Bellingham, in the form of a wall painting at Scottish Poetry Library |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development and promotion of artist's work; also led to collaboration with project postdoctoral researcher on an artist book with interviews ('For the Wall') |
URL | http://www.poetrybeyondtext.org/bellingham.html |
Title | Dark as a Pocket |
Description | Artists' book by Francesca Wilde and Nicola Lee |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of career of artist and poet |
URL | http://www.poetrybeyondtext.org/lee-wilde.html |
Title | Dinner and a Rose |
Description | Sarah Bodman and Nancy Campbell, Dinner and a Rose , Artists' book |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of careers of poet and artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/bodman-campbell.html |
Title | Engine/House |
Description | Lawrence Upton and Guy Begbie, Engine/House, a pop-up Artists' book |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. Development of careers of poet and artist. |
URL | http://www.poetrybeyondtext.org/begbie-upton.html |
Title | Flattened Scores |
Description | Steph Harvey, Flattened Scores, Artists' book |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of career of artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/harvey.html |
Title | Foreshore |
Description | Lawrence Upton and Guy Begbie, 'Foreshore', artists' book |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of careers of poet and artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/upton-begbie.html |
Title | Generosity |
Description | Thomas A. Clark, 'Generosity', printed card with poem |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/clark-thomas.html |
Title | Headline |
Description | Kevin Hutcheson, 'Headline', Fine art print with text |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. Development of career of artist. |
URL | http://www.poetrybeyondtext.org/hutcheson.html |
Title | Lot 76 |
Description | Michael Waight and Jim Carruth, Lot 76, Artists' book in the manner of a folded map, with text |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of careers of poet and artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/carruth-waight.html |
Title | Lyric, Manifest, Forme |
Description | Robert Sheppard and Peter Clarke, Lyric, Manifest, Forme, 3 Fine art prints with text |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of careers of poet and artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/clarke-sheppard.html |
Title | Mellisographia |
Description | Volume of poems by John Burnside in form of artists' book, as collaboration with artist Amy Shelton |
Type Of Art | Creative Writing |
Year Produced | 2011 |
Impact | Development of careers of poet and artist |
URL | http://www.poetrybeyondtext.org/burnside-shelton.html |
Title | One Time In A Tale of Herring |
Description | Will Mclean and Angus Martin, 'One Time In A Tale of Herring', Artists' book |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/martin-mclean.html |
Title | Pibroch |
Description | Marion Leven and Robin Robertson, 'Pibroch', Fine art print with poem. |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of careers of poet and artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/leven-robertson.html |
Title | QR-Comms |
Description | Giselle Beiguelman, 'QR-Comms, an animated sequence' |
Type Of Art | Artefact (including digital) |
Year Produced | 2011 |
Impact | Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/beiguelman.html |
Title | Ransom Note |
Description | Kevin Hutcheson, 'Ransom Note', Fine art print with text |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of career of artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/hutcheson.html |
Title | The Matter Propounded |
Description | David Gatten, 'The Matter Propounded, Of Its Possibility Or Impossibility, Treated In Four Parts', 16mm handmade-film |
Type Of Art | Film/Video/Animation |
Year Produced | 2011 |
Impact | Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/gatten.html |
Title | The Wren's Egg |
Description | Photograph of artefact and poem, collaboration between poet (Deryn Rees-Jones) and artist (Alice Maher) |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of poet's career into new innovative forms. |
URL | http://www.poetrybeyondtext.org/maher-rees-jones.html |
Title | This is Not a Pier |
Description | Sarah Turner, 'This is Not a Pier, a film' |
Type Of Art | Film/Video/Animation |
Year Produced | 2011 |
Impact | Development of careers of poet and artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/turner.html |
Title | Three Little Words |
Description | Murray Robertson and Jim Carruth, Three Little Words, Fine art print with text |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of careers of poet and artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/carruth-robertson.html |
Title | Uncertain Territories |
Description | Mary Modeen and Kathryn Gray, Uncertain Territories, Artists' book |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of careers of poet and artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/gray-modeen.html |
Title | Valdotavvo |
Description | Michael Waight, Valdotavvo, Artists' book |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of careers of poet and artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/waight.html |
Title | Vivam |
Description | Wooden sculpture with text and glass panels. Collaboration between poetr (Deryn Rees-Jones) and sculptor (Marion Smith) |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of career of poet and artist |
URL | http://www.poetrybeyondtext.org/smith-rees-jones.html |
Title | Wish You Were Here |
Description | Robert Hampson and Leena Nammari, Wish You Were Here, Artists' book |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of careers of poet and artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/hampson-namari.html |
Title | Words On a Wall |
Description | Installation by artist / poet David Bellingham, in the form of a wall painting at the Visual Reseach Centre, Dundee Contemporary Arts |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Promotion and development of artist's career |
URL | http://www.poetrybeyondtext.org/bellingham.html |
Title | Written on the Body |
Description | Nancy Gray, Written on the Body, Fine art print with collage and text on handmade paper |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of career of artist. Exhibitions at Scottish Poetry Library, Dundee Contemporary Arts and Royal Scottish Academy. |
URL | http://www.poetrybeyondtext.org/gray.html |
Title | poempondscroll |
Description | Photographic / Textual work in the form of a scroll on Chinese paper, by poet Valerie Gillies and artist Helen Douglas |
Type Of Art | Artwork |
Year Produced | 2011 |
Impact | Development of careers of poet and artist |
URL | http://www.poetrybeyondtext.org/douglas-gillies.html |
Description | - Effects of visual presentation of poetry on readers' aesthetic and evaluative responses. - Creative and analytical interpretations of visual-textual artworks. - Integration of methods of literary criticism, human experimental psychology and practice-based research in fine art and poetry. - Critical evaluation of aesthetic consequences of readers' perceptions of image and text interactions in literary and artistic works. - Dialogue between empirical, theoretical and creative modes of understanding (psychologists, literary critics, art historians, artists, poets) |
Exploitation Route | -Researchers and Practitioners in Poetry and Fine Art have continued to develop collaborations initiated as part of this project, or using project collaborations as a model. - Other researchers in literature, cognitive science and fine art studies have used experimental findings for their research. |
Sectors | Creative Economy Culture Heritage Museums and Collections |
URL | http://www.poetrybeyondtext.org/index.html |
Description | Articles on 'Space And Pattern In Linear And Postlinear Poetry: Empirical And Theoretical Approaches' and 'Reading Space in Visual Poetry: New Cognitive Perspectives' combined psychological testing methods (eye-tracking and interviews) with theoretical investigation to study the cognitive, emotive and aesthetic effects of visual-poetic works. Integrating the empirical methods of human experimental psychology with the critical and evaluative methods of the humanities, the research showed how cognitive processes associated with text-reading and image-viewing interacted in productive or inhibiting ways. Before dissemination in the form of the published articles, the results were shared with the volunteer participants (19 in Dundee; 14 in Kent), contributing to their personal development. Impact generation was built into the AHRC follow-on grant (with the Scottish Poetry Library). AH/1026138/1. The research leading to article 'Creative Practice and Experimental Method in Electronic Literature and Human Experimental Psychology' involved an investigation into convergences between the procedures of experimental psychology and the innovations of digital poets and artists, carried out by a psychologist (Fischer), literary scholars (Roberts, Schaffner and Otty) and digital artists and poets (John Cayley and Simon Biggs). The opening advisers' workshop of the project (May 2009) led to Cayley's innovative on-going digital poetic art-work (The Readers' Project) responding to results of pilot eye-tracking experiments. Cayley describes this work as 'strongly and productively influenced by Poetry Beyond Text', adding that as a result 'The Readers Project has now gained significant attention from the appropriate research community'. Building on this co-production of research with creative practitioners, a small grant from the University of Dundee Humanities Research Fund was used to initiate a collaboration between digital media artist Simon Biggs and Computer Programmer Mark Shovman to create Tower, an immersive 3D textual environment combining visualisation, speech recognition and predictive text algorithms, using the HIVE (Human Interactive Virtual Environment) housed in the University of Abertay, Dundee, and unveiled at the project's concluding conference. The practice-based research recorded in Poetry Beyond Text: Vision, Text and Cognition: Exhibition Catalogue and project website (http://www.poetrybeyondtext.org/) (includes research results, resources for readers, online gallery) of the references involved creative interventions in the practice and careers of 44 poets and artists (in diverse media). The commissioning of collaborative art works advanced creativity by pairing together poets, artists, sculptors and printmakers, with an invitation to engage specifically with the aesthetic, technical and artistic issues arising from the interaction of visual and poetic forms. The experience of collaboration has had substantial continuing effects on individual creative careers, and on the specific media and disciplines concerned. Those involved include well-established and distinguished poets (such as John Burnside, Robin Robertson, Jim Carruth, Thomas A. Clark and Deryn Rees-Jones) and artists (such as Will Mclean, David Bellingham), as well as younger and emerging artists and poets, and some art students, whose careers and profiles have been enhanced through the opportunity to exhibit and to engage in a collective process of creative research. The range was international, including John Cayley, based at Brown University, and Jim Andrews, a Canadian media artist. Discussions of the process of collaborative creation have included public events: Helen Douglas with Valerie Gillies (at the opening of the Dundee exhibition); Deryn Rees-Jones with Marion Smith; Jim Carruth with Murray Robertson and Michael Waight (at the Scottish Poetry Library). These events have shown how collaboration has prompted new insight into formal and thematic aspects of creative practice (Rees-Jones comments that the work 'extended my existing poetic compositions into mixed media form'). The benefits for the commissioned artist and writers also included new contacts and additional opportunities to engage the public. The exhibitions attracted good numbers for the venues: 412 at the Visual Research Centre in Dundee; c. 500 at the Scottish Poetry Library (14 May - 15 July 2011); 1,389 at the Moray Arts Centre (9 Aug - 30 Sep 2011), c. 3000 at the Royal Scottish Academy (12 Nov - 18 Dec 2011). The role of these exhibitions in enhancing cultural life is shown in visitor feedback. The research contributed substantially to the connections, profile, subject interest areas and events programmes of the Scottish Poetry Library. |
First Year Of Impact | 2010 |
Sector | Creative Economy |
Impact Types | Cultural Economic |
Description | 'Tower' Digital Interactive Work |
Organisation | Abertay University |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Commissioning of collaborative digital interactive work, by Professor Simon Biggs (Edinburgh College of Art) and Dr Mark Shovman (University of Abertay) |
Collaborator Contribution | University of Abertay: Provision of HIVE facility (Human Interactive Virtual Environment) for development and exhibition of work. |
Impact | 'Tower' (Interactive Art Work) |
Start Year | 2010 |
Description | Royal Scottish Academy Exhibition |
Organisation | Royal Conservatoire of Scotland |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Commissioning of works, design of exhibition, curation of exhibition |
Collaborator Contribution | Joint curation of exhibition; provision of exhibition space in RSA, publicity, support staff, etc. |
Impact | Exhibition, November - December 2011 |
Start Year | 2011 |
Description | Scottish Poetry Library |
Organisation | Scottish Poetry Library |
Country | United Kingdom |
Sector | Charity/Non Profit |
PI Contribution | Joint creation of exhibition of commissioned works and research outputs. |
Collaborator Contribution | Joint creation of exhibition of commissioned works and research outputs. |
Impact | Poetry Beyond Text Exhibition at Scottish Poetry Library, 13.05.11 - 15.07.11., with opening event on 15.5.11. |
Start Year | 2009 |
Description | Panel Discussion, Poetry and Visuality, STANZA Poetry Festival, 2013 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Panel Discussion, Poetry and Visuality, STANZA Poetry Festival, St Andrews, March 2013 |
Year(s) Of Engagement Activity | 2013 |
Description | Panel Discussion, with Deryn Rees-Jones and Marion Smith, Scottish Poetry Library, 2011 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Public/other audiences |
Results and Impact | Panel Discussion, with Deryn Rees-Jones (poet) and Marion Smith (sculptor), Scottish Poetry Library, May 2011 |
Year(s) Of Engagement Activity | 2011 |
Description | Panel Discussion: 'Poetry Breakfast: Watch this Space', STANZA Poetry Festival 2012 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Panel Discussion: 'Poetry Breakfast: Watch this Space', STANZA Poetry Festival, Byre Theatre, St Andrews,16 March 2012 |
Year(s) Of Engagement Activity | 2012 |