Sonic Phonic - Towards a Generative Alphabet
Lead Research Organisation:
University of Dundee
Department Name: Duncan of Jordanstone College
Abstract
The concept of an association between sound and image can be traced back to Greek philosophers Pythagoras and Aristotle who speculated on the correlation between musical scale and the rainbow spectrum. From traditional art studio practise (drawing, painting, montage) to the beginnings of media art, the concept of image, sound and text relationships have been explored by artists and writers. Periods of technological change such as advances in audio-visual media
technology at the turn of the
19th Centuryand digital revolution of the 21st Century have influenced advances in concepts
such as in avant-garde film and music in the 1920's, video art and new music in the 1960's and currently digital media art, digital narrative, artificial intelligence and computational processing. The rapid development and spread of new technologies has effected change in many spheres of human experience, computers have created and are transforming culture, motivating simultaneous widespread innovation in artistic and scientific research. The convergence of interactive technologies and digital environment has created expanded capacities and possibilities for the presentation of language, text and sound media. It has enabled a new relationship between the viewer and the interface, between the senses and the different modes of visual, aural and tactile perception. Recent research by Neuroscientist Antonio Damasio concerning the brain's preconscious engagement with images and perceptual cognition in relation to meaning has informed developments in the area of the perception of image, text and sound, which has the potential to lead towards a shift in our perceptual environment and possibility our cognitive evolution.
This research arises from converging interests in my practice as an artist relating to artworks exploring concepts of language, communication and visual perception and issues relating to the juxtaposition of media such as photography, digital imaging, printmaking and video in installation and multimedia design. Currently I have a particular interest in evolutionary computational processes that allow juxtaposition and layering of such concepts and objects within a system. It is these computationally blended conceptual and visual spaces that has influenced the concept of a cross fertilization of alphabetic media and the possibility of emergent audio/visual ideas for new constructs in the alphabet. The research questions have arisen through personal experience of children's' engagement with correlations made between images and reality, symbols and sound. The Alphabet, with its historical association with pictograms and ideograms and subsequent evolution to the indexing of letter symbol (Grapheme)-phonic sound (Phoneme) relationships, provides a context for digital and interactive media to re-frame the symbol-sound relationship by expanding the possibilities for the synesthetic juxtaposition of image, sound and text as well as moving image, animation and drawing.
Aims and Objectives
The aim of the research is the creation of an interactive alphabetic prototype to consider the development and emergent characteristics and properties of digital (image, sound and text) relationships, using the architecture of a generative database at the heart of the installation.
The objective of the interactive alphabet prototype is to facilitate, through the installation of the prototype artwork, the interaction of invited participants with the evolving generative alphabetic space and gather data from these experimental interactions to evaluate the behaviours of participant and database.
Potential Applications and Benefits
The Research will contribute to existing knowledge of
Concepts in image, text and sound correlation
The potential for technological change to influence the cognition and communication of culture
Explorations of the alphabet by artists and writers
Artworks using new interactive interfaces, databases and computational processing.
Benefits might include an intermedia approach to constructs of digital alphabets where future potential for image sound and text as a generative system may lead to new meaning and relationships in alphabets and narrative.
Practical applications for long-term research include online adaptive human-machine alphabetic databases for digital writing and narrative, education, language applications, entertainment, and contributions to the development of understanding in e-learning environments. With that potential, it is likely that future research will develop the prototype for online publication where adaptations may include the ability for uploading of new content including imagery, video, sound and text based media. New constructs could be developed through an online evolving collaborative alphabet or alternatively creation of individual bespoke alphabets/language for a range of applications such as for dysphasic stroke patients, where there is potential for research to examine the use of personal imagery, sounds and artefacts in conjunction with the database to trigger the re-learning of language. Through my background research process I have had discussions with psychologists who are interested in developing the potential of sonic phonic for this purpose. Other possibilities for further research lie in the relevance of kinesthetic haptic interface to the development of the generative database and digital content.
technology at the turn of the
19th Centuryand digital revolution of the 21st Century have influenced advances in concepts
such as in avant-garde film and music in the 1920's, video art and new music in the 1960's and currently digital media art, digital narrative, artificial intelligence and computational processing. The rapid development and spread of new technologies has effected change in many spheres of human experience, computers have created and are transforming culture, motivating simultaneous widespread innovation in artistic and scientific research. The convergence of interactive technologies and digital environment has created expanded capacities and possibilities for the presentation of language, text and sound media. It has enabled a new relationship between the viewer and the interface, between the senses and the different modes of visual, aural and tactile perception. Recent research by Neuroscientist Antonio Damasio concerning the brain's preconscious engagement with images and perceptual cognition in relation to meaning has informed developments in the area of the perception of image, text and sound, which has the potential to lead towards a shift in our perceptual environment and possibility our cognitive evolution.
This research arises from converging interests in my practice as an artist relating to artworks exploring concepts of language, communication and visual perception and issues relating to the juxtaposition of media such as photography, digital imaging, printmaking and video in installation and multimedia design. Currently I have a particular interest in evolutionary computational processes that allow juxtaposition and layering of such concepts and objects within a system. It is these computationally blended conceptual and visual spaces that has influenced the concept of a cross fertilization of alphabetic media and the possibility of emergent audio/visual ideas for new constructs in the alphabet. The research questions have arisen through personal experience of children's' engagement with correlations made between images and reality, symbols and sound. The Alphabet, with its historical association with pictograms and ideograms and subsequent evolution to the indexing of letter symbol (Grapheme)-phonic sound (Phoneme) relationships, provides a context for digital and interactive media to re-frame the symbol-sound relationship by expanding the possibilities for the synesthetic juxtaposition of image, sound and text as well as moving image, animation and drawing.
Aims and Objectives
The aim of the research is the creation of an interactive alphabetic prototype to consider the development and emergent characteristics and properties of digital (image, sound and text) relationships, using the architecture of a generative database at the heart of the installation.
The objective of the interactive alphabet prototype is to facilitate, through the installation of the prototype artwork, the interaction of invited participants with the evolving generative alphabetic space and gather data from these experimental interactions to evaluate the behaviours of participant and database.
Potential Applications and Benefits
The Research will contribute to existing knowledge of
Concepts in image, text and sound correlation
The potential for technological change to influence the cognition and communication of culture
Explorations of the alphabet by artists and writers
Artworks using new interactive interfaces, databases and computational processing.
Benefits might include an intermedia approach to constructs of digital alphabets where future potential for image sound and text as a generative system may lead to new meaning and relationships in alphabets and narrative.
Practical applications for long-term research include online adaptive human-machine alphabetic databases for digital writing and narrative, education, language applications, entertainment, and contributions to the development of understanding in e-learning environments. With that potential, it is likely that future research will develop the prototype for online publication where adaptations may include the ability for uploading of new content including imagery, video, sound and text based media. New constructs could be developed through an online evolving collaborative alphabet or alternatively creation of individual bespoke alphabets/language for a range of applications such as for dysphasic stroke patients, where there is potential for research to examine the use of personal imagery, sounds and artefacts in conjunction with the database to trigger the re-learning of language. Through my background research process I have had discussions with psychologists who are interested in developing the potential of sonic phonic for this purpose. Other possibilities for further research lie in the relevance of kinesthetic haptic interface to the development of the generative database and digital content.
