Innovating Sema: Community-building of Live Coding Language Design and Performance with Machine Learning

The Innovating Sema project will secure the long-term existence and use of our innovative live coding platform for machine learning. After an intense development process during the MIMIC grant (AH/R002657/1) we have developed a potential paradigm-shifting system that empowers users of programming languages for artistic expression to design their own languages for interfacing with machine learning. This has not been done before and through our user studies and workshops we are confident that our system works. Our project has the potential to demystify machine learning through hands-on use, thus generating understanding and trust in modern AI. It enables non-expert users to engage with creative AI. Now is the time to secure the longevity and wide use of the Sema system promoting understanding of computing through the two pillars of language design and machine learning.

The MIMIC project is a direct response to significant changes taking place in the domain of computing and the arts. Recent developments in Artificial Intelligence and Machine Learning lead to a revolution in how music and art is being created. However, these are early days concerning the integration of this new technology into software aimed at creatives. Due to the complexities of machine learning, and the lack of usable tools, such approaches have only been usable by experts. In order to study how we might bring these new fruitful technologies of new art making to the people, we created new, user-friendly technologies. They enable laypeople, both composers as well as amateur musicians, to understand and apply these new computational techniques in their own creative work.

The unexpected potential for impact we discovered during the development of Sema is twofold:

1) We were going to create a high level live coding language for machine learning. However, responding to survey feedback, we decided to generalise and scale our approach by creating a system for *other people to design their own* live coding languages, using our language grammar compilation technology. This makes artists equally the creators of their own tools as well as the creators of artistic work with those tools. By designing and installing our system online and building a community that contributes to its development, we are effectively collaborating with users in filling a musical toolbox with tools and instruments.

2) During the development of Sema, we become attuned to how distant machine learning is to the general public. People do not easily understand new AI and this technology is often seen as a negative force in society (from automation anxiety, filter bubbles, bias, and algorithms pushing people to the political extremes). Our project enables people to explore machine learning, and thereby gain an understanding of how machine learning and artificial neural networks are designed and trained, through hands-on experience. Music is an ideal platform to give people an understanding of machine learning as it is a collaborative, fun and risk-free environment for experimentation (unlike self driving cars or stock market algorithms).

By creating an online resource with our software (including language generator, live coding environment, machine learning architectures and models, and user contributed mini-languages) together with solid documentation and tutorials, we will secure the longevity and impact of our Sema project. Through an extensive user-engagement plan, we will create the conditions for an international community of users to grow independently of our own practice and research. This community will be initially formed by our own workshops, mailing lists, media appearances, social media communication, and dissemination through our own networks of colleagues who teach computer music internationally.