10060709Measuring Theme Success: QA approaches to user-led NLP trainingClosedCollaborative R&DInnovate UKWordnerds is a SaaS platform offering a unique approach to text analytics, used by large organisations to derive value from their Voice of Customer programmes. We have attracted high profile customers across the retail, transport and housing sectors through our USP of 'context themes': allowing users to train their own AI-based categories on meaning rather than keywords. As a start-up with a small team we've stayed at the cutting edge of text analytics by taking an agile approach to product development - delivering value quickly and moving on to the next feature to stay ahead of customer needs. Our context themes are unique and effective but we feel now is the time to refine the user experience of training them, to make them easier to use, measure and prove. Working with the experts at TUV SUD National Engineering Laboratory (known as NEL) we will be undertaking research and development to: * Provide externally-validated benchmark metrics for the Wordnerds categorisation technology (including the number of samples required to reach an acceptable accuracy level, so that users know when 'training' is complete; and any variation across different types of categorisation - e.g. emotional, motivational, or topical categories). * Assess the potential for alternative data science approaches to enhance our categorisation process, e.g. SeTFT architecture or an ensemble model approach, by building and testing proof of concept software. * Explore ways of illustrating category accuracy in a visually impactful way. We hope to validate and implement a combination of measures that will markedly improve the user experience and confidence of our customers. This will in turn lead to more effective use of insights from our platform, and greater return on investment.