Largest text-to-speech AI model yet shows ’emergent abilities’

Amazon researchers have developed the largest text-to-speech model to date, claiming it exhibits improved natural speech abilities for complex sentences, potentially helping this technology move past the uncanny valley. The model, known as BASE TTS, uses 100,000 hours of public domain speech data in multiple languages, boasting 980 million parameters for the largest version.

The researchers aimed to observe a leap in the model’s capabilities as it grew in size, similar to what occurred with language models. The team found that as the text-to-speech model expanded, it displayed emergent abilities in handling various complexities, such as compound nouns, emotions, foreign words, paralinguistics, punctuations, questions, and syntactic complexities. These challenges are typically stumbling blocks for traditional text-to-speech engines, but BASE TTS demonstrated significant improvement in addressing them.

The model’s medium-sized version exhibited the desired performance jump, primarily in emergent abilities rather than overall speech quality. While still an experimental model, the research suggests that the model’s size and training data contribute to its capacity to handle linguistic challenges effectively.

BASE TTS’s unique feature is its “streamable” nature, allowing it to generate speech moment by moment at a low bitrate. Additionally, the team has incorporated speech metadata like emotionality and prosody into a separate stream for efficient delivery. Despite the promising advancements, the researchers have opted not to disclose the model’s source code and data to prevent misuse, acknowledging potential risks from malicious actors in the future.

With text-to-speech technology potentially reaching a turning point in 2024, this advancement holds notable implications for accessibility and communication. Despite the model’s current experimental status, future research will focus on identifying the inflection point for emergent abilities and optimizing the training and deployment processes. As the technology evolves, the practical applications and impact of BASE TTS could be significant, particularly in enhancing user experiences and accessibility features.