Snowflake says its new LLM outperforms Meta’s Llama 3 on half the training

Snowflake, a leading data warehousing cloud vendor, has made a significant advancement in artificial intelligence (AI) by introducing an open-source large language model called Arctic. Snowflake claims that Arctic outperforms Meta’s Llama 3 and offers enhanced efficiency in enterprise tasks like SQL coding for database retrieval while requiring less training compute budget. The company has made Arctic’s parameters and model code available under the Apache 2.0 license, showcasing its commitment to open-source innovation.

Snowflake CEO Sridhar Ramaswamy emphasized the groundbreaking nature of this development, highlighting Snowflake’s AI research team’s cutting-edge work. Arctic’s performance comparisons with industry giants reflect its competitiveness in tasks like textual understanding and reasoning, even with significantly lower compute budgets. Despite not yet publishing a formal paper, Snowflake has provided technical insights on GitHub, revealing an approach that combines a transformer attention-based model with a “mixture of experts” technique to optimize computing and memory resources.

Through its “Dense – MoE Hybrid Transformer,” Arctic integrates a dense transformer model with a mixture of experts, resulting in a powerful AI model with 480 billion total parameters. Snowflake’s dedication to transparency is evident through the publication of blog posts and a comprehensive Arctic Cookbook webpage, offering in-depth discussions on the model’s architecture and training processes.

Furthermore, Snowflake has demonstrated Arctic’s capabilities through a live demo on the Hugging Face repository, showcasing its inference abilities in real-time scenarios. ZDNET’s interaction with Arctic’s AI prompt revealed its proficiency in explaining the differences between the base and instruct variants, highlighting Arctic’s adaptability for instruction following and task-oriented conversations.