Kuala Lumpur, Malaysia, Dec 4, 2023 – Alibaba Cloud has opened the open-sourcing of two large language models (LLMs): Qwen-72B and Qwen-1.8B, variants of the proprietary foundation model Tongyi Qianwen, on ModelScope and the collaborative AI platform Hugging Face, marking a significant advancement in AI technology.
The significance of this development cannot be overstated. Alibaba Cloud has made available an extensive range of LLMs, with parameters ranging from 1.8 billion to 72 billion, alongside innovative multimodal LLMs capable of audio and visual understanding. These models include Qwen-Audio and Qwen-Audio-Chat, which are pre-trained audio understanding models fine-tuned for conversational applications, marking a new era in research and commercial AI use.
Jingren Zhou, CTO of Alibaba Cloud, emphasizes the company’s vision: “Building up an open-source ecosystem is critical to promoting the development of LLM and AI applications building. We aspire to become the most open cloud and make generative AI capabilities accessible to everyone. To achieve that goal, we’ll continue to share our cutting-edge technology and facilitate the development of the open-source community together with our partners.”
The 72-billion-parameter model, pre-trained on over 3 trillion tokens, sets a new standard in AI, outperforming major open-source models across various benchmarks. It showcases advanced capabilities in role-playing, language style transfer, and a variety of complex tasks, heralding a new wave of personalized AI applications like chatbots.
Alibaba Cloud offers the Qwen-72B model’s code, model weights, and documentation for free research use, and at no cost for commercial use to companies with fewer than 100 million monthly active users.
Furthermore, the company has open-sourced the 1.8-billion-parameter version of its LLM, optimized for edge devices. This ‘lightweight’ LLM enables efficient inference on devices with limited computational resources, such as smartphones, presenting a cost-effective option for individuals and organizations.
In addition to text processing LLMs, Alibaba Cloud has introduced Qwen-Audio and Qwen-Audio-Chat. These models boast enhanced audio understanding capabilities, able to process a variety of audio formats and perform tasks like multi-language transcription and emotion detection in speech.
This announcement represents Alibaba Cloud’s ongoing commitment to offering multi-modal large language models to the open-source community. It follows the launch of Qwen-VL and Qwen-VL-Chat, models that understand visual information.
The open-sourced LLM models have already seen significant traction, with over 1.5 million downloads on ModelScope and Hugging Face, contributing to ModelScope’s status as China’s largest AI model community.
For more information on these revolutionary models, visit the ModelScope, Hugging Face, and GitHub pages. Additionally, a demo of Qwen-Audio is available here.
