Supports 380,000 words input at a time! Tencent Hunyuan launches 256k long article model, open to enterprises and individual developers through Tencent Cloud​

AILarge model technology is becoming a key force in promoting the development of high-quality productivity and plays an important role in the integration with thousands of industries. Tencent's Hunyuan large model has expanded the model to trillions of parameter scale by adopting the hybrid expert model (MoE) structure, adding "Brain” Capacity not only improves prediction performance, but also drives down inference costs. As a general model, Tencent Hunyuan leads the industry in Chinese performance, especially in text generation, mathematical logic and multi-turn dialogue.

Recently, Tencent Hunyuan large model officially released the 256k long article model, and made it available to the majority of enterprises and individual developers through Tencent Cloud Open to support wider innovation and applications. Tencent Hunyuan256k model version has the ability to process ultra-long text exceeding 38 million characters. In dialogue application scenarios, this model can"memory"more dialogue content, effectively avoiding "Forgot" information and other issues. In addition, it has excellent contextual analysis capabilities to provide more precise and relevant feedback to conversation participants, helping them make more informed decisions.

In addition, this model version also shows strong performance in reading comprehension of long documents and large-scale data analysis. It can provide strong work support for professionals in finance, medical, education, travel and other industries, and significantly improve their work efficiency. The model has also been deeply optimized in terms of inference performance, ensuring that users can enjoy a smoother and more efficient experience in actual applications on platforms such as Tencent Cloud.

##Reduce "forgetfulness" and make large models smarter

In large model products, handling conversational requirements is a core function. However, due to the limitations of long text processing capabilities, traditional large models are prone to "losing direction" or "Memory Loss" As the length of the conversation increases, the amount of forgotten information also increases.

Tencent Hunyuan256k model is specially optimized for this challenge. It adopts an advanced"Expert Hybrid"(MoE) architecture, And it integrates innovative technologies such as RoPE-NTK and Flash Attention V2, while maintaining the ability to support general short texts (less than 4,000 characters), while achieving a breakthrough in the depth and breadth of long text processing.

Currently, Tencent Hunyuan’s large model has the ability to understand 256k of ultra-long context in a single process The number of characters exceeds 38. After rigorous "finding a needle in a haystack"After task testing, the model’s accuracy in long text processing has reached 99.99%, which is also in a leading position internationally.

Continuous and stable iteration, the efficiency of large model application is improved

Tencent Hunyuan Large Model is the first in the industry to adopt the hybrid expert model (MoE) structure, and has accumulated a large number of self-developed technologies in the process. In the previous version 32K, this model has significantly surpassed similar open source models on the market and demonstrated excellent performance in a variety of application scenarios.

After a new iteration, Tencent Hunyuan256k#GSB evaluation in the general field , compared to the previous version, the winning rate is 50.72%. At the same time, the training set of Tencent Hunyuan256k integrates high-quality annotated data such as long text data, translation data, and multi-document question and answer data in medical, financial and other fields, which makes the model In practical applications, especially in the medical and financial industries that require frequent analysis and processing of large amounts of long text data, it can provide more accurate and efficient work support.

For example, when a financial report issued by the central bank is input into the Tencent Hunyuan256k# model, the model can quickly refine and summarize The main points of the report were processed to a satisfactory level in terms of speed and accuracy.

##Inference performance optimization, bringing stronger large models Comprehension

At the same time, Tencent Hunyuan256k has made in-depth optimization on inference performance. Model's QPM## compared to FP16 accuracy in INT8 accuracy mode # (query rate per second) achieved a significant improvement of 23.9%, while the first word time only increased by 5.7% . These improvements significantly enhance the model's responsiveness and overall efficiency in real-world applications.