tencent / Youtu-LLM-2B

🎯 Brief Introduction

Youtu-LLM is a new, small, yet powerful LLM, contains only 1.96B parameters, supports 128k long context, and has native agentic talents. On general evaluations, Youtu-LLM significantly outperforms SOTA LLMs of similar size in terms of Commonsense, STEM, Coding and Long Context capabilities; in agent-related testing, Youtu-LLM surpasses larger-sized leaders and is truly capable of completing multiple end2end agent tasks.

Youtu-LLM has the following features:

• Type: Autoregressive Causal Language Models with Dense MLA
• Release versions: Base and Instruct
• Number of Parameters: 1.96B
• Number of Layers: 32
• Number of Attention Heads (MLA): 16 for Q/K/V
• MLA Rank: 1,536 for Q, 512 for K/V
• MLA Dim: 128 for QK Nope, 64 for QK Rope, and 128 for V
• Context Length: 131,072
• Vocabulary Size: 128,256

📊 Performance Comparisons

Instruct Model

General Benchmarks

Agentic Benchmarks

🚀 Quick Start

This guide will help you quickly deploy and invoke the Youtu-LLM-2B model. This model supports "Reasoning Mode", enabling it to generate higher-quality responses through Chain of Thought (CoT).

---

1. Environment Preparation

Ensure your Python environment has the transformers library installed and that the version meets the requirements.

pip install "transformers>=4.56" torch accelerate

---

2. Core Code Example

The following example demonstrates how to load the model, enable Reasoning Mode, and use the re module to parse the "Thought Process" and the "Final Answer" from the output.

import re
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# 1. Configure Model
model_id = "tencent/Youtu-LLM-2B"

# 2. Initialize Tokenizer and Model
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    trust_remote_code=True
)

# 3. Construct Dialogue Input
prompt = "Hello"
messages = [{"role": "user", "content": prompt}]

# Use apply_chat_template to construct input; set enable_thinking=True to activate Reasoning Mode
input_ids = tokenizer.apply_chat_template(
    messages, 
    tokenize=True, 
    add_generation_prompt=True, 
    return_tensors="pt",
    enable_thinking=True
).to(model.device)

# 4. Generate Response
outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    do_sample=True,
    temperature=1.0,
    top_p=0.95,
    repetition_penalty=1.05
)

# 5. Parse Results
full_response = tokenizer.decode(outputs[0], skip_special_tokens=True)

def parse_reasoning(text):
    """Extract thought process within <think> tags and the subsequent answer content"""
    thought_pattern = r"<think>(.*?)</think>"
    match = re.search(thought_pattern, text, re.DOTALL)
    
    if match:
        thought = match.group(1).strip()
        answer = text.split("</think>")[-1].strip()
    else:
        thought = "(No explicit thought process generated)"
        answer = text
    return thought, answer

thought, final_answer = parse_reasoning(full_response)

print(f"\n{'='*20} Thought Process {'='*20}\n{thought}")
print(f"\n{'='*20} Final Answer {'='*20}\n{final_answer}")

---

3. Key Configuration Details

Reasoning Mode Toggle

Controlled via the enable_thinking parameter in the apply_chat_template method:

• True (Recommended Default): Activates Chain of Thought; ideal for complex logic and reasoning tasks.
• False: Outputs results directly; faster response time, suitable for simple conversations.

Recommended Decoding Parameters

Depending on your use case, we suggest adjusting the following hyperparameters for optimal generation:

Tip: When using Reasoning Mode, a higher temperature helps the model perform deeper, more divergent thinking.

---

4. vLLM Deployment

We provide support for deploying the model using vLLM 0.10.2 . The recommended Docker image is vllm/vllm-openai:v0.10.2 .

Integration Steps

First, execute the following commands to integrate the Youtu-LLM model files into the vLLM framework. Note: Please extract our provided modified vllm zip file first. Then, replace <local_modified_vllm_path> with the path to the extracted vllm directory, and replace <vllm_path> with the installation path of vLLM.

cp <local_modified_vllm_path>/0_10_2_official/youtu_llm.py <vllm_path>/vllm/model_executor/models/youtu_llm.py
cp <local_modified_vllm_path>/0_10_2_official/configuration_youtu.py <vllm_path>/vllm/model_executor/models/configuration_youtu.py
cp <local_modified_vllm_path>/0_10_2_official/__init__.py <vllm_path>/vllm/config/__init__.py
cp <local_modified_vllm_path>/0_10_2_official/registry.py <vllm_path>/vllm/model_executor/models/registry.py

Service Startup

Once integrated, you can deploy the model using the following command:

vllm serve <model_path> --trust-remote-code

Tool Call Support: To enable tool calling capabilities, please append the following arguments to the startup command:

--enable-auto-tool-choice --tool-call-parser hermes

---

💡 Highlights

Showcase of General Tasks

Showcase of Specialized Tasks

Note : For specialized tasks, in-domain post-training is further applied.