数据集 / Qwen/QwQ-32B

Qwen/QwQ-32B 已完整同步

QwQ-32B

Introduction

QwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini.

This repo contains the QwQ 32B model, which has the following features:

  • Type: Causal Language Models
  • Training Stage: Pretraining & Post-training (Supervised Finetuning and Reinforcement Learning)
  • Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias
  • Number of Parameters: 32.5B
  • Number of Paramaters (Non-Embedding): 31.0B
  • Number of Layers: 64
  • Number of Attention Heads (GQA): 40 for Q and 8 for KV
  • Context Length: Full 131,072 tokens
    • For prompts exceeding 8,192 tokens in length, you must enable YaRN as outlined in this section.

Note: For the best experience, please review the usage guidelines before deploying QwQ models.

You can try our demo or access QwQ models via QwenChat.

For more details, please refer to our blog, GitHub, and Documentation.

Requirements

QwQ is based on Qwen2.5, whose code has been in the latest Hugging face transformers. We advise you to use the latest version of transformers.

With transformers<4.37.0, you will encounter the following error:

KeyError: 'qwen2'

Quickstart

Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/QwQ-32B"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "How many r's are in the word \"strawberry\""
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=32768
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

Usage Guidelines

To achieve optimal performance, we recommend the following settings:

  1. Enforce Thoughtful Output: Ensure the model starts with "<think>\n" to prevent generating empty thinking content, which can degrade output quality. If you use apply_chat_template and set add_generation_prompt=True, this is already automatically implemented, but it may cause the response to lack the <think> tag at the beginning. This is normal behavior.

  2. Sampling Parameters:

    • Use Temperature=0.6, TopP=0.95, MinP=0 instead of Greedy decoding to avoid endless repetitions.
    • Use TopK between 20 and 40 to filter out rare token occurrences while maintaining the diversity of the generated output.
    • For supported frameworks, you can adjust the presence_penalty parameter between 0 and 2 to r

27 个文件

浏览文件

数据集版权信息

本数据集的许可证为 Apache License 2.0。如有违反相关条款，请联系 WEHUB，我们将及时处理。 查看许可证