cyberagent / CAT-Thinking-8B

CAT-Thinking 🐱

Tiny Language Model that thinks in Japanese

CAT-Thinking is trained to generate reasoning trace in Japanese by reinforcement learning. The model is based on Qwen3-Swallow-v0.2 which is a continual pretraining model based on Qwen3 to read and write fluently in Japanese.

Usage

CAT-Thinking is designed to reason in Japanese even if the input text is in English. The model is trained with the maximum output token length of 4096. We recommend setting max_new_tokens to at least 4096, and larger for difficult problems. Although the model is trained to respond within 4096 tokens, it tends to generate longer responses, especially for difficult and/or confusing instructions. It often gets stuck in repetition, especially when the instruction is confusing (e.g., two contradicting instructions are given). To mitigate the probability of repetition, we find repetition_penalty=1.05 or larger to be useful.

The model is compatible with all the libraries Qwen-3-8B is. To run with transformers, run the following command.

pip install transformers

from transformers import pipeline

# Load the model
chat_pipeline = pipeline("text-generation", model="CyberAgent/CAT-Thinking-8B")

prompt = "You have two cats, one male and one female. A female cat gives birth to up to 12 kittens per year.\n" + \
  "Assume you don't spay them. In three years, how many cats might you need to take care of at most?"

user_input = [{"role": "user", "content": prompt}]

response = chat_pipeline(user_input, max_new_tokens=8192, temperature=0.8, top_p=0.95)

print(response[0]['generated_text'])

Evaluation

We conducted evaluation on coding and math tasks in Japanese and English. We compare CAT-Thinking with its counterparts, Qwen-3-8B (Qwen-3) and Qwen3-Swallow-8B-RL-v0.2 (Swallow) Random sampling (temperature=0.8, top_p=0.95, max_new_tokens=4096) is used for all runs.

• mbpp
• HumanEval
• JHumanEval
• LiveCodeBenchv6
• GPQA Main (En)
• GPQA Main (Ja)
• PolyMath (En)
• PolyMath (Ja)
• AIME 26

For all instances, CAT-Thinking generated its reasoning trace in Japanese while the comparison models generated theirs in English. The results show that it is feasible to think in Japanese while retaining the model's performance on English tasks. The benchmarks were published before the models were released, thus they should be interpreted as reference points.

A detailed experimental evaluation will be presented in a technical report.

Generation Examples

The reasoning trace of CAT-Thinking is in Japanese but not a natural one. In particular, it tends to start with a weird phrase such as:「済みの問題を解くことは許可されていますか？」「済みの数学問題かを確認する必要がありますか？」. It occurred during the second GRPO training and the model ended up learning to start with this phrase.

import math

def prime_num(num: int) -> bool:
    """
    与えられた整数が素数かどうかを判定して bool を返す関数。

    Parameters
    ----------
    num : int
        判定したい整数。負の数や 0, 1 は素数ではないとみなす。

    Returns
    -------
    bool
        素数であれば True、そうでなければ False。
    """
    # 1 以下の数は素数ではない
    if num <= 1:
        return False

    # 2 は唯一の偶数の素数
    if num == 2:
        return True

    # 偶数は 2 以外すべて合成数
    if num % 2 == 0:
        return False

    # 3 から√num までの奇数で割り切れないか調べる
    limit = int(math.isqrt(num))   # 整数平方根を取得（Python3.8 以降）
    for i in range(3, limit + 1, 2):
        if num % i == 0:
            return False

    # ここまで来たら割り切れる約数が無かった → 素数
    return True

print(prime_num(2))   # True
print(prime_num(15))  # False
print(prime_num(29))  # True
print(prime_num(1))   # False

Training Procedure

The model is trained with GRPO with a warm-start. We first generate a teacher dataset using gpt-oss-120b as a reference. Since the reasoning traces are in English, we translate them into Japanese using CAT-Translate-7b . We train the Swallow model using the generated synthesized dataset with full-parameter SFT.

Then, we run GRPO with a permissive reward model which gives partial rewards for being able to (1) follow the reasoning format, (2) generate reasoning trace and the main text in Japanese, and (3) answer the question in an instructed format. In this way, the model learns to follow the reasoning format and generate its reasoning trace in Japanese. Since this training phase focuses on learning the superficial format rather than reasoning competence itself, we use LoRA.

Finally, we train the model with GRPO using a strict reward model that gives a reward only if the model follows all format constraints and also generates the correct answer.

The training data consists of synthesized math and coding dataset generated by gpt-oss-120b. Most of the instructions used for the training are in English. We speculate this to be the reason why the model underperforms on Japanese benchmarks.

License

The model is licensed under the Apache 2.0 License .

Citation

@misc{cat-thinking-2026,
  title={CAT-Thinking: How Hard is it to Learn Reasoning in non-English Language? Case Study in Japanese},
  author={Yuu Jinnai},
  year={2026},
  url={https://huggingface.co/collections/cyberagent/cat-thinking-8b}
}