ilsp / Llama-Krikri-8B-Instruct-v1.5

🚨 There is no guarantee that you are using the latest improved versions from 3rd party quantizations as the model's weights are getting reuploaded! 🚨

Llama-Krikri-8B-Instruct-v1.5: An Instruction-tuned Large Language Model for the Greek language

Following the release of Meltemi-7B on the 26th March 2024, we are happy to welcome Krikri to the family of ILSP open Greek LLMs. Krikri is built on top of Llama-3.1-8B , extending its capabilities for Greek through continual pretraining on a large corpus of high-quality and locally relevant Greek texts. We present Llama-Krikri-8B-Instruct , along with the base model, Llama-Krikri-8B-Base

Llama-Krikri-8B-Instruct-v1.5 is the chat-focused variant of Llama-Krikri-8B-Instruct

Model Information

Base Model

• Vocabulary extension of the Llama-3.1 tokenizer with Greek tokens
• 128k context length (approximately 80,000 Greek words)
• We extend the pretraining of Llama-3.1-8B with added proficiency for the Greek language, by utilizing a large training corpus.
  • This corpus includes 56.7 billion monolingual Greek tokens, constructed from publicly available resources.
  • Additionaly, to mitigate catastrophic forgetting and ensure that the model has bilingual capabilities, we use additional sub-corpora with monolingual English texts (21 billion tokens) and Greek-English parallel data (5.5 billion tokens).
  • The training corpus also contains 7.8 billion math and code tokens.
  • This corpus has been processed, filtered, and deduplicated to ensure data quality and is outlined below:

Chosen subsets of the 91 billion corpus were upsampled resulting in a size of 110 billion tokens .

Instruct Model

Llama-Krikri-8B-Instruct-v1.5 is the result of post-training Llama-Kriki-8B-Base and features:

• Enhanced chat capabilities and instruction-following in both Greek and English.
• Document translation from Greek to English, French, German, Italian, Portuguese, Spanish and vice versa.
• Great performance on generation, comprehension, and editing tasks, such as summarization, creative content creation, text modification, entity recognition, sentiment analysis, etc.
• Domain-specifc expertise for specialized legal, financial, medical, and scientific applications.
• Retrieval-Augmented Generation (RAG) utilizing multiple documents with 128k context length.
• Improved coding and agentic capabilities with correct formatting and tool use.
• Conversion or structured extraction (e.g., XML, JSON) in data-to-text & text-to-data settings.
• Analytical thinking and Chain-of-Thought (CoT) reasoning for problem-solving.

Post-training Methodology

We used a multi-stage process in order to build Llama-Krikri-8B-Instruct-v1.5 which includes:

• 2-stage Supervised Fine-Tuning with a combination of Greek & English instruction-response pairs (& multi-turn conversations)
  • Stage 1 : 856,946 instruction-response pairs (371,379 Greek + 485,567 English)
  • Stage 2 : 638,408 instruction-response pairs (279,948 Greek + 358,460 English)
• Alignment with a combination of Greek & English preference triplets (Instruction - Chosen Response - Rejected Response)
  • Stage 1 Length Normalized DPO : 92,394 preference triplets (47,132 Greek + 45,262 English)
  • Stage 2 Length Normalized DPO : Pseudo On-policy sampled and scored preference triplets

Post-training Data Construction

To build the SFT & DPO data, we utilized various methodologies including:

• Collecting existing high-quality datasets such as Tulu 3 , SmolTalk , MAGPIE Ultra , Orca Agent Instruct , IFEval Like Data , UltraFeedback , NVIDIA HelpSteer2 , Intel Orca , UltraMedical , and other datasets focused on safety, truthfulness, and instruction-following.
• Translating various data into Greek using an in-house translation tool.
• Regenerating translated data and contrasting the translated with the regenerated responses (i.e., for creating preference triplets).
• Distilling (with the MAGPIE methodology) models which exhibit strong performance in Greek, such as Gemma 2 27B IT .
• Scoring data with the Skywork Reward Gemma 2 27B v0.2 Reward Model and filtering using rule-based filters.
• Creating data for sentence and document translation using high-quality parallel corpora mainly from ELRC-SHARE .
• Synthetically extracting question-answer pairs and multi-turn dialogues from diverse sources such as Wikipedia, EUR-LEX, Greek School Books, and Kallipos.

How to use

With Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cuda"

model = AutoModelForCausalLM.from_pretrained("ilsp/Llama-Krikri-8B-Instruct-v1.5")
tokenizer = AutoTokenizer.from_pretrained("ilsp/Llama-Krikri-8B-Instruct-v1.5")

model.to(device)

system_prompt = "Είσαι το Κρικρί, ένα εξαιρετικά ανεπτυγμένο μοντέλο Τεχνητής Νοημοσύνης για τα ελληνικα και εκπαιδεύτηκες από το ΙΕΛ του Ε.Κ. \"Αθηνά\"."
user_prompt = "Σε τι διαφέρει ένα κρικρί από ένα λάμα;"

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_prompt},
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
input_prompt = tokenizer(prompt, return_tensors='pt').to(device)
outputs = model.generate(input_prompt['input_ids'], max_new_tokens=256, do_sample=True)

print(tokenizer.batch_decode(outputs)[0])

With OpenAI compatible server via vLLM

vllm serve ilsp/Llama-Krikri-8B-Instruct-v1.5 \
  --enforce-eager \
  --dtype 'bfloat16' \
  --api-key token-abc123

Then, the model can be used through Python using:

from openai import OpenAI

api_key = "token-abc123"
base_url = "http://localhost:8000/v1"

client = OpenAI(
    api_key=api_key,
    base_url=base_url,
)

system_prompt = "Είσαι ένα ανεπτυγμένο μεταφραστικό σύστημα που απαντάει με λίστες Python. Δεν γράφεις τίποτα άλλο στις απαντήσεις σου πέρα από τις μεταφρασμένες λίστες."
user_prompt = "Δώσε μου την παρακάτω λίστα με μεταφρασμένο κάθε string της στα ελληνικά: ['Ethics of duty', 'Postmodern ethics', 'Consequentialist ethics', 'Utilitarian ethics', 'Deontological ethics', 'Virtue ethics', 'Relativist ethics']"

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_prompt},
]

response = client.chat.completions.create(model="ilsp/Llama-Krikri-8B-Instruct-v1.5",
                                          messages=messages,
                                          temperature=0.0,
                                          top_p=0.95,
                                          max_tokens=8192,
                                          stream=False)

print(response.choices[0].message.content)
# ['Ηθική καθήκοντος', 'Μεταμοντέρνα ηθική', 'Συνεπειοκρατική ηθική', 'Ωφελιμιστική ηθική', 'Δεοντολογική ηθική', 'Ηθική αρετών', 'Σχετικιστική ηθική']

Evaluation

In the table below, we report the scores for our chat evaluation suite which includes:

• Greek IFEval (strict average)
• English IFEval (strict average)
• Greek MT-Bench using gpt-4o-2024-08-06 as the judge model.
• English MT-Bench using gpt-4o-2024-08-06 as the judge model.

Llama-Krikri-8B-Instruct-v1.5 is released as an alternative to the already existing Llama-Krikri-8B-Instruct exhibiting higher performance in Greek chat benchmarks, while Llama-Krikri-8B-Instruct exhibits the strongest performance in instruction following for both Greek and English across all the models we tested.

We also used the Arena-Hard-Auto automatic evaluation tool, as well the translated (and post-edited) version for Greek that is publicly available here . We report 2 scores for Arena-Hard-Auto:

• No Style Control: The original version of the benchmark.
• With Style Control: The benchmark with style control methods for Markdown elements. You can read more about the methodology and technical background in this blogspot .

As Llama-Krikri-8B-Instruct-v1.5 is focused on improving the model's Greek chat capabilities, below, we show the scores for the Greek version of Arena-Hard-Auto for various open and closed chat models that were determined using Sonnet 3.7 as the judge model * and gpt-4o-mini-2024-07-18 as the baseline model (i.e., by default 50% score).

Llama-Krikri-8B-Instruct-v1.5 exhibits very strong chat capabilities by scoring higher than larger and highly-performant multilingual open-source models and closed-source models , such as GPT-4o-Mini , Mistral Small 3.1 24B , and Llama 4 Scout 109B A16B .

* Please note that judge models are biased towards student models trained on distilled data from them. You can read more here .

🚨 More information on post-training, methodology, and evaluation coming soon. 🚨

Acknowledgements

The ILSP team utilized Amazon's cloud computing services, which were made available via GRNET under the OCRE Cloud framework , providing Amazon Web Services for the Greek Academic and Research Community.

Citation

@misc{roussis2025krikriadvancingopenlarge,
      title={Krikri: Advancing Open Large Language Models for Greek}, 
      author={Dimitris Roussis and Leon Voukoutis and Georgios Paraskevopoulos and Sokratis Sofianopoulos and Prokopis Prokopidis and Vassilis Papavasileiou and Athanasios Katsamanis and Stelios Piperidis and Vassilis Katsouros},
      year={2025},
      eprint={2505.13772},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.13772}, 
}