Etherll / Mellum-4b-sft-rust

Etherll/Mellum-4b-sft-rust

Etherll/Mellum-4b-sft-rust is a large language model (LLM) fine-tuned specifically for Rust code Fill-in-the-Middle (FIM) tasks. It is built upon JetBrains/Mellum-4b-base model.

This model has been fine-tuned on the Etherll/CodeFIM-Rust-Mellum dataset, which comprises approximately 57,000 Rust-specific FIM examples, to enhance its proficiency in completing Rust code snippets accurately and contextually.

A GGUF version for CPU inference is also available: Etherll/Mellum-4b-sft-rust-GGUF .

Model Description

This model leverages the LLaMA-style architecture of Mellum-4b-base (4 billion parameters) and its extensive pre-training on over 4 trillion tokens. The fine-tuning process focused on adapting the model to the nuances of Rust syntax and common coding patterns for FIM tasks.

Key Features:

• Specialized for Rust: Optimized for Fill-in-the-Middle tasks in Rust.
• Based on Mellum-4b-base: Benefits from JetBrains' robust base model.
• Efficient: Suitable for both cloud and local deployment.
• IDE Integration Ready: Designed for use in developer tooling, and works particularly well with Continue.dev for an enhanced coding assistant experience.

Fine-tuning Data

• Dataset: Etherll/CodeFIM-Rust-Mellum
• Size: ~57,000 rows
• Focus: Rust code Fill-in-the-Middle

FIM Format

This model is trained to recognize a specific format for Fill-in-the-Middle tasks. When providing input for FIM, please use the following structure:

<filename>{{{filename}}}
<fim_suffix>{{{suffix_code}}}<fim_prefix>{{{prefix_code}}}<fim_middle>

How to Use

With Continue.dev

For the best integrated development experience, it's highly recommended to use this model with Continue.dev .

Refer to the Continue.dev documentation for instructions on how to add custom LLMs.

GGUF Version

A GGUF version is available at Etherll/Mellum-4b-sft-rust-GGUF . This format is suitable for local inference on CPU (and GPU with appropriate llama.cpp/Ollama builds) using tools like:

• llama.cpp
• Ollama
• LM Studio

Support & Community

If you need any help, have questions, or just want to chat, feel free to message me on Discord: etherl