Sutra-Instruct-350M
is a custom-built, 350-million parameter causal language model trained using nanaGPT architecture.
🧠 Model Architecture & Details
Architecture:
Custom nanoGPT-based Transformer
Parameter Count:
350M
Format:
safetensors
Embeddings:
Tied (
lm_head
and
wte
share memory)
Creator:
Abhiray
📚 Training Pipeline
This model was not fine-tuned from an existing corporate base model (like Llama or Mistral). Its brain was initialized from absolute zero and trained through a rigorous two-phase pipeline:
Phase 1: Pre-Training (The Foundation)
The base logic was built by streaming a highly curated mix of academic and coding datasets:
HuggingFaceFW/fineweb-edu
(High-level English and academic structure)
open-web-math/open-web-math
(Mathematical logic and formatting)
bigcode/starcoderdata
(Python syntax and code structure)
roneneldan/TinyStories
(Basic grammar and narrative flow)
Phase 2: Supervised Fine-Tuning (SFT)
Once the model learned how to speak, it was fine-tuned using the
yahma/alpaca-cleaned
dataset to teach it the standard
Instruction:
and
Response:
conversational format.
⚙️ Recommended Generation Settings
Because this is a compact 350M parameter model, standard generation settings may result in looping or wild hallucinations. For the absolute best outputs, use the following configuration:
Temperature:
0.5
Top-K:
50
Repetition Penalty:
1.3
Max Length:
400-500
one can use generation_config.json file in repo
⚠️ Limitations & Bias
Hallucinations:
As a 350M parameter model, Sutra does not have the physical parameter count to act as a factual encyclopedia. It will confidently hallucinate historical dates, math solutions, and trivia.
Coding:
While it understands Python syntax and will output beautifully formatted code blocks (thanks to StarCoder), complex logical scripts may fail.
Best Use Case:
Sutra excels at structural formatting, grammar, summarizing provided context, generate short stories, and acting as a lightweight, lightning-fast local testing model.
Runs of Abiray Sutra-Instruct-350M on huggingface.co
0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About Sutra-Instruct-350M huggingface.co Model
Sutra-Instruct-350M huggingface.co is an AI model on huggingface.co that provides Sutra-Instruct-350M's model effect (), which can be used instantly with this Abiray Sutra-Instruct-350M model. huggingface.co supports a free trial of the Sutra-Instruct-350M model, and also provides paid use of the Sutra-Instruct-350M. Support call Sutra-Instruct-350M model through api, including Node.js, Python, http.
Sutra-Instruct-350M huggingface.co is an online trial and call api platform, which integrates Sutra-Instruct-350M's modeling effects, including api services, and provides a free online trial of Sutra-Instruct-350M, you can try Sutra-Instruct-350M online for free by clicking the link below.
Abiray Sutra-Instruct-350M online free url in huggingface.co:
Sutra-Instruct-350M is an open source model from GitHub that offers a free installation service, and any user can find Sutra-Instruct-350M on GitHub to install. At the same time, huggingface.co provides the effect of Sutra-Instruct-350M install, users can directly use Sutra-Instruct-350M installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
Sutra-Instruct-350M install url in huggingface.co: