CMSManhattan / JiRack_GPT3_8b

Creating a 8b-parameter LLM from Scratch in Google Colab

This guide shows how to train/create a ~8b parameter Llama-style GPT model from scratch using free Google Colab resources. I took from 1b as teplate as readme file

Step-by-Step Instructions

1. Open Google Colab
   Go to → https://colab.research.google.com
   Sign in with your Google account (or create one).

2. Open Google Drive
   Go to → https://drive.google.com
   Make sure you are logged in with the same Google account.

3. Upload the Python script(s)

   • Download the training script (e.g., gpt_jit_modern_8b.py and any required files).
   • Upload them to your Google Drive → MyDrive folder (root).

4. Open a new Colab notebook
   In Colab, click File → Open notebook → Google Drive and navigate to your MyDrive folder.

5. Mount Google Drive in Colab Run this cell first:

   from google.colab import drive
   drive.mount('/content/drive')
   

6. %cd /content/drive/MyDrive
   !ls -l
   

7. Create model

   python gpt_jit_modern_8b.py
   

8. See in terminal

• Creating 0.94B-parameter Llama-style model on cpu...
• Model weights checked: no NaN/Inf found.
• Model ready. Parameters: xxB
• Logits min/max: -4.722229480743408 4.230556488037109
• Any NaN in logits: False
• Test forward pass OK (without JIT) → Logits: torch.Size([1, 256, 50257])
• Tracing model for JIT export (length: 256)...
• Success! JIT model saved → /content/drive/MyDrive/models/gpt_modern_8b_class.script.pt (xx GB)
• Original state_dict saved for debugging → gpt_modern_1b_class.state_dict.pt

Files Generated

• After successful run, you will have:

• gpt_modern_1b_class.script.pt → JIT-traced model ready for fast inference (~xx GB)

• gpt_modern_1b_class.state_dict.pt → Original PyTorch state dict (useful for conversion to Hugging Face format)

Tips

• Use Colab Pro or Pro+ for faster training and higher RAM (124B+ models may require it).

• The script runs on CPU by default but can be modified to use GPU/TPU.

• To convert the .pt file to Hugging Face Transformers format, additional conversion scripts are usually needed.

• Enjoy building your own LLM from scratch for free!

Deep learning

• Edit file fine_tune1b_with_validation_no_torchscript.py
• Setup dataset path and number of EPOCHS
• Run the script

   python fine_tune1b_with_validation_no_torchscript.py
  

Test your model (LLM)

• Edit chatbot_state_dict_8b.py
• Setup TEMPERATURE = 0.7 and TOP_K = 50 or other values
• Setup correct path to trained model LAST_TRAINED_PATH = Path("models/gpt_last_modern_8b_class.state_dict.pt")
• and run chatbot to test you model

  python chatbot_state_dict_8b.py
  

After valudation model

• So you can add mode EPOCHS and contunue your trainings

• then run chatbot to check model again

• onece it done call me I will comvert it to ONNX format and make Web service with REST API on ONNX Runtime libary

• So About PyTorch script . You can use Pytorch script for AI classification task .

• Do not Jit for Chatbot task . Use just state dict PyTorch for GPT (Chatbot) tasks

Welcom to CMS Manhattan

• Copyright (c) 2025 CMS Manhattan

• All rights reserved.

• Author: Konstantin Vladimirovich Grabko

• Email: [email protected]

• Phone: +1(516)777-0945

• This file is part of a project authored by CMS Manhattan. You may use, distribute, and modify

• this code under the terms of the GNU GENERAL PUBLIC LICENSE, Version 3, 29 June 2007.

• Please read http://www.gnu.org/licenses/ .