Ram07 / hbitnet-1b

HBITNET-1B - H-BitLinear BitNet Model

1B parameters H-BitLinear BitNet model with 2-bit quantized weights. CUDA kernels with FWHT optimization.

Model Details

• Architecture : H-BitLinear BitNet
• Parameters : 1B parameters
• Precision : 2-bit quantized + FP16
• Framework : PyTorch
• Training Data : FineWeb-Edu dataset
• Hardware Optimization : CUDA kernels with FWHT (Fast Walsh-Hadamard Transform) optimization
• Performance : Expected 2-4x speedup compared to standard BitNet implementation

Usage

Direct Usage with Hugging Face Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
model_name = "Ram07/hbitnet-1b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Generate text
prompt = "The future of artificial intelligence is"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,
    do_sample=True
)

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text)

Using BitNet's Custom Inference Engine

from bitnet.inference import BitNetInferenceEngine

# Initialize inference engine
engine = BitNetInferenceEngine(
    model_path="Ram07/hbitnet-1b",
    device="cuda"  # or "cpu"
)

# Generate with custom parameters
output = engine.generate(
    prompt="In a world where technology advances rapidly,",
    max_length=50,
    temperature=0.8,
    top_p=0.9
)
print(output)

Model Configuration

{"vocab_size": 128256,
  "hidden_size": 1536,
  "num_hidden_layers": 20,
  "num_attention_heads": 16,
  "max_position_embeddings": 512,
  "quantization": {
    "weight_bits": 2,
    "activation_bits": 8
  },
  "hw_bitlinear": true,
  "fwht_optimization": true
}

Performance Benchmarks

Inference Speed

• Tokens/sec : ~0.12-0.14 (CPU), Expected higher with CUDA
• ** Time to First Token (TTFT)**: ~6-8 seconds
• Memory Usage : Significantly reduced due to 2-bit quantization
• CUDA Optimization : FWHT kernels provide hardware acceleration

Model Comparison

Use Cases

• Text generation
• Conversational AI
• Few-shot learning
• Research applications
• Edge deployment
• High-performance inference

Limitations

• Language : Trained primarily on English text
• Context Length : Limited to 512 tokens in current configuration
• Quantization : 2-bit quantization may impact certain downstream tasks
• Hardware : Optimized for NVIDIA GPUs with CUDA support

Citation

@misc{bitnetHBITNET-1B,
  title={H-BitLinear BitNet 1B parameters},
  author={Ram07},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/Ram07/hbitnet-1b}
}

Repository Structure

Ram07/hbitnet-1b/
├── model.safetensors    # Main model weights (SafeTensors format)
├── config.json         # Model configuration
├── README.md           # This file
├── modelcard.yaml      # Model card metadata
└── repository.yaml     # Repository structure metadata

Licensing

This model is licensed under the Apache License 2.0. See the LICENSE file for details.

Contact

For questions about this model, please visit the BitNet GitHub repository or open an issue.

Generated on 2025-10-04