Sponsored by YouArt
Toolify Toolify logo
AI Models
opt-125m-INT4-v2-0.13-dev

torchao-testing / opt-125m-INT4-v2-0.13-dev

Run on huggingface.co
huggingface.co
Total runs: 33
24-hour runs: 0
7-day runs: 0
30-day runs: 2
Model's Last Updated: August 19 2025
text-generation

Introduction of opt-125m-INT4-v2-0.13-dev

Model Details of opt-125m-INT4-v2-0.13-dev

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, TorchAoConfig

model_id = "facebook/opt-125m"

from torchao.quantization import Int4WeightOnlyConfig
quant_config = Int4WeightOnlyConfig(group_size=128, packing_format="plain", version=2)

quantization_config = TorchAoConfig(quant_type=quant_config)
quantized_model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="cuda",
    torch_dtype=torch.bfloat16,
    quantization_config=quantization_config,
)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Push to hub
USER_ID = "torchao-testing"
MODEL_NAME = model_id.split("/")[-1]
save_to = f"{USER_ID}/{MODEL_NAME}-INT4-v2-0.13-dev"
quantized_model.push_to_hub(save_to, safe_serialization=False)
tokenizer.push_to_hub(save_to)

# Manual Testing
prompt = "Hey, are you conscious? Can you talk to me?"
print("Prompt:", prompt)
inputs = tokenizer(
    prompt,
    return_tensors="pt",
).to("cuda")
generated_ids = quantized_model.generate(**inputs, max_new_tokens=128)
output_text = tokenizer.batch_decode(
    generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print("Response:", output_text[0][len(prompt) :])

Runs of torchao-testing opt-125m-INT4-v2-0.13-dev on huggingface.co

33
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
2
30-day runs

More Information About opt-125m-INT4-v2-0.13-dev huggingface.co Model

opt-125m-INT4-v2-0.13-dev huggingface.co

opt-125m-INT4-v2-0.13-dev huggingface.co is an AI model on huggingface.co that provides opt-125m-INT4-v2-0.13-dev's model effect (), which can be used instantly with this torchao-testing opt-125m-INT4-v2-0.13-dev model. huggingface.co supports a free trial of the opt-125m-INT4-v2-0.13-dev model, and also provides paid use of the opt-125m-INT4-v2-0.13-dev. Support call opt-125m-INT4-v2-0.13-dev model through api, including Node.js, Python, http.

opt-125m-INT4-v2-0.13-dev huggingface.co Url

https://huggingface.co/torchao-testing/opt-125m-INT4-v2-0.13-dev

torchao-testing opt-125m-INT4-v2-0.13-dev online free

opt-125m-INT4-v2-0.13-dev huggingface.co is an online trial and call api platform, which integrates opt-125m-INT4-v2-0.13-dev's modeling effects, including api services, and provides a free online trial of opt-125m-INT4-v2-0.13-dev, you can try opt-125m-INT4-v2-0.13-dev online for free by clicking the link below.

torchao-testing opt-125m-INT4-v2-0.13-dev online free url in huggingface.co:

https://huggingface.co/torchao-testing/opt-125m-INT4-v2-0.13-dev

opt-125m-INT4-v2-0.13-dev install

opt-125m-INT4-v2-0.13-dev is an open source model from GitHub that offers a free installation service, and any user can find opt-125m-INT4-v2-0.13-dev on GitHub to install. At the same time, huggingface.co provides the effect of opt-125m-INT4-v2-0.13-dev install, users can directly use opt-125m-INT4-v2-0.13-dev installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

opt-125m-INT4-v2-0.13-dev install url in huggingface.co:

https://huggingface.co/torchao-testing/opt-125m-INT4-v2-0.13-dev

Url of opt-125m-INT4-v2-0.13-dev

opt-125m-INT4-v2-0.13-dev huggingface.co Url

Provider of opt-125m-INT4-v2-0.13-dev huggingface.co

torchao-testing
ORGANIZATIONS
huggingface.co

Other API from torchao-testing