NeuML / Llama-3.1_OpenScholar-8B-AWQ

huggingface.co
Total runs: 1.3K
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: November 28 2024
text-generation

Introduction of Llama-3.1_OpenScholar-8B-AWQ

Model Details of Llama-3.1_OpenScholar-8B-AWQ

Llama-3.1_OpenScholar-8B with AWQ Quantization

This is Llama-3.1_OpenScholar-8B with AWQ Quantization applied using the following code.

Based on this example code .

import torch

from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer

# Input and output path
path = "OpenScholar/Llama-3.1_OpenScholar-8B"
output = "Llama-3.1_OpenScholar-8B-AWQ"

# Quantization config
config = {
    "zero_point": True,
    "q_group_size": 128,
    "w_bit": 4,
    "version": "GEMM"
}

# Load model
model = AutoAWQForCausalLM.from_pretrained(
    model_path=path,
    low_cpu_mem_usage=True,
    use_cache=False,
    safetensors=False,
    device_map="cuda",
    torch_dtype=torch.bfloat16
)
tokenizer = AutoTokenizer.from_pretrained(path)

# Quantize
model.quantize(tokenizer, quant_config=config)

# Save quantized model
model.save_quantized(output)

# Save tokenizer
# Note: Transformers >= 4.45.0 doubles size of tokenizer.json
# See https://github.com/huggingface/transformers/issues/34744
tokenizer.save_pretrained(output)

print(f'Model is quantized and saved to "{output}"')

Runs of NeuML Llama-3.1_OpenScholar-8B-AWQ on huggingface.co

1.3K
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About Llama-3.1_OpenScholar-8B-AWQ huggingface.co Model

More Llama-3.1_OpenScholar-8B-AWQ license Visit here:

https://choosealicense.com/licenses/apache-2.0

Llama-3.1_OpenScholar-8B-AWQ huggingface.co

Llama-3.1_OpenScholar-8B-AWQ huggingface.co is an AI model on huggingface.co that provides Llama-3.1_OpenScholar-8B-AWQ's model effect (), which can be used instantly with this NeuML Llama-3.1_OpenScholar-8B-AWQ model. huggingface.co supports a free trial of the Llama-3.1_OpenScholar-8B-AWQ model, and also provides paid use of the Llama-3.1_OpenScholar-8B-AWQ. Support call Llama-3.1_OpenScholar-8B-AWQ model through api, including Node.js, Python, http.

Llama-3.1_OpenScholar-8B-AWQ huggingface.co Url

https://huggingface.co/NeuML/Llama-3.1_OpenScholar-8B-AWQ

NeuML Llama-3.1_OpenScholar-8B-AWQ online free

Llama-3.1_OpenScholar-8B-AWQ huggingface.co is an online trial and call api platform, which integrates Llama-3.1_OpenScholar-8B-AWQ's modeling effects, including api services, and provides a free online trial of Llama-3.1_OpenScholar-8B-AWQ, you can try Llama-3.1_OpenScholar-8B-AWQ online for free by clicking the link below.

NeuML Llama-3.1_OpenScholar-8B-AWQ online free url in huggingface.co:

https://huggingface.co/NeuML/Llama-3.1_OpenScholar-8B-AWQ

Llama-3.1_OpenScholar-8B-AWQ install

Llama-3.1_OpenScholar-8B-AWQ is an open source model from GitHub that offers a free installation service, and any user can find Llama-3.1_OpenScholar-8B-AWQ on GitHub to install. At the same time, huggingface.co provides the effect of Llama-3.1_OpenScholar-8B-AWQ install, users can directly use Llama-3.1_OpenScholar-8B-AWQ installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Llama-3.1_OpenScholar-8B-AWQ install url in huggingface.co:

https://huggingface.co/NeuML/Llama-3.1_OpenScholar-8B-AWQ

Url of Llama-3.1_OpenScholar-8B-AWQ

Llama-3.1_OpenScholar-8B-AWQ huggingface.co Url

Provider of Llama-3.1_OpenScholar-8B-AWQ huggingface.co

NeuML
ORGANIZATIONS

Other API from NeuML

huggingface.co

Total runs: 2.0K
Run Growth: 1.6K
Growth Rate: 83.32%
Updated:January 27 2025
huggingface.co

Total runs: 1.2K
Run Growth: 254
Growth Rate: 22.72%
Updated:February 21 2023
huggingface.co

Total runs: 1.1K
Run Growth: -170
Growth Rate: -16.18%
Updated:January 27 2025
huggingface.co

Total runs: 853
Run Growth: 250
Growth Rate: 29.31%
Updated:February 21 2023
huggingface.co

Total runs: 521
Run Growth: -99.9K
Growth Rate: -19182.34%
Updated:January 27 2025
huggingface.co

Total runs: 285
Run Growth: 98
Growth Rate: 34.39%
Updated:January 27 2025
huggingface.co

Total runs: 207
Run Growth: 127
Growth Rate: 61.95%
Updated:April 21 2026
huggingface.co

Total runs: 31
Run Growth: 14
Growth Rate: 45.16%
Updated:October 10 2025
huggingface.co

Total runs: 12
Run Growth: -7
Growth Rate: -53.85%
Updated:November 23 2024
huggingface.co

Total runs: 9
Run Growth: 1
Growth Rate: 12.50%
Updated:July 21 2026
huggingface.co

Total runs: 8
Run Growth: -75
Growth Rate: -937.50%
Updated:April 13 2026
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated:November 11 2025