yasserrmd / Qwen2.5-1.5B-Instruct-lutmac

huggingface.co
Total runs: 5
24-hour runs: 0
7-day runs: 1
30-day runs: 2
Model's Last Updated: December 29 2025

Introduction of Qwen2.5-1.5B-Instruct-lutmac

Model Details of Qwen2.5-1.5B-Instruct-lutmac

Qwen2.5-1.5B-Instruct-lutmac

This repository contains ultra-low-bit quantized versions of Qwen2.5-1.5B-Instruct , optimized for the LutMac inference engine.

Available Variants
Precision Format Size Description
8-bit .lutmac 1.49 GB Standard Int8 quantization.
6-bit .lutmac 1.18 GB Int6 quantization for high efficiency.
5-bit .lutmac 1.03 GB Int5 quantization balanced weight.
4-bit .lutmac 870 MB Optimized 4-bit quantization with tied embeddings (8-bit).
3-bit .lutmac 714 MB Int3 quantization for memory-constrained devices.
2-bit .lutmac 578 MB 2-bit quantization using Hadamard Rotation and RRQ.
1.58-bit .lutmac 578 MB Ternary quantization {-1, 0, +1} (Sign-Magnitude encoding).
1-bit .lutmac 402 MB Binary quantization {-1, +1} (Purely bit-serial).
How to Run Inference

To run these models, you need the LutMac engine installed. You can find the source code and build instructions at: https://github.com/YASSERRMD/lutmac

1. Build the Engine
git clone https://github.com/YASSERRMD/lutmac.git
cd lutmac
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j4
2. Run the Model

Download your preferred .lutmac file and the tokenizer.json from this repository.

./lutmac-inference \
    --model ./qwen2.5-1.5b-instruct-4bit.lutmac \
    --tokenizer ./tokenizer.json \
    --prompt "What is the capital of France?" \
    --max-tokens 100 \
    --streaming
Quantization Details

These models were quantized using the bit-serial LUT engine methodology. Sub-4-bit models utilize Hadamard Rotation (FWHT) on both weights and activations to mitigate the impact of outliers, ensuring stability even at extreme compression rates.

  • 4-bit and above : Symmetric integer quantization.
  • Sub-4-bit : Recursive Residual Quantization (RRQ) combined with Incoherence Processing (QuIP-based rotation).

Experimental Project : This is part of ongoing research into ultra-low-bit CPU inference. Contributors and feedback are welcome at the main repository .

Runs of yasserrmd Qwen2.5-1.5B-Instruct-lutmac on huggingface.co

5
Total runs
0
24-hour runs
0
3-day runs
1
7-day runs
2
30-day runs

More Information About Qwen2.5-1.5B-Instruct-lutmac huggingface.co Model

More Qwen2.5-1.5B-Instruct-lutmac license Visit here:

https://choosealicense.com/licenses/apache-2.0

Qwen2.5-1.5B-Instruct-lutmac huggingface.co

Qwen2.5-1.5B-Instruct-lutmac huggingface.co is an AI model on huggingface.co that provides Qwen2.5-1.5B-Instruct-lutmac's model effect (), which can be used instantly with this yasserrmd Qwen2.5-1.5B-Instruct-lutmac model. huggingface.co supports a free trial of the Qwen2.5-1.5B-Instruct-lutmac model, and also provides paid use of the Qwen2.5-1.5B-Instruct-lutmac. Support call Qwen2.5-1.5B-Instruct-lutmac model through api, including Node.js, Python, http.

Qwen2.5-1.5B-Instruct-lutmac huggingface.co Url

https://huggingface.co/yasserrmd/Qwen2.5-1.5B-Instruct-lutmac

yasserrmd Qwen2.5-1.5B-Instruct-lutmac online free

Qwen2.5-1.5B-Instruct-lutmac huggingface.co is an online trial and call api platform, which integrates Qwen2.5-1.5B-Instruct-lutmac's modeling effects, including api services, and provides a free online trial of Qwen2.5-1.5B-Instruct-lutmac, you can try Qwen2.5-1.5B-Instruct-lutmac online for free by clicking the link below.

yasserrmd Qwen2.5-1.5B-Instruct-lutmac online free url in huggingface.co:

https://huggingface.co/yasserrmd/Qwen2.5-1.5B-Instruct-lutmac

Qwen2.5-1.5B-Instruct-lutmac install

Qwen2.5-1.5B-Instruct-lutmac is an open source model from GitHub that offers a free installation service, and any user can find Qwen2.5-1.5B-Instruct-lutmac on GitHub to install. At the same time, huggingface.co provides the effect of Qwen2.5-1.5B-Instruct-lutmac install, users can directly use Qwen2.5-1.5B-Instruct-lutmac installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Qwen2.5-1.5B-Instruct-lutmac install url in huggingface.co:

https://huggingface.co/yasserrmd/Qwen2.5-1.5B-Instruct-lutmac

Url of Qwen2.5-1.5B-Instruct-lutmac

Qwen2.5-1.5B-Instruct-lutmac huggingface.co Url

Provider of Qwen2.5-1.5B-Instruct-lutmac huggingface.co

yasserrmd
ORGANIZATIONS

Other API from yasserrmd

huggingface.co

Total runs: 28
Run Growth: 3
Growth Rate: 10.71%
Updated:October 19 2025