To run these models, you need the
LutMac
engine installed. You can find the source code and build instructions at:
https://github.com/YASSERRMD/lutmac
1. Build the Engine
git clone https://github.com/YASSERRMD/lutmac.git
cd lutmac
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j4
2. Run the Model
Download your preferred
.lutmac
file and the
tokenizer.json
from this repository.
./lutmac-inference \
--model ./qwen2.5-1.5b-instruct-4bit.lutmac \
--tokenizer ./tokenizer.json \
--prompt "What is the capital of France?" \
--max-tokens 100 \
--streaming
Quantization Details
These models were quantized using the bit-serial LUT engine methodology. Sub-4-bit models utilize
Hadamard Rotation (FWHT)
on both weights and activations to mitigate the impact of outliers, ensuring stability even at extreme compression rates.
Experimental Project
: This is part of ongoing research into ultra-low-bit CPU inference. Contributors and feedback are welcome at the
main repository
.
Runs of yasserrmd Qwen2.5-1.5B-Instruct-lutmac on huggingface.co
5
Total runs
0
24-hour runs
0
3-day runs
1
7-day runs
2
30-day runs
More Information About Qwen2.5-1.5B-Instruct-lutmac huggingface.co Model
More Qwen2.5-1.5B-Instruct-lutmac license Visit here:
Qwen2.5-1.5B-Instruct-lutmac huggingface.co is an AI model on huggingface.co that provides Qwen2.5-1.5B-Instruct-lutmac's model effect (), which can be used instantly with this yasserrmd Qwen2.5-1.5B-Instruct-lutmac model. huggingface.co supports a free trial of the Qwen2.5-1.5B-Instruct-lutmac model, and also provides paid use of the Qwen2.5-1.5B-Instruct-lutmac. Support call Qwen2.5-1.5B-Instruct-lutmac model through api, including Node.js, Python, http.
Qwen2.5-1.5B-Instruct-lutmac huggingface.co is an online trial and call api platform, which integrates Qwen2.5-1.5B-Instruct-lutmac's modeling effects, including api services, and provides a free online trial of Qwen2.5-1.5B-Instruct-lutmac, you can try Qwen2.5-1.5B-Instruct-lutmac online for free by clicking the link below.
yasserrmd Qwen2.5-1.5B-Instruct-lutmac online free url in huggingface.co:
Qwen2.5-1.5B-Instruct-lutmac is an open source model from GitHub that offers a free installation service, and any user can find Qwen2.5-1.5B-Instruct-lutmac on GitHub to install. At the same time, huggingface.co provides the effect of Qwen2.5-1.5B-Instruct-lutmac install, users can directly use Qwen2.5-1.5B-Instruct-lutmac installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
Qwen2.5-1.5B-Instruct-lutmac install url in huggingface.co: