nisten / quad-mixtrals-gguf

huggingface.co
Total runs: 46
24-hour runs: 0
7-day runs: -11
30-day runs: -74
Model's Last Updated: December 28 2023

Introduction of quad-mixtrals-gguf

Model Details of quad-mixtrals-gguf

Experimental quants of 4 expert MoE mixtrals in various GGUF formats.

Original model used for custom quants: NeverSleep/Mistral-11B-SynthIAirOmniMix
https://huggingface.co/NeverSleep/Mistral-11B-SynthIAirOmniMix

Goal is to have the best performing MoE < 10gb

Experimental q8 and q4 files for training/finetuning too.

No sparsity tricks yet.

8.4gb custom 2bit quant works ok up until 512 token length then starts looping.

  • Install llama.cpp from github and run it:
git clone https://github.com/ggerganov/llama.cpp

cd llama.cpp

make -j 

wget https://huggingface.co/nisten/quad-mixtrals-gguf/resolve/main/4mixq2.gguf

./server -m 4mixq2.gguf --host "my.internal.ip.or.my.cloud.host.name.goes.here.com" -c 512

limit output to 500 tokens

Runs of nisten quad-mixtrals-gguf on huggingface.co

46
Total runs
0
24-hour runs
0
3-day runs
-11
7-day runs
-74
30-day runs

More Information About quad-mixtrals-gguf huggingface.co Model

More quad-mixtrals-gguf license Visit here:

https://choosealicense.com/licenses/apache-2.0

quad-mixtrals-gguf huggingface.co

quad-mixtrals-gguf huggingface.co is an AI model on huggingface.co that provides quad-mixtrals-gguf's model effect (), which can be used instantly with this nisten quad-mixtrals-gguf model. huggingface.co supports a free trial of the quad-mixtrals-gguf model, and also provides paid use of the quad-mixtrals-gguf. Support call quad-mixtrals-gguf model through api, including Node.js, Python, http.

quad-mixtrals-gguf huggingface.co Url

https://huggingface.co/nisten/quad-mixtrals-gguf

nisten quad-mixtrals-gguf online free

quad-mixtrals-gguf huggingface.co is an online trial and call api platform, which integrates quad-mixtrals-gguf's modeling effects, including api services, and provides a free online trial of quad-mixtrals-gguf, you can try quad-mixtrals-gguf online for free by clicking the link below.

nisten quad-mixtrals-gguf online free url in huggingface.co:

https://huggingface.co/nisten/quad-mixtrals-gguf

quad-mixtrals-gguf install

quad-mixtrals-gguf is an open source model from GitHub that offers a free installation service, and any user can find quad-mixtrals-gguf on GitHub to install. At the same time, huggingface.co provides the effect of quad-mixtrals-gguf install, users can directly use quad-mixtrals-gguf installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

quad-mixtrals-gguf install url in huggingface.co:

https://huggingface.co/nisten/quad-mixtrals-gguf

Url of quad-mixtrals-gguf

quad-mixtrals-gguf huggingface.co Url

Provider of quad-mixtrals-gguf huggingface.co

nisten
ORGANIZATIONS

Other API from nisten

huggingface.co

Total runs: 117
Run Growth: -2
Growth Rate: -1.71%
Updated:March 06 2024
huggingface.co

Total runs: 80
Run Growth: 59
Growth Rate: 73.75%
Updated:December 18 2024
huggingface.co

Total runs: 45
Run Growth: 45
Growth Rate: 100.00%
Updated:September 16 2025
huggingface.co

Total runs: 21
Run Growth: -1
Growth Rate: -4.76%
Updated:December 23 2024
huggingface.co

Total runs: 6
Run Growth: 0
Growth Rate: 0.00%
Updated:February 07 2025
huggingface.co

Total runs: 5
Run Growth: 0
Growth Rate: 0.00%
Updated:January 07 2025
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated:August 18 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated:August 20 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated:July 19 2023