legraphista / codegeex4-all-9b-IMat-GGUF

huggingface.co
Total runs: 630
24-hour runs: 0
7-day runs: 84
30-day runs: -4.4K
Model's Last Updated: July 06 2024
text-generation

Introduction of codegeex4-all-9b-IMat-GGUF

Model Details of codegeex4-all-9b-IMat-GGUF

codegeex4-all-9b-IMat-GGUF

Llama.cpp imatrix quantization of THUDM/codegeex4-all-9b

Original Model: THUDM/codegeex4-all-9b
Original dtype: BF16 ( bfloat16 )
Quantized by: https://github.com/ggerganov/llama.cpp/pull/8031
IMatrix dataset: here


Files
IMatrix

Status: โœ… Available
Link: here

Common Quants
Filename Quant type File Size Status Uses IMatrix Is Split
codegeex4-all-9b.Q8_0.gguf Q8_0 9.99GB โœ… Available โšช Static ๐Ÿ“ฆ No
codegeex4-all-9b.Q6_K.gguf Q6_K 8.26GB โœ… Available โšช Static ๐Ÿ“ฆ No
codegeex4-all-9b.Q4_K.gguf Q4_K 6.25GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.Q3_K.gguf Q3_K 5.06GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.Q2_K.gguf Q2_K 3.99GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
All Quants
Filename Quant type File Size Status Uses IMatrix Is Split
codegeex4-all-9b.BF16.gguf BF16 18.81GB โœ… Available โšช Static ๐Ÿ“ฆ No
codegeex4-all-9b.FP16.gguf F16 18.81GB โœ… Available โšช Static ๐Ÿ“ฆ No
codegeex4-all-9b.Q8_0.gguf Q8_0 9.99GB โœ… Available โšช Static ๐Ÿ“ฆ No
codegeex4-all-9b.Q6_K.gguf Q6_K 8.26GB โœ… Available โšช Static ๐Ÿ“ฆ No
codegeex4-all-9b.Q5_K.gguf Q5_K 7.14GB โœ… Available โšช Static ๐Ÿ“ฆ No
codegeex4-all-9b.Q5_K_S.gguf Q5_K_S 6.69GB โœ… Available โšช Static ๐Ÿ“ฆ No
codegeex4-all-9b.Q4_K.gguf Q4_K 6.25GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.Q4_K_S.gguf Q4_K_S 5.75GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ4_NL.gguf IQ4_NL 5.46GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ4_XS.gguf IQ4_XS 5.25GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.Q3_K.gguf Q3_K 5.06GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.Q3_K_L.gguf Q3_K_L 5.28GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.Q3_K_S.gguf Q3_K_S 4.59GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ3_M.gguf IQ3_M 4.81GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ3_S.gguf IQ3_S 4.59GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ3_XS.gguf IQ3_XS 4.43GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ3_XXS.gguf IQ3_XXS 4.26GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.Q2_K.gguf Q2_K 3.99GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.Q2_K_S.gguf Q2_K_S 3.96GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ2_M.gguf IQ2_M 3.93GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ2_S.gguf IQ2_S 3.77GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ2_XS.gguf IQ2_XS 3.61GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ2_XXS.gguf IQ2_XXS 3.43GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ1_M.gguf IQ1_M 3.22GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
codegeex4-all-9b.IQ1_S.gguf IQ1_S 3.10GB โœ… Available ๐ŸŸข IMatrix ๐Ÿ“ฆ No
Downloading using huggingface-cli

If you do not have hugginface-cli installed:

pip install -U "huggingface_hub[cli]"

Download the specific file you want:

huggingface-cli download legraphista/codegeex4-all-9b-IMat-GGUF --include "codegeex4-all-9b.Q8_0.gguf" --local-dir ./

If the model file is big, it has been split into multiple files. In order to download them all to a local folder, run:

huggingface-cli download legraphista/codegeex4-all-9b-IMat-GGUF --include "codegeex4-all-9b.Q8_0/*" --local-dir ./
# see FAQ for merging GGUF's

Inference
Llama.cpp
llama.cpp/main -m codegeex4-all-9b.Q8_0.gguf --color -i -p "prompt here"

FAQ
Why is the IMatrix not applied everywhere?

According to this investigation , it appears that lower quantizations are the only ones that benefit from the imatrix input (as per hellaswag results).

How do I merge a split GGUF?
  1. Make sure you have gguf-split available
  2. Locate your GGUF chunks folder (ex: codegeex4-all-9b.Q8_0 )
  3. Run gguf-split --merge codegeex4-all-9b.Q8_0/codegeex4-all-9b.Q8_0-00001-of-XXXXX.gguf codegeex4-all-9b.Q8_0.gguf
    • Make sure to point gguf-split to the first chunk of the split.

Got a suggestion? Ping me @legraphista !

Runs of legraphista codegeex4-all-9b-IMat-GGUF on huggingface.co

630
Total runs
0
24-hour runs
0
3-day runs
84
7-day runs
-4.4K
30-day runs

More Information About codegeex4-all-9b-IMat-GGUF huggingface.co Model

More codegeex4-all-9b-IMat-GGUF license Visit here:

https://choosealicense.com/licenses/codegeex4

codegeex4-all-9b-IMat-GGUF huggingface.co

codegeex4-all-9b-IMat-GGUF huggingface.co is an AI model on huggingface.co that provides codegeex4-all-9b-IMat-GGUF's model effect (), which can be used instantly with this legraphista codegeex4-all-9b-IMat-GGUF model. huggingface.co supports a free trial of the codegeex4-all-9b-IMat-GGUF model, and also provides paid use of the codegeex4-all-9b-IMat-GGUF. Support call codegeex4-all-9b-IMat-GGUF model through api, including Node.js, Python, http.

codegeex4-all-9b-IMat-GGUF huggingface.co Url

https://huggingface.co/legraphista/codegeex4-all-9b-IMat-GGUF

legraphista codegeex4-all-9b-IMat-GGUF online free

codegeex4-all-9b-IMat-GGUF huggingface.co is an online trial and call api platform, which integrates codegeex4-all-9b-IMat-GGUF's modeling effects, including api services, and provides a free online trial of codegeex4-all-9b-IMat-GGUF, you can try codegeex4-all-9b-IMat-GGUF online for free by clicking the link below.

legraphista codegeex4-all-9b-IMat-GGUF online free url in huggingface.co:

https://huggingface.co/legraphista/codegeex4-all-9b-IMat-GGUF

codegeex4-all-9b-IMat-GGUF install

codegeex4-all-9b-IMat-GGUF is an open source model from GitHub that offers a free installation service, and any user can find codegeex4-all-9b-IMat-GGUF on GitHub to install. At the same time, huggingface.co provides the effect of codegeex4-all-9b-IMat-GGUF install, users can directly use codegeex4-all-9b-IMat-GGUF installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

codegeex4-all-9b-IMat-GGUF install url in huggingface.co:

https://huggingface.co/legraphista/codegeex4-all-9b-IMat-GGUF

Url of codegeex4-all-9b-IMat-GGUF

codegeex4-all-9b-IMat-GGUF huggingface.co Url

Provider of codegeex4-all-9b-IMat-GGUF huggingface.co

legraphista
ORGANIZATIONS

Other API from legraphista