Sponsored by Vmake Labs Video Enhancer
Toolify Toolify logo
AI Models
DeepSeek-Coder-V2-Lite-Base-IMat-GGUF

legraphista / DeepSeek-Coder-V2-Lite-Base-IMat-GGUF

Run on huggingface.co
huggingface.co
Total runs: 3.3K
24-hour runs: 0
7-day runs: 86
30-day runs: 1.9K
Model's Last Updated: June 18 2024
text-generation

Introduction of DeepSeek-Coder-V2-Lite-Base-IMat-GGUF

Model Details of DeepSeek-Coder-V2-Lite-Base-IMat-GGUF

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF

Llama.cpp imatrix quantization of deepseek-ai/DeepSeek-Coder-V2-Lite-Base

Original Model: deepseek-ai/DeepSeek-Coder-V2-Lite-Base
Original dtype: BF16 ( bfloat16 )
Quantized by: llama.cpp b3166
IMatrix dataset: here

Files
IMatrix

Status: ✅ Available
Link: here

Common Quants
Filename Quant type File Size Status Uses IMatrix Is Split
DeepSeek-Coder-V2-Lite-Base.Q8_0.gguf Q8_0 16.70GB ✅ Available ⚪ Static 📦 No
DeepSeek-Coder-V2-Lite-Base.Q6_K.gguf Q6_K 14.07GB ✅ Available ⚪ Static 📦 No
DeepSeek-Coder-V2-Lite-Base.Q4_K.gguf Q4_K 10.36GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.Q3_K.gguf Q3_K 8.13GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.Q2_K.gguf Q2_K 6.43GB ✅ Available 🟢 IMatrix 📦 No
All Quants
Filename Quant type File Size Status Uses IMatrix Is Split
DeepSeek-Coder-V2-Lite-Base.BF16.gguf BF16 31.42GB ✅ Available ⚪ Static 📦 No
DeepSeek-Coder-V2-Lite-Base.FP16.gguf F16 31.42GB ✅ Available ⚪ Static 📦 No
DeepSeek-Coder-V2-Lite-Base.Q8_0.gguf Q8_0 16.70GB ✅ Available ⚪ Static 📦 No
DeepSeek-Coder-V2-Lite-Base.Q6_K.gguf Q6_K 14.07GB ✅ Available ⚪ Static 📦 No
DeepSeek-Coder-V2-Lite-Base.Q5_K.gguf Q5_K 11.85GB ✅ Available ⚪ Static 📦 No
DeepSeek-Coder-V2-Lite-Base.Q5_K_S.gguf Q5_K_S 11.14GB ✅ Available ⚪ Static 📦 No
DeepSeek-Coder-V2-Lite-Base.Q4_K.gguf Q4_K 10.36GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.Q4_K_S.gguf Q4_K_S 9.53GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ4_NL.gguf IQ4_NL 8.91GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ4_XS.gguf IQ4_XS 8.57GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.Q3_K.gguf Q3_K 8.13GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.Q3_K_L.gguf Q3_K_L 8.46GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.Q3_K_S.gguf Q3_K_S 7.49GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ3_M.gguf IQ3_M 7.55GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ3_S.gguf IQ3_S 7.49GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ3_XS.gguf IQ3_XS 7.12GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ3_XXS.gguf IQ3_XXS 6.96GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.Q2_K.gguf Q2_K 6.43GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.Q2_K_S.gguf Q2_K_S 6.46GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ2_M.gguf IQ2_M 6.33GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ2_S.gguf IQ2_S 6.01GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ2_XS.gguf IQ2_XS 5.97GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ2_XXS.gguf IQ2_XXS 5.64GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ1_M.gguf IQ1_M 5.24GB ✅ Available 🟢 IMatrix 📦 No
DeepSeek-Coder-V2-Lite-Base.IQ1_S.gguf IQ1_S 4.99GB ✅ Available 🟢 IMatrix 📦 No
Downloading using huggingface-cli

If you do not have hugginface-cli installed: 

pip install -U "huggingface_hub[cli]"

Download the specific file you want: 

huggingface-cli download legraphista/DeepSeek-Coder-V2-Lite-Base-IMat-GGUF --include "DeepSeek-Coder-V2-Lite-Base.Q8_0.gguf" --local-dir ./

If the model file is big, it has been split into multiple files. In order to download them all to a local folder, run: 

huggingface-cli download legraphista/DeepSeek-Coder-V2-Lite-Base-IMat-GGUF --include "DeepSeek-Coder-V2-Lite-Base.Q8_0/*" --local-dir ./
# see FAQ for merging GGUF's
Inference
Simple chat template 
<｜begin▁of▁sentence｜>User: {user_prompt}

Assistant: {assistant_response}<｜end▁of▁sentence｜>User: {next_user_prompt}
Chat template with system prompt 
<｜begin▁of▁sentence｜>{system_prompt}

User: {user_prompt}

Assistant: {assistant_response}<｜end▁of▁sentence｜>User: {next_user_prompt}
Llama.cpp 
llama.cpp/main -m DeepSeek-Coder-V2-Lite-Base.Q8_0.gguf --color -i -p "prompt here (according to the chat template)"
FAQ
Why is the IMatrix not applied everywhere?

According to this investigation , it appears that lower quantizations are the only ones that benefit from the imatrix input (as per hellaswag results).

How do I merge a split GGUF?
  1. Make sure you have gguf-split available
  2. Locate your GGUF chunks folder (ex: DeepSeek-Coder-V2-Lite-Base.Q8_0 )
  3. Run gguf-split --merge DeepSeek-Coder-V2-Lite-Base.Q8_0/DeepSeek-Coder-V2-Lite-Base.Q8_0-00001-of-XXXXX.gguf DeepSeek-Coder-V2-Lite-Base.Q8_0.gguf
    • Make sure to point gguf-split to the first chunk of the split.

Got a suggestion? Ping me @legraphista !

Runs of legraphista DeepSeek-Coder-V2-Lite-Base-IMat-GGUF on huggingface.co

3.3K
Total runs
0
24-hour runs
-136
3-day runs
86
7-day runs
1.9K
30-day runs

More Information About DeepSeek-Coder-V2-Lite-Base-IMat-GGUF huggingface.co Model

More DeepSeek-Coder-V2-Lite-Base-IMat-GGUF license Visit here:

https://choosealicense.com/licenses/deepseek-license

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF huggingface.co

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF huggingface.co is an AI model on huggingface.co that provides DeepSeek-Coder-V2-Lite-Base-IMat-GGUF's model effect (), which can be used instantly with this legraphista DeepSeek-Coder-V2-Lite-Base-IMat-GGUF model. huggingface.co supports a free trial of the DeepSeek-Coder-V2-Lite-Base-IMat-GGUF model, and also provides paid use of the DeepSeek-Coder-V2-Lite-Base-IMat-GGUF. Support call DeepSeek-Coder-V2-Lite-Base-IMat-GGUF model through api, including Node.js, Python, http.

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF huggingface.co Url

https://huggingface.co/legraphista/DeepSeek-Coder-V2-Lite-Base-IMat-GGUF

legraphista DeepSeek-Coder-V2-Lite-Base-IMat-GGUF online free

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF huggingface.co is an online trial and call api platform, which integrates DeepSeek-Coder-V2-Lite-Base-IMat-GGUF's modeling effects, including api services, and provides a free online trial of DeepSeek-Coder-V2-Lite-Base-IMat-GGUF, you can try DeepSeek-Coder-V2-Lite-Base-IMat-GGUF online for free by clicking the link below.

legraphista DeepSeek-Coder-V2-Lite-Base-IMat-GGUF online free url in huggingface.co:

https://huggingface.co/legraphista/DeepSeek-Coder-V2-Lite-Base-IMat-GGUF

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF install

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF is an open source model from GitHub that offers a free installation service, and any user can find DeepSeek-Coder-V2-Lite-Base-IMat-GGUF on GitHub to install. At the same time, huggingface.co provides the effect of DeepSeek-Coder-V2-Lite-Base-IMat-GGUF install, users can directly use DeepSeek-Coder-V2-Lite-Base-IMat-GGUF installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF install url in huggingface.co:

https://huggingface.co/legraphista/DeepSeek-Coder-V2-Lite-Base-IMat-GGUF

Url of DeepSeek-Coder-V2-Lite-Base-IMat-GGUF

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF huggingface.co Url

Provider of DeepSeek-Coder-V2-Lite-Base-IMat-GGUF huggingface.co

legraphista
ORGANIZATIONS
huggingface.co

Other API from legraphista