llama.cpp/main -m DeepSeek-Coder-V2-Instruct-0724.Q8_0.gguf --color -i -p "prompt here (according to the chat template)"
FAQ
Why is the IMatrix not applied everywhere?
According to
this investigation
, it appears that lower quantizations are the only ones that benefit from the imatrix input (as per hellaswag results).
Download the appropriate zip for your system from the latest release
Unzip the archive and you should be able to find
gguf-split
Locate your GGUF chunks folder (ex:
DeepSeek-Coder-V2-Instruct-0724.Q8_0
)
Run
gguf-split --merge DeepSeek-Coder-V2-Instruct-0724.Q8_0/DeepSeek-Coder-V2-Instruct-0724.Q8_0-00001-of-XXXXX.gguf DeepSeek-Coder-V2-Instruct-0724.Q8_0.gguf
Make sure to point
gguf-split
to the first chunk of the split.
DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF huggingface.co is an AI model on huggingface.co that provides DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF's model effect (), which can be used instantly with this legraphista DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF model. huggingface.co supports a free trial of the DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF model, and also provides paid use of the DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF. Support call DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF model through api, including Node.js, Python, http.
DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF huggingface.co is an online trial and call api platform, which integrates DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF's modeling effects, including api services, and provides a free online trial of DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF, you can try DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF online for free by clicking the link below.
legraphista DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF online free url in huggingface.co:
DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF is an open source model from GitHub that offers a free installation service, and any user can find DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF on GitHub to install. At the same time, huggingface.co provides the effect of DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF install, users can directly use DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF install url in huggingface.co: