If the model file is big, it has been split into multiple files. In order to download them all to a local folder, run:
huggingface-cli download legraphista/Phi-3-mini-128k-instruct-IMat-GGUF --include "Phi-3-mini-128k-instruct.Q8_0/*" --local-dir Phi-3-mini-128k-instruct.Q8_0
# see FAQ for merging GGUF's
Inference
Simple chat template
<s><|user|>
Can you provide ways to eat combinations of bananas and dragonfruits?<|end|>
<|assistant|>
Sure! Here are some ways to eat bananas and dragonfruits together:
1. Banana and dragonfruit smoothie: Blend bananas and dragonfruits together with some milk and honey.
2. Banana and dragonfruit salad: Mix sliced bananas and dragonfruits together with some lemon juice and honey.<|end|>
<|user|>
What about solving an 2x + 3 = 7 equation?<|end|>
<|assistant|>
Llama.cpp
llama.cpp/main -m Phi-3-mini-128k-instruct.Q8_0.gguf --color -i -p "prompt here (according to the chat template)"
FAQ
Why is the IMatrix not applied everywhere?
According to
this investigation
, it appears that lower quantizations are the only ones that benefit from the imatrix input (as per hellaswag results).
Phi-3-mini-128k-instruct-IMat-GGUF huggingface.co is an AI model on huggingface.co that provides Phi-3-mini-128k-instruct-IMat-GGUF's model effect (), which can be used instantly with this legraphista Phi-3-mini-128k-instruct-IMat-GGUF model. huggingface.co supports a free trial of the Phi-3-mini-128k-instruct-IMat-GGUF model, and also provides paid use of the Phi-3-mini-128k-instruct-IMat-GGUF. Support call Phi-3-mini-128k-instruct-IMat-GGUF model through api, including Node.js, Python, http.
Phi-3-mini-128k-instruct-IMat-GGUF huggingface.co is an online trial and call api platform, which integrates Phi-3-mini-128k-instruct-IMat-GGUF's modeling effects, including api services, and provides a free online trial of Phi-3-mini-128k-instruct-IMat-GGUF, you can try Phi-3-mini-128k-instruct-IMat-GGUF online for free by clicking the link below.
legraphista Phi-3-mini-128k-instruct-IMat-GGUF online free url in huggingface.co:
Phi-3-mini-128k-instruct-IMat-GGUF is an open source model from GitHub that offers a free installation service, and any user can find Phi-3-mini-128k-instruct-IMat-GGUF on GitHub to install. At the same time, huggingface.co provides the effect of Phi-3-mini-128k-instruct-IMat-GGUF install, users can directly use Phi-3-mini-128k-instruct-IMat-GGUF installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
Phi-3-mini-128k-instruct-IMat-GGUF install url in huggingface.co: