Q_4_K_M is recommanded (good for running with CPU as well)
Q_5_K_M (good in general)
Q_6_K is good also; if you want a better result; take this one instead of Q_5_K_M
Q_8_0 which is very good; need a reasonable size of RAM otherwise you might expect a long wait
16-bit and 32-bit are not provided here; since the file size is similar to the original safetensors; once you have a GPU, go ahead with the saftetensors, pretty much the same
how to run it
use any connector for interacting with gguf; i.e.,
gguf-connector
phi3 huggingface.co is an AI model on huggingface.co that provides phi3's model effect (), which can be used instantly with this calcuis phi3 model. huggingface.co supports a free trial of the phi3 model, and also provides paid use of the phi3. Support call phi3 model through api, including Node.js, Python, http.
phi3 huggingface.co is an online trial and call api platform, which integrates phi3's modeling effects, including api services, and provides a free online trial of phi3, you can try phi3 online for free by clicking the link below.
phi3 is an open source model from GitHub that offers a free installation service, and any user can find phi3 on GitHub to install. At the same time, huggingface.co provides the effect of phi3 install, users can directly use phi3 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.