Sponsored by Skywork
Toolify Toolify logo
AI Models
chatglm3-ggml

Chars / chatglm3-ggml

Run on huggingface.co
huggingface.co
Total runs: 0
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: November 06 2023

Introduction of chatglm3-ggml

Model Details of chatglm3-ggml

ChatGLM3 模型量化

本项目用于演示如何使用 ChatGLM3 模型，并通过 ChatGLM.cpp 工具仓库对模型进行量化。

拉取模型和工具仓库

首先，需要将 ChatGLM3 模型和 ChatGLM.cpp 工具仓库拉取到本地： 

git lfs install
git clone https://huggingface.co/THUDM/chatglm3-6b
git clone https://github.com/li-plus/chatglm.cpp.git
安装依赖

在开始量化之前，请确保已安装以下依赖项： 

python3 -m pip install -U -q pip
python3 -m pip install -q torch tabulate tqdm transformers accelerate sentencepiece
模型量化

使用 ChatGLM.cpp 工具仓库中的 convert.py 脚本，可以对 ChatGLM3 模型进行量化。 

types=("q4_0" "q4_1" "q5_0" "q5_1" "q8_0")

for type_str in "${types[@]}"; do
  python3 ./chatglm.cpp/chatglm_cpp/convert.py -i ./chatglm3-6b -t ${type_str} -o chatglm-ggml_${type_str}.bin
done

上述代码会对 ChatGLM3 模型进行多个量化操作，并生成对应的量化模型文件。量化类型包括："q4_0"、"q4_1"、"q5_0"、"q5_1" 和 "q8_0"。

请确保已在终端中切换到正确的工作目录，然后执行上述代码段。

量化后的模型文件将以 chatglm-ggml_{type_str}.bin 的形式保存在当前目录下。

如果您有任何问题，请随时联系我们。

模型地址： https://huggingface.co/THUDM/chatglm3-6b

工具仓库地址： https://github.com/li-plus/chatglm.cpp

Runs of Chars chatglm3-ggml on huggingface.co

0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About chatglm3-ggml huggingface.co Model

More chatglm3-ggml license Visit here:

https://choosealicense.com/licenses/apache-2.0

chatglm3-ggml huggingface.co

chatglm3-ggml huggingface.co is an AI model on huggingface.co that provides chatglm3-ggml's model effect (), which can be used instantly with this Chars chatglm3-ggml model. huggingface.co supports a free trial of the chatglm3-ggml model, and also provides paid use of the chatglm3-ggml. Support call chatglm3-ggml model through api, including Node.js, Python, http.

chatglm3-ggml huggingface.co Url

https://huggingface.co/Chars/chatglm3-ggml

Chars chatglm3-ggml online free

chatglm3-ggml huggingface.co is an online trial and call api platform, which integrates chatglm3-ggml's modeling effects, including api services, and provides a free online trial of chatglm3-ggml, you can try chatglm3-ggml online for free by clicking the link below.

Chars chatglm3-ggml online free url in huggingface.co:

https://huggingface.co/Chars/chatglm3-ggml

chatglm3-ggml install

chatglm3-ggml is an open source model from GitHub that offers a free installation service, and any user can find chatglm3-ggml on GitHub to install. At the same time, huggingface.co provides the effect of chatglm3-ggml install, users can directly use chatglm3-ggml installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

chatglm3-ggml install url in huggingface.co:

https://huggingface.co/Chars/chatglm3-ggml

Url of chatglm3-ggml

chatglm3-ggml huggingface.co Url

Provider of chatglm3-ggml huggingface.co

Chars
ORGANIZATIONS
huggingface.co

Other API from Chars

 huggingface.co
Chars/AI-baki

Total runs: 1
Run Growth: -2
Growth Rate: -200.00%
Updated:December 24 2023
huggingface.co
Chars/rockyou2024

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated:July 14 2024