Total runs:
90
Run Growth:
0
Growth Rate:
0.00%
Sponsored by
PostMcpAi
iproskurina / Mistral-7B-gptq-4bit
Model's Last Updated: September 23 2024
text-generation
Introduction of Mistral-7B-gptq-4bit
Model Details of Mistral-7B-gptq-4bit
Mistral 7B v0.1 - GPTQ
- Model creator: Mistral AI
- Original model: Mistral 7B v0.1
The model published in this repo was quantized to 4bit using AutoGPTQ .
Quantization details
All quantization parameters were taken from GPTQ paper .
GPTQ calibration data consisted of 128 random 2048 token segments from the C4 dataset .
The grouping size used for quantization is equal to 128.
How to use this GPTQ model from Python code
Install the necessary packages
pip install accelerate==0.26.1 datasets==2.16.1 dill==0.3.7 gekko==1.0.6 multiprocess==0.70.15 peft==0.7.1 rouge==1.0.1 sentencepiece==0.1.99
git clone https://github.com/upunaprosk/AutoGPTQ
cd AutoGPTQ
pip install -v .
Recommended transformers version: 4.35.2.
You can then use the following code
from transformers import AutoTokenizer, TextGenerationPipeline,AutoModelForCausalLM
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
pretrained_model_dir = "iproskurina/Mistral-7B-gptq-4bit"
tokenizer = AutoTokenizer.from_pretrained(pretrained_model_dir, use_fast=True)
model = AutoGPTQForCausalLM.from_quantized(pretrained_model_dir, device="cuda:0", model_basename="model")
pipeline = TextGenerationPipeline(model=model, tokenizer=tokenizer)
print(pipeline("auto-gptq is")[0]["generated_text"])
Runs of iproskurina Mistral-7B-gptq-4bit on huggingface.co
292
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About Mistral-7B-gptq-4bit huggingface.co Model
Url of Mistral-7B-gptq-4bit
Mistral-7B-gptq-4bit huggingface.co Url
Provider of Mistral-7B-gptq-4bit huggingface.co
Other API from iproskurina
Updated:January 28 2024
Total runs:
87
Run Growth:
56
Growth Rate:
70.00%
Total runs:
83
Run Growth:
0
Growth Rate:
0.00%
Total runs:
49
Run Growth:
44
Growth Rate:
89.80%
Total runs:
48
Run Growth:
0
Growth Rate:
0.00%
Total runs:
45
Run Growth:
0
Growth Rate:
0.00%
Total runs:
37
Run Growth:
23
Growth Rate:
63.89%
Total runs:
33
Run Growth:
0
Growth Rate:
0.00%
Total runs:
31
Run Growth:
0
Growth Rate:
0.00%
Total runs:
31
Run Growth:
0
Growth Rate:
0.00%
Total runs:
31
Run Growth:
0
Growth Rate:
0.00%
Total runs:
31
Run Growth:
9
Growth Rate:
30.00%
Total runs:
29
Run Growth:
-5
Growth Rate:
-17.24%
Total runs:
28
Run Growth:
-2
Growth Rate:
-7.14%
Total runs:
28
Run Growth:
23
Growth Rate:
85.19%
Total runs:
27
Run Growth:
0
Growth Rate:
0.00%
Total runs:
27
Run Growth:
0
Growth Rate:
0.00%
Total runs:
26
Run Growth:
19
Growth Rate:
76.00%
Total runs:
26
Run Growth:
21
Growth Rate:
87.50%
Total runs:
26
Run Growth:
13
Growth Rate:
50.00%
Total runs:
25
Run Growth:
0
Growth Rate:
0.00%
Total runs:
25
Run Growth:
-3
Growth Rate:
-11.11%
Total runs:
22
Run Growth:
0
Growth Rate:
0.00%
Total runs:
22
Run Growth:
7
Growth Rate:
31.82%
Total runs:
22
Run Growth:
0
Growth Rate:
0.00%
Total runs:
22
Run Growth:
0
Growth Rate:
0.00%
Total runs:
21
Run Growth:
0
Growth Rate:
0.00%
Total runs:
21
Run Growth:
21
Growth Rate:
100.00%
Total runs:
20
Run Growth:
0
Growth Rate:
0.00%
Total runs:
20
Run Growth:
0
Growth Rate:
0.00%
Total runs:
20
Run Growth:
0
Growth Rate:
0.00%
Total runs:
20
Run Growth:
0
Growth Rate:
0.00%
Total runs:
20
Run Growth:
0
Growth Rate:
0.00%
Total runs:
19
Run Growth:
0
Growth Rate:
0.00%
Total runs:
19
Run Growth:
-35
Growth Rate:
-184.21%
Total runs:
19
Run Growth:
0
Growth Rate:
0.00%
Total runs:
19
Run Growth:
0
Growth Rate:
0.00%
Total runs:
19
Run Growth:
0
Growth Rate:
0.00%
Total runs:
19
Run Growth:
19
Growth Rate:
100.00%
Total runs:
18
Run Growth:
0
Growth Rate:
0.00%
Total runs:
18
Run Growth:
0
Growth Rate:
0.00%
Total runs:
17
Run Growth:
0
Growth Rate:
0.00%
Total runs:
17
Run Growth:
0
Growth Rate:
0.00%
Total runs:
17
Run Growth:
0
Growth Rate:
0.00%
Total runs:
17
Run Growth:
-5
Growth Rate:
-31.25%
Total runs:
17
Run Growth:
0
Growth Rate:
0.00%
Total runs:
16
Run Growth:
0
Growth Rate:
0.00%
Total runs:
16
Run Growth:
0
Growth Rate:
0.00%
Total runs:
16
Run Growth:
0
Growth Rate:
0.00%
Total runs:
16
Run Growth:
0
Growth Rate:
0.00%
Total runs:
15
Run Growth:
-4
Growth Rate:
-26.67%
Total runs:
15
Run Growth:
9
Growth Rate:
60.00%
Total runs:
15
Run Growth:
0
Growth Rate:
0.00%
Total runs:
15
Run Growth:
2
Growth Rate:
13.33%
Total runs:
15
Run Growth:
4
Growth Rate:
26.67%
Total runs:
15
Run Growth:
0
Growth Rate:
0.00%
Total runs:
15
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
9
Growth Rate:
69.23%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
10
Growth Rate:
71.43%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
13
Run Growth:
3
Growth Rate:
27.27%
Total runs:
13
Run Growth:
8
Growth Rate:
66.67%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
-3
Growth Rate:
-25.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
-2
Growth Rate:
-16.67%
Total runs:
12
Run Growth:
-2
Growth Rate:
-16.67%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11
Run Growth:
-23
Growth Rate:
-209.09%