Total runs:
783.8K
Run Growth:
348.1K
Growth Rate:
44.58%
Sponsored by
PoYo.AI
Intel / neural-chat-7b-v1-1
Model's Last Updated: January 15 2024
text-generation
Introduction of neural-chat-7b-v1-1
Model Details of neural-chat-7b-v1-1
Model Details: Neural-Chat-v1-1
This model is a fine-tuned model for chat based on mosaicml/mpt-7b with a max sequence length of 2048 on the dataset Intel/neural-chat-dataset-v1-1 , which is a compilation of open-source datasets.
Prompt of "an image of a brain that has to do with LLMs" from https://clipdrop.co/stable-diffusion-turbo.
| Model Detail | Description |
|---|---|
| Model Authors | Intel. The NeuralChat team with members from DCAI/AISE/AIPT. Core team members: Kaokao Lv, Liang Lv, Chang Wang, Wenxin Zhang, Xuhui Ren, and Haihao Shen. |
| Date | July, 2023 |
| Version | v1-1 |
| Type | 7B Large Language Model |
| Paper or Other Resources | Base model: mosaicml/mpt-7b ; Dataset: Intel/neural-chat-dataset-v1-1 |
| License | Apache 2.0 |
| Questions or Comments | Community Tab and Intel DevHub Discord |
| Intended Use | Description |
|---|---|
| Primary intended uses | You can use the fine-tuned model for several language-related tasks. Checkout the LLM Leaderboard to see this model's performance relative to other LLMs. |
| Primary intended users | Anyone doing inference on language-related tasks. |
| Out-of-scope uses | This model in most cases will need to be fine-tuned for your particular task. The model should not be used to intentionally create hostile or alienating environments for people. |
How To Use
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 8
- total_train_batch_size: 64
- total_eval_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.02
- num_epochs: 3.0
Use The Model
Loading the model with Transformers
import transformers
model = transformers.AutoModelForCausalLM.from_pretrained(
'Intel/neural-chat-7b-v1-1',
trust_remote_code=True
)
Inference with INT8
Follow the instructions at the GitHub repository to install the necessary dependencies for quantization to INT8. Use the below command to quantize the model using Intel Neural Compressor to accelerate inference.
python run_generation.py \
--model Intel/neural-chat-7b-v1-1 \
--quantize \
--sq \
--alpha 0.95 \
--ipex
| Factors | Description |
|---|---|
| Groups | More details about the dataset can be found at Intel/neural-chat-dataset-v1-1 . |
| Instrumentation | The performance of the model can vary depending on the inputs to the model. In this case, the prompts provided can drastically change the prediction of the language model. |
| Environment | - |
| Card Prompts | Model deployment on varying hardware and software will change model performance. |
| Metrics | Description |
|---|---|
| Model performance measures | The model metrics are: ARC, HellaSwag, MMLU, and TruthfulQA. Bias evaluation was also evaluated using using Toxicity Rito (see Quantitative Analyses below). The model performance was evaluated against other LLMs according to the standards at the time the model was published. |
| Decision thresholds | No decision thresholds were used. |
| Approaches to uncertainty and variability | - |
Training Data
The training data are from Intel/neural-chat-dataset-v1-1 . The total number of instruction samples is about 1.1M, and the number of tokens is 326M. This dataset is composed of several other datasets:
| Type | Language | Dataset | Number |
|---|---|---|---|
| HC3 | en | HC3 | 24K |
| dolly | en | databricks-dolly-15k | 15K |
| alpaca-zh | zh | tigerbot-alpaca-zh-0.5m | 500K |
| alpaca-en | en | TigerResearch/tigerbot-alpaca-en-50k | 50K |
| math | en | tigerbot-gsm-8k-en | 8K |
| general | en | tigerbot-stackexchange-qa-en-0.5m | 500K |
Note: There is no contamination from the GSM8k test set, as this is not a part of this dataset.
Quantitative Analyses
LLM metrics
We used the same evaluation metrics as HuggingFaceH4/open_llm_leaderboard , which uses Eleuther AI Language Model Evaluation Harness , a unified framework to test generative language models on a large number of different evaluation tasks.
| Model | Average ⬆️ | ARC (25-s) ⬆️ | HellaSwag (10-s) ⬆️ | MMLU (5-s) ⬆️ | TruthfulQA (MC) (0-s) ⬆️ |
|---|---|---|---|---|---|
| mosaicml/mpt-7b | 47.4 | 47.61 | 77.56 | 31 | 33.43 |
| mosaicml/mpt-7b-chat | 49.95 | 46.5 | 75.55 | 37.60 | 40.17 |
| Intel/neural-chat-dataset-v1-1 | 51.41 | 50.09 | 76.69 | 38.79 | 40.07 |
Bias evaluation
Following the blog evaluating-llm-bias , we selected 10000 samples randomly from allenai/real-toxicity-prompts to evaluate toxicity bias.
| Model | Toxicity Rito ↓ |
|---|---|
| mosaicml/mpt-7b | 0.027 |
| Intel/neural-chat-dataset-v1-1 | 0.0264 |
Examples
-
code generation
-
summarization
-
trip
Ethical Considerations and Limitations
Neural-chat-7b-v1-1 can produce factually incorrect output, and should not be relied on to produce factually accurate information. neural-chat-7b-v1-1 was trained on various instruction/chat datasets based on mosaicml/mpt-7b . Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
Therefore, before deploying any applications of the model, developers should perform safety testing.
Caveats and Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
Here are some useful GitHub repository links to learn more about Intel's open-source AI software:
Disclaimer
The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please cosult an attorney before using this model for commercial purposes.
Runs of Intel neural-chat-7b-v1-1 on huggingface.co
20
Total runs
0
24-hour runs
1
3-day runs
2
7-day runs
-95
30-day runs
More Information About neural-chat-7b-v1-1 huggingface.co Model
Url of neural-chat-7b-v1-1
neural-chat-7b-v1-1 huggingface.co Url
Provider of neural-chat-7b-v1-1 huggingface.co
Other API from Intel
Updated:February 09 2024
Total runs:
285.5K
Run Growth:
241.7K
Growth Rate:
84.66%
Total runs:
271.1K
Run Growth:
102.5K
Growth Rate:
37.82%
Total runs:
135.9K
Run Growth:
117.4K
Growth Rate:
86.36%
Total runs:
103.8K
Run Growth:
-11.7K
Growth Rate:
-11.27%
Total runs:
69.3K
Run Growth:
-256.0K
Growth Rate:
-369.34%
Total runs:
40.1K
Run Growth:
38.8K
Growth Rate:
99.90%
Total runs:
32.7K
Run Growth:
25.2K
Growth Rate:
77.06%
Total runs:
31.3K
Run Growth:
-26.6K
Growth Rate:
-85.97%
Total runs:
25.9K
Run Growth:
4.9K
Growth Rate:
18.82%
Total runs:
24.9K
Run Growth:
24.9K
Growth Rate:
100.00%
Total runs:
22.0K
Run Growth:
6.7K
Growth Rate:
30.56%
Total runs:
21.1K
Run Growth:
4.2K
Growth Rate:
20.05%
Total runs:
18.7K
Run Growth:
-2.0K
Growth Rate:
-10.57%
Total runs:
17.9K
Run Growth:
849
Growth Rate:
4.75%
Total runs:
15.5K
Run Growth:
-65.4K
Growth Rate:
-420.79%
Total runs:
15.4K
Run Growth:
14.9K
Growth Rate:
96.66%
Total runs:
11.6K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9.1K
Run Growth:
9.1K
Growth Rate:
100.00%
Total runs:
6.8K
Run Growth:
-20.5K
Growth Rate:
-300.86%
Total runs:
5.4K
Run Growth:
-40
Growth Rate:
-0.74%
Total runs:
5.3K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
5.0K
Run Growth:
-96
Growth Rate:
-1.91%
Total runs:
4.2K
Run Growth:
2.9K
Growth Rate:
70.20%
Total runs:
3.5K
Run Growth:
-216
Growth Rate:
-6.12%
Total runs:
3.5K
Run Growth:
2.9K
Growth Rate:
82.29%
Total runs:
3.4K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
3.3K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
3.3K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
3.2K
Run Growth:
3.0K
Growth Rate:
93.71%
Total runs:
3.1K
Run Growth:
875
Growth Rate:
28.41%
Total runs:
2.5K
Run Growth:
2.3K
Growth Rate:
95.61%
Total runs:
1.9K
Run Growth:
1.9K
Growth Rate:
100.00%
Total runs:
1.9K
Run Growth:
1.2K
Growth Rate:
65.21%
Total runs:
1.8K
Run Growth:
160
Growth Rate:
8.91%
Total runs:
1.7K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
1.7K
Run Growth:
-83
Growth Rate:
-4.90%
Total runs:
1.6K
Run Growth:
-1.8K
Growth Rate:
-115.87%
Total runs:
1.4K
Run Growth:
-171
Growth Rate:
-12.01%
Total runs:
1.2K
Run Growth:
457
Growth Rate:
38.08%
Total runs:
997
Run Growth:
-29
Growth Rate:
-2.91%
Total runs:
984
Run Growth:
652
Growth Rate:
66.26%
Total runs:
966
Run Growth:
718
Growth Rate:
74.33%
Total runs:
907
Run Growth:
-1.4K
Growth Rate:
-154.02%
Total runs:
852
Run Growth:
-29
Growth Rate:
-3.40%
Total runs:
843
Run Growth:
0
Growth Rate:
0.00%
Total runs:
809
Run Growth:
0
Growth Rate:
0.00%
Total runs:
808
Run Growth:
689
Growth Rate:
85.27%
Total runs:
790
Run Growth:
790
Growth Rate:
100.00%
Total runs:
744
Run Growth:
651
Growth Rate:
87.50%
Total runs:
646
Run Growth:
-1.5K
Growth Rate:
-225.89%
Total runs:
633
Run Growth:
558
Growth Rate:
88.15%
Total runs:
612
Run Growth:
487
Growth Rate:
79.58%
Total runs:
610
Run Growth:
308
Growth Rate:
50.49%
Total runs:
602
Run Growth:
37
Growth Rate:
6.15%
Total runs:
598
Run Growth:
-391
Growth Rate:
-65.38%
Total runs:
530
Run Growth:
12
Growth Rate:
2.26%
Total runs:
530
Run Growth:
530
Growth Rate:
100.00%
Total runs:
529
Run Growth:
252
Growth Rate:
47.64%
Total runs:
522
Run Growth:
499
Growth Rate:
95.59%
Total runs:
515
Run Growth:
209
Growth Rate:
40.58%
Total runs:
490
Run Growth:
-1.3K
Growth Rate:
-266.06%
Total runs:
457
Run Growth:
364
Growth Rate:
79.65%
Total runs:
456
Run Growth:
59
Growth Rate:
12.94%
Total runs:
450
Run Growth:
3
Growth Rate:
0.67%
Total runs:
431
Run Growth:
418
Growth Rate:
96.98%
Total runs:
418
Run Growth:
372
Growth Rate:
89.00%
Total runs:
416
Run Growth:
70
Growth Rate:
16.83%
Total runs:
382
Run Growth:
52
Growth Rate:
14.29%
Total runs:
365
Run Growth:
48
Growth Rate:
13.15%
Total runs:
355
Run Growth:
10
Growth Rate:
2.82%
Total runs:
340
Run Growth:
340
Growth Rate:
100.00%
Total runs:
317
Run Growth:
317
Growth Rate:
100.00%
Total runs:
311
Run Growth:
-13
Growth Rate:
-4.18%
Total runs:
309
Run Growth:
-791
Growth Rate:
-254.34%
Total runs:
309
Run Growth:
285
Growth Rate:
92.23%
Total runs:
306
Run Growth:
-3.5K
Growth Rate:
-1094.62%
Total runs:
298
Run Growth:
0
Growth Rate:
0.00%
Total runs:
290
Run Growth:
-2
Growth Rate:
-0.69%
Total runs:
284
Run Growth:
-1.4K
Growth Rate:
-480.99%
Total runs:
251
Run Growth:
71
Growth Rate:
28.29%
Total runs:
238
Run Growth:
0
Growth Rate:
0.00%
Total runs:
230
Run Growth:
0
Growth Rate:
0.00%
Total runs:
230
Run Growth:
0
Growth Rate:
0.00%
Total runs:
229
Run Growth:
80
Growth Rate:
35.09%
Total runs:
209
Run Growth:
19
Growth Rate:
9.09%
Total runs:
200
Run Growth:
200
Growth Rate:
100.00%
Total runs:
198
Run Growth:
42
Growth Rate:
21.21%
Total runs:
192
Run Growth:
134
Growth Rate:
70.16%
Total runs:
184
Run Growth:
-1.8K
Growth Rate:
-988.59%