I took layers of the original Mistral-7B, and duplicated some layer, this is the first frankeinstein method that I found "acceptable" to expend Mistral.
It seems that the first 8 layers of the model is very important, having duplicate of those layers in the model make me think it confuse the model.
UPDATE: Forced mergekit to output bfloat16 file, should be the same thing, but since the base model is bfloat16, wanted it to stay bf16 like the OG model. Even if it was written bfloat16 in the config file earlier, it was float16.
Description
This repo contains fp16 files of Mistral-11B-v0.1.
Mistral-11B-v0.1 huggingface.co is an AI model on huggingface.co that provides Mistral-11B-v0.1's model effect (), which can be used instantly with this Undi95 Mistral-11B-v0.1 model. huggingface.co supports a free trial of the Mistral-11B-v0.1 model, and also provides paid use of the Mistral-11B-v0.1. Support call Mistral-11B-v0.1 model through api, including Node.js, Python, http.
Mistral-11B-v0.1 huggingface.co is an online trial and call api platform, which integrates Mistral-11B-v0.1's modeling effects, including api services, and provides a free online trial of Mistral-11B-v0.1, you can try Mistral-11B-v0.1 online for free by clicking the link below.
Undi95 Mistral-11B-v0.1 online free url in huggingface.co:
Mistral-11B-v0.1 is an open source model from GitHub that offers a free installation service, and any user can find Mistral-11B-v0.1 on GitHub to install. At the same time, huggingface.co provides the effect of Mistral-11B-v0.1 install, users can directly use Mistral-11B-v0.1 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.