solidrust / Mistral-7B-v0.1-flashback-v2-instruct-AWQ

huggingface.co
Total runs: 22
24-hour runs: 0
7-day runs: 5
30-day runs: 11
Model's Last Updated: September 03 2024
text-generation

Introduction of Mistral-7B-v0.1-flashback-v2-instruct-AWQ

Model Details of Mistral-7B-v0.1-flashback-v2-instruct-AWQ

timpal0l/Mistral-7B-v0.1-flashback-v2-instruct AWQ

Model SUmmary

Mistral-7B-v0.1-flashback-v2-instruct is an instruct based version of the base model timpal0l/Mistral-7B-v0.1-flashback-v2 . It has been finetuned on a the machine translated instruct dataset OpenHermes2.5 .

How to use
Install the necessary packages
pip install --upgrade autoawq autoawq-kernels
Example Python code
from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer, TextStreamer

model_path = "solidrust/Mistral-7B-v0.1-flashback-v2-instruct-AWQ"
system_message = "You are Mistral-7B-v0.1-flashback-v2-instruct, incarnated as a powerful AI. You were created by timpal0l."

# Load model
model = AutoAWQForCausalLM.from_quantized(model_path,
                                          fuse_layers=True)
tokenizer = AutoTokenizer.from_pretrained(model_path,
                                          trust_remote_code=True)
streamer = TextStreamer(tokenizer,
                        skip_prompt=True,
                        skip_special_tokens=True)

# Convert prompt to tokens
prompt_template = """\
<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant"""

prompt = "You're standing on the surface of the Earth. "\
        "You walk one mile south, one mile west and one mile north. "\
        "You end up exactly where you started. Where are you?"

tokens = tokenizer(prompt_template.format(system_message=system_message,prompt=prompt),
                  return_tensors='pt').input_ids.cuda()

# Generate output
generation_output = model.generate(tokens,
                                  streamer=streamer,
                                  max_new_tokens=512)
About AWQ

AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference with equivalent or better quality compared to the most commonly used GPTQ settings.

AWQ models are currently supported on Linux and Windows, with NVidia GPUs only. macOS users: please use GGUF models instead.

It is supported by:

Runs of solidrust Mistral-7B-v0.1-flashback-v2-instruct-AWQ on huggingface.co

22
Total runs
0
24-hour runs
0
3-day runs
5
7-day runs
11
30-day runs

More Information About Mistral-7B-v0.1-flashback-v2-instruct-AWQ huggingface.co Model

More Mistral-7B-v0.1-flashback-v2-instruct-AWQ license Visit here:

https://choosealicense.com/licenses/mit

Mistral-7B-v0.1-flashback-v2-instruct-AWQ huggingface.co

Mistral-7B-v0.1-flashback-v2-instruct-AWQ huggingface.co is an AI model on huggingface.co that provides Mistral-7B-v0.1-flashback-v2-instruct-AWQ's model effect (), which can be used instantly with this solidrust Mistral-7B-v0.1-flashback-v2-instruct-AWQ model. huggingface.co supports a free trial of the Mistral-7B-v0.1-flashback-v2-instruct-AWQ model, and also provides paid use of the Mistral-7B-v0.1-flashback-v2-instruct-AWQ. Support call Mistral-7B-v0.1-flashback-v2-instruct-AWQ model through api, including Node.js, Python, http.

Mistral-7B-v0.1-flashback-v2-instruct-AWQ huggingface.co Url

https://huggingface.co/solidrust/Mistral-7B-v0.1-flashback-v2-instruct-AWQ

solidrust Mistral-7B-v0.1-flashback-v2-instruct-AWQ online free

Mistral-7B-v0.1-flashback-v2-instruct-AWQ huggingface.co is an online trial and call api platform, which integrates Mistral-7B-v0.1-flashback-v2-instruct-AWQ's modeling effects, including api services, and provides a free online trial of Mistral-7B-v0.1-flashback-v2-instruct-AWQ, you can try Mistral-7B-v0.1-flashback-v2-instruct-AWQ online for free by clicking the link below.

solidrust Mistral-7B-v0.1-flashback-v2-instruct-AWQ online free url in huggingface.co:

https://huggingface.co/solidrust/Mistral-7B-v0.1-flashback-v2-instruct-AWQ

Mistral-7B-v0.1-flashback-v2-instruct-AWQ install

Mistral-7B-v0.1-flashback-v2-instruct-AWQ is an open source model from GitHub that offers a free installation service, and any user can find Mistral-7B-v0.1-flashback-v2-instruct-AWQ on GitHub to install. At the same time, huggingface.co provides the effect of Mistral-7B-v0.1-flashback-v2-instruct-AWQ install, users can directly use Mistral-7B-v0.1-flashback-v2-instruct-AWQ installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Mistral-7B-v0.1-flashback-v2-instruct-AWQ install url in huggingface.co:

https://huggingface.co/solidrust/Mistral-7B-v0.1-flashback-v2-instruct-AWQ

Url of Mistral-7B-v0.1-flashback-v2-instruct-AWQ

Mistral-7B-v0.1-flashback-v2-instruct-AWQ huggingface.co Url

Provider of Mistral-7B-v0.1-flashback-v2-instruct-AWQ huggingface.co

solidrust
ORGANIZATIONS

Other API from solidrust