Sponsored by Vmake Labs Video Enhancer
Toolify Toolify logo
AI Models
vicuna-7b-v1.5-awq

casperhansen / vicuna-7b-v1.5-awq

Run on huggingface.co
huggingface.co
Total runs: 27
24-hour runs: 2
7-day runs: -5
30-day runs: 5
Model's Last Updated: November 01 2023
text-generation

Introduction of vicuna-7b-v1.5-awq

Model Details of vicuna-7b-v1.5-awq

To use this model, you must have AutoAWQ installed. 

pip install autoawq

Example generation with streaming: 

from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer, TextStreamer

quant_path = "casperhansen/vicuna-7b-v1.5-awq"
quant_file = "awq_model_w4_g128.pt"

# Load model
model = AutoAWQForCausalLM.from_quantized(quant_path, quant_file, fuse_layers=True)
tokenizer = AutoTokenizer.from_pretrained(quant_path, trust_remote_code=True)
streamer = TextStreamer(tokenizer, skip_special_tokens=True)

# Convert prompt to tokens
prompt_template = """\
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.

USER: {prompt}
ASSISTANT:"""

tokens = tokenizer(
    prompt_template.format(prompt="How are you today?"), 
    return_tensors='pt'
).input_ids.cuda()

# Generate output
generation_output = model.generate(
    tokens, 
    streamer=streamer,
    max_new_tokens=512
)

Runs of casperhansen vicuna-7b-v1.5-awq on huggingface.co

27
Total runs
2
24-hour runs
1
3-day runs
-5
7-day runs
5
30-day runs

More Information About vicuna-7b-v1.5-awq huggingface.co Model

More vicuna-7b-v1.5-awq license Visit here:

https://choosealicense.com/licenses/llama2

vicuna-7b-v1.5-awq huggingface.co

vicuna-7b-v1.5-awq huggingface.co is an AI model on huggingface.co that provides vicuna-7b-v1.5-awq's model effect (), which can be used instantly with this casperhansen vicuna-7b-v1.5-awq model. huggingface.co supports a free trial of the vicuna-7b-v1.5-awq model, and also provides paid use of the vicuna-7b-v1.5-awq. Support call vicuna-7b-v1.5-awq model through api, including Node.js, Python, http.

vicuna-7b-v1.5-awq huggingface.co Url

https://huggingface.co/casperhansen/vicuna-7b-v1.5-awq

casperhansen vicuna-7b-v1.5-awq online free

vicuna-7b-v1.5-awq huggingface.co is an online trial and call api platform, which integrates vicuna-7b-v1.5-awq's modeling effects, including api services, and provides a free online trial of vicuna-7b-v1.5-awq, you can try vicuna-7b-v1.5-awq online for free by clicking the link below.

casperhansen vicuna-7b-v1.5-awq online free url in huggingface.co:

https://huggingface.co/casperhansen/vicuna-7b-v1.5-awq

vicuna-7b-v1.5-awq install

vicuna-7b-v1.5-awq is an open source model from GitHub that offers a free installation service, and any user can find vicuna-7b-v1.5-awq on GitHub to install. At the same time, huggingface.co provides the effect of vicuna-7b-v1.5-awq install, users can directly use vicuna-7b-v1.5-awq installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

vicuna-7b-v1.5-awq install url in huggingface.co:

https://huggingface.co/casperhansen/vicuna-7b-v1.5-awq

Url of vicuna-7b-v1.5-awq

vicuna-7b-v1.5-awq huggingface.co Url

Provider of vicuna-7b-v1.5-awq huggingface.co

casperhansen
ORGANIZATIONS
huggingface.co

Other API from casperhansen