cmarkea / Meta-Llama-3.1-70B-Instruct-4bit

huggingface.co
Total runs: 13
24-hour runs: 0
7-day runs: 0
30-day runs: -7
Model's Last Updated: November 14 2024
text-generation

Introduction of Meta-Llama-3.1-70B-Instruct-4bit

Model Details of Meta-Llama-3.1-70B-Instruct-4bit

Converted version of Meta-Llama-3.1-70B-Instruct to 4-bit using bitsandbytes. For more information about the model, refer to the model's page.

Impact on performance

Impact of quantization on a set of models.

Evaluation of the model was conducted using the PoLL (Pool of LLM) technique, assessing performance on 100 French questions with scores aggregated from six evaluations (two per evaluator). The evaluators included GPT-4o, Gemini-1.5-pro, and Claude3.5-sonnet.

Performance Scores (on a scale of 5):

Model Score # params (Billion) size (GB)
gpt-4o 4.13 N/A N/A
mistralai/Mixtral-8x7B-Instruct-v0.1 3.71 46.7 93.4
cmarkea/Mixtral-8x7B-Instruct-v0.1-4bit 3.68 46.7 23.35
meta-llama/Meta-Llama-3.1-70B-Instruct 3.68 70.06 140.12
gpt-3.5-turbo 3.66 175 350
cmarkea/Meta-Llama-3.1-70B-Instruct-4bit 3.64 70.06 35.3
TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ 3.56 46.7 46.7
meta-llama/Meta-Llama-3.1-8B-Instruct 3.25 8.03 16.06
mistralai/Mistral-7B-Instruct-v0.2 1.98 7.25 14.5
cmarkea/bloomz-7b1-mt-sft-chat 1.69 7.07 14.14
cmarkea/bloomz-3b-dpo-chat 1.68 3 6
cmarkea/bloomz-3b-sft-chat 1.51 3 6
croissantllm/CroissantLLMChat-v0.1 1.19 1.3 2.7
cmarkea/bloomz-560m-sft-chat 1.04 0.56 1.12
OpenLLM-France/Claire-Mistral-7B-0.1 0.38 7.25 14.5

The impact of quantization is negligible.

Prompt Pattern

Here is a reminder of the command pattern to interact with the model:

<|start_header_id|>user<|end_header_id|>\n\n{user_prompt_1}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n{model_answer_1}...

Runs of cmarkea Meta-Llama-3.1-70B-Instruct-4bit on huggingface.co

13
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
-7
30-day runs

More Information About Meta-Llama-3.1-70B-Instruct-4bit huggingface.co Model

More Meta-Llama-3.1-70B-Instruct-4bit license Visit here:

https://choosealicense.com/licenses/llama3.1

Meta-Llama-3.1-70B-Instruct-4bit huggingface.co

Meta-Llama-3.1-70B-Instruct-4bit huggingface.co is an AI model on huggingface.co that provides Meta-Llama-3.1-70B-Instruct-4bit's model effect (), which can be used instantly with this cmarkea Meta-Llama-3.1-70B-Instruct-4bit model. huggingface.co supports a free trial of the Meta-Llama-3.1-70B-Instruct-4bit model, and also provides paid use of the Meta-Llama-3.1-70B-Instruct-4bit. Support call Meta-Llama-3.1-70B-Instruct-4bit model through api, including Node.js, Python, http.

Meta-Llama-3.1-70B-Instruct-4bit huggingface.co Url

https://huggingface.co/cmarkea/Meta-Llama-3.1-70B-Instruct-4bit

cmarkea Meta-Llama-3.1-70B-Instruct-4bit online free

Meta-Llama-3.1-70B-Instruct-4bit huggingface.co is an online trial and call api platform, which integrates Meta-Llama-3.1-70B-Instruct-4bit's modeling effects, including api services, and provides a free online trial of Meta-Llama-3.1-70B-Instruct-4bit, you can try Meta-Llama-3.1-70B-Instruct-4bit online for free by clicking the link below.

cmarkea Meta-Llama-3.1-70B-Instruct-4bit online free url in huggingface.co:

https://huggingface.co/cmarkea/Meta-Llama-3.1-70B-Instruct-4bit

Meta-Llama-3.1-70B-Instruct-4bit install

Meta-Llama-3.1-70B-Instruct-4bit is an open source model from GitHub that offers a free installation service, and any user can find Meta-Llama-3.1-70B-Instruct-4bit on GitHub to install. At the same time, huggingface.co provides the effect of Meta-Llama-3.1-70B-Instruct-4bit install, users can directly use Meta-Llama-3.1-70B-Instruct-4bit installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Meta-Llama-3.1-70B-Instruct-4bit install url in huggingface.co:

https://huggingface.co/cmarkea/Meta-Llama-3.1-70B-Instruct-4bit

Url of Meta-Llama-3.1-70B-Instruct-4bit

Meta-Llama-3.1-70B-Instruct-4bit huggingface.co Url

Provider of Meta-Llama-3.1-70B-Instruct-4bit huggingface.co

cmarkea
ORGANIZATIONS

Other API from cmarkea