EmbeddedLLM / llama-2-7b-chat-int4-onnx-directml

huggingface.co
Total runs: 7
24-hour runs: 1
7-day runs: 0
30-day runs: -6
Model's Last Updated: June 19 2024
text-generation

Introduction of llama-2-7b-chat-int4-onnx-directml

Model Details of llama-2-7b-chat-int4-onnx-directml

Llama-2-7b-chat ONNX models for DirectML

This repository hosts the optimized versions of meta-llama/Llama-2-7b-chat-hf to accelerate inference with ONNX Runtime for DirectML.

Usage on Windows (Intel / AMD / Nvidia / Qualcomm)
conda create -n onnx python=3.10
conda activate onnx
winget install -e --id GitHub.GitLFS
pip install huggingface-hub[cli]
huggingface-cli download EmbeddedLLM/llama-2-7b-chat-int4-onnx-directml --local-dir .\llama-2-7b-chat
pip install numpy==1.26.4
Invoke-WebRequest -Uri "https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/phi3-qa.py" -OutFile "phi3-qa.py"
pip install onnxruntime-directml
pip install --pre onnxruntime-genai-directml
conda install conda-forge::vs2015_runtime
python phi3-qa.py -m .\llama-2-7b-chat
What is DirectML

DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported hardware and drivers, including all DirectX 12-capable GPUs from vendors such as AMD, Intel, NVIDIA, and Qualcomm.

Runs of EmbeddedLLM llama-2-7b-chat-int4-onnx-directml on huggingface.co

7
Total runs
1
24-hour runs
1
3-day runs
0
7-day runs
-6
30-day runs

More Information About llama-2-7b-chat-int4-onnx-directml huggingface.co Model

More llama-2-7b-chat-int4-onnx-directml license Visit here:

https://choosealicense.com/licenses/llama2

llama-2-7b-chat-int4-onnx-directml huggingface.co

llama-2-7b-chat-int4-onnx-directml huggingface.co is an AI model on huggingface.co that provides llama-2-7b-chat-int4-onnx-directml's model effect (), which can be used instantly with this EmbeddedLLM llama-2-7b-chat-int4-onnx-directml model. huggingface.co supports a free trial of the llama-2-7b-chat-int4-onnx-directml model, and also provides paid use of the llama-2-7b-chat-int4-onnx-directml. Support call llama-2-7b-chat-int4-onnx-directml model through api, including Node.js, Python, http.

llama-2-7b-chat-int4-onnx-directml huggingface.co Url

https://huggingface.co/EmbeddedLLM/llama-2-7b-chat-int4-onnx-directml

EmbeddedLLM llama-2-7b-chat-int4-onnx-directml online free

llama-2-7b-chat-int4-onnx-directml huggingface.co is an online trial and call api platform, which integrates llama-2-7b-chat-int4-onnx-directml's modeling effects, including api services, and provides a free online trial of llama-2-7b-chat-int4-onnx-directml, you can try llama-2-7b-chat-int4-onnx-directml online for free by clicking the link below.

EmbeddedLLM llama-2-7b-chat-int4-onnx-directml online free url in huggingface.co:

https://huggingface.co/EmbeddedLLM/llama-2-7b-chat-int4-onnx-directml

llama-2-7b-chat-int4-onnx-directml install

llama-2-7b-chat-int4-onnx-directml is an open source model from GitHub that offers a free installation service, and any user can find llama-2-7b-chat-int4-onnx-directml on GitHub to install. At the same time, huggingface.co provides the effect of llama-2-7b-chat-int4-onnx-directml install, users can directly use llama-2-7b-chat-int4-onnx-directml installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

llama-2-7b-chat-int4-onnx-directml install url in huggingface.co:

https://huggingface.co/EmbeddedLLM/llama-2-7b-chat-int4-onnx-directml

Url of llama-2-7b-chat-int4-onnx-directml

llama-2-7b-chat-int4-onnx-directml huggingface.co Url

Provider of llama-2-7b-chat-int4-onnx-directml huggingface.co

EmbeddedLLM
ORGANIZATIONS

Other API from EmbeddedLLM