model=huggingface/falcon-40b-gptq
num_shard=2
volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run
docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:0.8 --model-id $model --num-shard $num_shard --quantize gptq
Runs of huggingface falcon-40b-gptq on huggingface.co
77
Total runs
0
24-hour runs
5
3-day runs
13
7-day runs
55
30-day runs
More Information About falcon-40b-gptq huggingface.co Model
falcon-40b-gptq huggingface.co
falcon-40b-gptq huggingface.co is an AI model on huggingface.co that provides falcon-40b-gptq's model effect (), which can be used instantly with this huggingface falcon-40b-gptq model. huggingface.co supports a free trial of the falcon-40b-gptq model, and also provides paid use of the falcon-40b-gptq. Support call falcon-40b-gptq model through api, including Node.js, Python, http.
falcon-40b-gptq huggingface.co is an online trial and call api platform, which integrates falcon-40b-gptq's modeling effects, including api services, and provides a free online trial of falcon-40b-gptq, you can try falcon-40b-gptq online for free by clicking the link below.
huggingface falcon-40b-gptq online free url in huggingface.co:
falcon-40b-gptq is an open source model from GitHub that offers a free installation service, and any user can find falcon-40b-gptq on GitHub to install. At the same time, huggingface.co provides the effect of falcon-40b-gptq install, users can directly use falcon-40b-gptq installed effect in huggingface.co for debugging and trial. It also supports api for free installation.