Sponsored by ThumbnailCreator.com
Toolify Toolify logo
AI Models
CausalLM_35b-beta-long-GGUF-imatrix

NeoChen1024 / CausalLM_35b-beta-long-GGUF-imatrix

Run on huggingface.co
huggingface.co
Total runs: 158
24-hour runs: -6
7-day runs: -90
30-day runs: -5
Model's Last Updated: September 19 2024

Introduction of CausalLM_35b-beta-long-GGUF-imatrix

Model Details of CausalLM_35b-beta-long-GGUF-imatrix

GGUF quants of CausalLM/35b-beta-long, here I have: 

IQ1_S  (8.0G, 16.8624 +/- 0.24892, fits into 10GiB VRAM, just for kicks and giggles, not really usable)
IQ1_M  (8.6G, 13.9588 +/- 0.19871, fits into 12GiB VRAM, just for kicks and giggles, not really usable)
IQ2_M  ( 12G, 10.1401 +/- 0.14062, fits into 16GiB VRAM + 6144 context with q4_1 KV cache)
IQ4_XS ( 18G, 9.4489 +/- 0.13005, fits into 24GiB VRAM + 8192 context with q4_1 KV cache, also room for 2048 ubatch)
IQ4_NL ( 19G, 9.4632 +/- 0.13056, fits into 24GiB VRAM + 8192 context with q4_1 KV cache)
Q4_K_M ( 21G, 9.3738 +/- 0.12900, fits into 24GiB VRAM + 6144 context with q4_1 KV cache, also good for CPU inference on E5-26xx v3/v4)
Q8_0   ( 35G, 9.3277 +/- 0.12781, probably isn't practical for anything unless you have big GPU array, imatrix derived from it)

Perplexity measured with -fa -ctv q4_1 -ctk q4_1 -c 2048 -ub 2048 on UTF-8 text version of "Wired Love" from Project Gutenberg .

Runs of NeoChen1024 CausalLM_35b-beta-long-GGUF-imatrix on huggingface.co

158
Total runs
-6
24-hour runs
-84
3-day runs
-90
7-day runs
-5
30-day runs

More Information About CausalLM_35b-beta-long-GGUF-imatrix huggingface.co Model

CausalLM_35b-beta-long-GGUF-imatrix huggingface.co

CausalLM_35b-beta-long-GGUF-imatrix huggingface.co is an AI model on huggingface.co that provides CausalLM_35b-beta-long-GGUF-imatrix's model effect (), which can be used instantly with this NeoChen1024 CausalLM_35b-beta-long-GGUF-imatrix model. huggingface.co supports a free trial of the CausalLM_35b-beta-long-GGUF-imatrix model, and also provides paid use of the CausalLM_35b-beta-long-GGUF-imatrix. Support call CausalLM_35b-beta-long-GGUF-imatrix model through api, including Node.js, Python, http.

CausalLM_35b-beta-long-GGUF-imatrix huggingface.co Url

https://huggingface.co/NeoChen1024/CausalLM_35b-beta-long-GGUF-imatrix

NeoChen1024 CausalLM_35b-beta-long-GGUF-imatrix online free

CausalLM_35b-beta-long-GGUF-imatrix huggingface.co is an online trial and call api platform, which integrates CausalLM_35b-beta-long-GGUF-imatrix's modeling effects, including api services, and provides a free online trial of CausalLM_35b-beta-long-GGUF-imatrix, you can try CausalLM_35b-beta-long-GGUF-imatrix online for free by clicking the link below.

NeoChen1024 CausalLM_35b-beta-long-GGUF-imatrix online free url in huggingface.co:

https://huggingface.co/NeoChen1024/CausalLM_35b-beta-long-GGUF-imatrix

CausalLM_35b-beta-long-GGUF-imatrix install

CausalLM_35b-beta-long-GGUF-imatrix is an open source model from GitHub that offers a free installation service, and any user can find CausalLM_35b-beta-long-GGUF-imatrix on GitHub to install. At the same time, huggingface.co provides the effect of CausalLM_35b-beta-long-GGUF-imatrix install, users can directly use CausalLM_35b-beta-long-GGUF-imatrix installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

CausalLM_35b-beta-long-GGUF-imatrix install url in huggingface.co:

https://huggingface.co/NeoChen1024/CausalLM_35b-beta-long-GGUF-imatrix

Url of CausalLM_35b-beta-long-GGUF-imatrix

CausalLM_35b-beta-long-GGUF-imatrix huggingface.co Url

Provider of CausalLM_35b-beta-long-GGUF-imatrix huggingface.co

NeoChen1024
ORGANIZATIONS
huggingface.co

Other API from NeoChen1024