This is an imatrix gguf conversion of
xtuner/llava-llama-3-8b-v1_1-transformers
.
Mainly intended to be used as the text encoder for Hunyuan Video, but possible to use for vision tasks with the
mmproj
file from the xtuner gguf repository.
The imatrix dataset used was
calibration_datav3.txt
by
Bartowski
, which was used for all quants under Q6_K. Tested against wikitext / no imatrix and it outperformed both.
Note that the
vocab_size
is different between the
transformers
(128 320) and the
hf
(128 256) repositories. This used the former as it was what was used in the official Hunyuan Video code.
IQ quants will be slow in ComfyUI due to using numpy fallback.