This is a GGUF conversion of
Google's UMT5 xxl model
, specifically the encoder part.
The weights can be used with
./llama-embedding
or with the
ComfyUI-GGUF
custom node together with image/video generation models.
This is a
non imatrix
quant as llama.cpp doesn't support imatrix creation for T5 models at the time of writing. It's therefore recommended to use
Q5_K_M or larger
for the best results, although smaller models may also still provide decent results in resource constrained scenarios.