Total runs:
18.1K
Run Growth:
12.4K
Growth Rate:
48.39%
Sponsored by
AdsCreator.com
CompendiumLabs / bge-small-en-v1.5-gguf
Model's Last Updated: February 18 2024
Introduction of bge-small-en-v1.5-gguf
Model Details of bge-small-en-v1.5-gguf
bge-small-en-v1.5-gguf
Source model: https://huggingface.co/BAAI/bge-small-en-v1.5
Quantized and unquantized embedding models in GGUF format for use with
llama.cpp
. A large benefit over
transformers
is almost guaranteed and the benefit over ONNX will vary based on the application, but this seems to provide a large speedup on CPU and a modest speedup on GPU for larger models. Due to the relatively small size of these models, quantization will not provide huge benefits, but it does generate up to a 30% speedup on CPU with minimal loss in accuracy.
Files Available
| Filename | Quantization | Size |
|---|---|---|
| bge-small-en-v1.5-f32.gguf | F32 | 128 MB |
| bge-small-en-v1.5-f16.gguf | F16 | 65 MB |
| bge-small-en-v1.5-q8_0.gguf | Q8_0 | 36 MB |
| bge-small-en-v1.5-q4_k_m.gguf | Q4_K_M | 24 MB |
Usage
These model files can be used with pure
llama.cpp
or with the
llama-cpp-python
Python bindings
from llama_cpp import Llama
model = Llama(gguf_path, embedding=True)
embed = model.embed(texts)
Here
texts
can either be a string or a list of strings, and the return value is a list of embedding vectors. The inputs are grouped into batches automatically for efficient execution. There is also LangChain integration through
langchain_community.embeddings.LlamaCppEmbeddings
.
Runs of CompendiumLabs bge-small-en-v1.5-gguf on huggingface.co
63.5K
Total runs
164
24-hour runs
492
3-day runs
1.4K
7-day runs
54.9K
30-day runs
More Information About bge-small-en-v1.5-gguf huggingface.co Model
Url of bge-small-en-v1.5-gguf
bge-small-en-v1.5-gguf huggingface.co Url
Provider of bge-small-en-v1.5-gguf huggingface.co
Other API from CompendiumLabs
Updated:February 18 2024
Total runs:
3.1K
Run Growth:
1.5K
Growth Rate:
49.33%
Total runs:
1.0K
Run Growth:
-250
Growth Rate:
-23.58%
Total runs:
662
Run Growth:
-303
Growth Rate:
-46.33%
Total runs:
443
Run Growth:
101
Growth Rate:
23.49%
Total runs:
92
Run Growth:
-17
Growth Rate:
-18.48%
Total runs:
14
Run Growth:
4
Growth Rate:
28.57%