Total runs:
30.4K
Run Growth:
30.4K
Growth Rate:
100.00%
Sponsored by
APIMart
NexaAI / octo-net-gguf
Model's Last Updated: May 24 2024
Introduction of octo-net-gguf
Model Details of octo-net-gguf
Octopus V4-GGUF: Graph of language models
- Original Model - Nexa AI Website - Octopus-v4 Github - ArXiv - Domain LLM Leaderbaord
Acknowledgement
:
We sincerely thank our community members,
Mingyuan
and
Zoey
, for their extraordinary contributions to this quantization effort. Please explore
Octopus-v4
for our original huggingface model.
Get Started
To run the models, please download them to your local machine using either git clone or Hugging Face Hub
git clone https://huggingface.co/NexaAIDev/octopus-v4-gguf
Run with llama.cpp (Recommended)
- Clone and compile:
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
# Compile the source code:
make
- Execute the Model:
Run the following command in the terminal:
./main -m ./path/to/octopus-v4-Q4_K_M.gguf -n 256 -p "<|system|>You are a router. Below is the query from the users, please call the correct function and generate the parameters to call the function.<|end|><|user|>Tell me the result of derivative of x^3 when x is 2?<|end|><|assistant|>"
Run with Ollama
Since our models have not been uploaded to the Ollama server, please download the models and manually import them into Ollama by following these steps:
- Install Ollama on your local machine. You can also following the guide from Ollama GitHub repository
git clone https://github.com/ollama/ollama.git ollama
- Locate the local Ollama directory:
cd ollama
-
Create a
Modelfilein your directory
touch Modelfile
-
In the Modelfile, include a
FROMstatement with the path to your local model, and the default parameters:
FROM ./path/to/octopus-v4-Q4_K_M.gguf
PARAMETER temperature 0
PARAMETER num_ctx 1024
PARAMETER stop <nexa_end>
- Use the following command to add the model to Ollama:
ollama create octopus-v4-Q4_K_M -f Modelfile
- Verify that the model has been successfully imported:
ollama ls
- Run the model
ollama run octopus-v4-Q4_K_M "<|system|>You are a router. Below is the query from the users, please call the correct function and generate the parameters to call the function.<|end|><|user|>Tell me the result of derivative of x^3 when x is 2?<|end|><|assistant|>"
Dataset and Benchmark
- Utilized questions from MMLU to evaluate the performances.
- Evaluated with the Ollama llm-benchmark method.
Quantized GGUF Models
| Name | Quant method | Bits | Size | Respons (token/second) | Use Cases |
|---|---|---|---|---|---|
| Octopus-v4.gguf | 7.64 GB | 27.64 | extremely large | ||
| Octopus-v4-Q2_K.gguf | Q2_K | 2 | 1.42 GB | 54.20 | extremely not recommended, high loss |
| Octopus-v4-Q3_K.gguf | Q3_K | 3 | 1.96 GB | 51.22 | not recommended |
| Octopus-v4-Q3_K_S.gguf | Q3_K_S | 3 | 1.68 GB | 51.78 | not very recommended |
| Octopus-v4-Q3_K_M.gguf | Q3_K_M | 3 | 1.96 GB | 50.86 | not very recommended |
| Octopus-v4-Q3_K_L.gguf | Q3_K_L | 3 | 2.09 GB | 50.05 | not very recommended |
| Octopus-v4-Q4_0.gguf | Q4_0 | 4 | 2.18 GB | 65.76 | good quality, recommended |
| Octopus-v4-Q4_1.gguf | Q4_1 | 4 | 2.41 GB | 69.01 | slow, good quality, recommended |
| Octopus-v4-Q4_K.gguf | Q4_K | 4 | 2.39 GB | 55.76 | slow, good quality, recommended |
| Octopus-v4-Q4_K_S.gguf | Q4_K_S | 4 | 2.19 GB | 53.98 | high quality, recommended |
| Octopus-v4-Q4_K_M.gguf | Q4_K_M | 4 | 2.39 GB | 58.39 | some functions loss, not very recommended |
| Octopus-v4-Q5_0.gguf | Q5_0 | 5 | 2.64 GB | 61.98 | slow, good quality |
| Octopus-v4-Q5_1.gguf | Q5_1 | 5 | 2.87 GB | 63.44 | slow, good quality |
| Octopus-v4-Q5_K.gguf | Q5_K | 5 | 2.82 GB | 58.28 | moderate speed, recommended |
| Octopus-v4-Q5_K_S.gguf | Q5_K_S | 5 | 2.64 GB | 59.95 | moderate speed, recommended |
| Octopus-v4-Q5_K_M.gguf | Q5_K_M | 5 | 2.82 GB | 53.31 | fast, good quality, recommended |
| Octopus-v4-Q6_K.gguf | Q6_K | 6 | 3.14 GB | 52.15 | large, not very recommended |
| Octopus-v4-Q8_0.gguf | Q8_0 | 8 | 4.06 GB | 50.10 | very large, good quality |
| Octopus-v4-f16.gguf | f16 | 16 | 7.64 GB | 30.61 | extremely large |
Quantized with llama.cpp
Runs of NexaAI octo-net-gguf on huggingface.co
554
Total runs
0
24-hour runs
27
3-day runs
77
7-day runs
77
30-day runs
More Information About octo-net-gguf huggingface.co Model
Url of octo-net-gguf
octo-net-gguf huggingface.co Url
Provider of octo-net-gguf huggingface.co
Other API from NexaAI
Updated:October 28 2025
Total runs:
4.4K
Run Growth:
2.3K
Growth Rate:
52.99%
Total runs:
3.9K
Run Growth:
731
Growth Rate:
18.87%
Total runs:
2.4K
Run Growth:
-4.4K
Growth Rate:
-178.72%
Total runs:
1.4K
Run Growth:
37
Growth Rate:
2.66%
Total runs:
1.0K
Run Growth:
-71
Growth Rate:
-6.81%
Total runs:
905
Run Growth:
142
Growth Rate:
15.69%
Total runs:
875
Run Growth:
-221
Growth Rate:
-25.26%
Total runs:
758
Run Growth:
-660
Growth Rate:
-87.07%
Total runs:
523
Run Growth:
-559
Growth Rate:
-106.88%
Total runs:
502
Run Growth:
4
Growth Rate:
0.80%
Total runs:
438
Run Growth:
256
Growth Rate:
58.45%
Total runs:
353
Run Growth:
-895
Growth Rate:
-253.54%
Total runs:
301
Run Growth:
31
Growth Rate:
10.30%
Total runs:
292
Run Growth:
-13
Growth Rate:
-4.45%
Total runs:
148
Run Growth:
63
Growth Rate:
42.57%
Total runs:
145
Run Growth:
6
Growth Rate:
4.14%
Total runs:
126
Run Growth:
34
Growth Rate:
26.98%
Total runs:
92
Run Growth:
-44
Growth Rate:
-47.83%
Total runs:
79
Run Growth:
-38
Growth Rate:
-48.10%
Total runs:
63
Run Growth:
-45
Growth Rate:
-71.43%
Total runs:
57
Run Growth:
-179
Growth Rate:
-314.04%
Total runs:
57
Run Growth:
6
Growth Rate:
10.53%
Total runs:
51
Run Growth:
0
Growth Rate:
0.00%
Total runs:
51
Run Growth:
7
Growth Rate:
13.73%
Total runs:
45
Run Growth:
-336
Growth Rate:
-746.67%
Total runs:
44
Run Growth:
3
Growth Rate:
6.82%
Total runs:
34
Run Growth:
-6
Growth Rate:
-17.65%
Total runs:
34
Run Growth:
-17
Growth Rate:
-50.00%
Total runs:
32
Run Growth:
-40
Growth Rate:
-125.00%
Total runs:
30
Run Growth:
-3
Growth Rate:
-10.00%
Total runs:
29
Run Growth:
0
Growth Rate:
0.00%
Total runs:
28
Run Growth:
-15
Growth Rate:
-53.57%
Total runs:
26
Run Growth:
-2
Growth Rate:
-7.69%
Total runs:
25
Run Growth:
-39
Growth Rate:
-156.00%
Total runs:
24
Run Growth:
-220
Growth Rate:
-916.67%
Total runs:
24
Run Growth:
-6
Growth Rate:
-25.00%
Total runs:
23
Run Growth:
1
Growth Rate:
4.35%
Total runs:
21
Run Growth:
9
Growth Rate:
42.86%
Total runs:
21
Run Growth:
2
Growth Rate:
9.52%
Total runs:
18
Run Growth:
14
Growth Rate:
77.78%
Total runs:
17
Run Growth:
13
Growth Rate:
76.47%
Total runs:
15
Run Growth:
3
Growth Rate:
20.00%
Total runs:
14
Run Growth:
1
Growth Rate:
7.14%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
-1
Growth Rate:
-8.33%
Total runs:
12
Run Growth:
4
Growth Rate:
33.33%
Total runs:
11
Run Growth:
-60
Growth Rate:
-545.45%
Total runs:
11
Run Growth:
-12
Growth Rate:
-109.09%
Total runs:
11
Run Growth:
-23
Growth Rate:
-209.09%
Total runs:
11
Run Growth:
1
Growth Rate:
9.09%
Total runs:
10
Run Growth:
4
Growth Rate:
40.00%
Total runs:
10
Run Growth:
3
Growth Rate:
30.00%
Total runs:
9
Run Growth:
-3
Growth Rate:
-33.33%
Total runs:
8
Run Growth:
2
Growth Rate:
25.00%
Total runs:
8
Run Growth:
0
Growth Rate:
0.00%
Total runs:
7
Run Growth:
4
Growth Rate:
57.14%
Total runs:
5
Run Growth:
-11
Growth Rate:
-220.00%
Total runs:
5
Run Growth:
0
Growth Rate:
0.00%
Total runs:
5
Run Growth:
-2
Growth Rate:
-40.00%
Total runs:
4
Run Growth:
-17
Growth Rate:
-425.00%