Total runs:
2.8K
Run Growth:
193
Growth Rate:
6.91%
Sponsored by
iMean AI Trip Planner
mav23 / llama-3-firefunction-v2-GGUF
Model's Last Updated: November 12 2024
Introduction of llama-3-firefunction-v2-GGUF
Model Details of llama-3-firefunction-v2-GGUF
FireFunction V2: Fireworks Function Calling Model
Try on Fireworks | API Docs | Demo App | Discord
FireFunction is a state-of-the-art function calling model with a commercially viable license. View detailed info in our announcement blog . Key info and highlights:
Comparison with other models:
- Competitive with GPT-4o at function-calling, scoring 0.81 vs 0.80 on a medley of public evaluations
- Trained on Llama 3 and retains Llama 3’s conversation and instruction-following capabilities, scoring 0.84 vs Llama 3’s 0.89 on MT bench
- Significant quality improvements over FireFunction v1 across the broad range of metrics
General info:
🐾 Successor of the FireFunction model
🔆 Support of parallel function calling (unlike FireFunction v1) and good instruction following
💡 Hosted on the Fireworks platform at < 10% of the cost of GPT 4o and 2x the speed
Intended Use and Limitations
Supported usecases
The model was tuned to perfom well on a range of usecases including:
- general instruction following
- multi-turn chat mixing vanilla messages with function calls
- single- and parallel function calling
- up to 20 function specs supported at once
- structured information extraction
The model has an 8k context window, like Llama 3
Out-of-Scope Use
The model was not optimized for the following use cases:
- 100+ function specs
- nested function calling
Metrics
| Benchmark | Firefunction v1 | Firefunction v2 | Llama 3 70b Instruct | Gpt-4o |
|---|---|---|---|---|
| Gorilla simple | 0.91 | 0.94 | 0.925 | 0.88 |
| Gorilla multiple_function | 0.92 | 0.91 | 0.86 | 0.91 |
| Gorilla parallel_function | 0 | 0.9 | 0.86 | 0.89 |
| Gorilla parallel_multiple_function | 0 | 0.8 | 0.615 | 0.72 |
| Nexus parallel | 0.38 | 0.53 | 0.3 | 0.47 |
| Mtbench | 0.73 | 0.84 | 0.89 | 0.93 |
| Average | 0.49 | 0.82 | 0.74 | 0.8 |
Example Usage
See documentation for more detail.
from transformers import AutoModelForCausalLM, AutoTokenizer
import json
from datetime import datetime
device = "cuda" # the device to load the model onto
model = AutoModelForCausalLM.from_pretrained("fireworks-ai/firefunction-v2", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("fireworks-ai/firefunction-v2")
function_spec = [
{
"name": "get_stock_price",
"description": "Get the current stock price",
"parameters": {
"type": "object",
"properties": {
"symbol": {
"type": "string",
"description": "The stock symbol, e.g. AAPL, GOOG"
}
},
"required": [
"symbol"
]
}
},
{
"name": "check_word_anagram",
"description": "Check if two words are anagrams of each other",
"parameters": {
"type": "object",
"properties": {
"word1": {
"type": "string",
"description": "The first word"
},
"word2": {
"type": "string",
"description": "The second word"
}
},
"required": [
"word1",
"word2"
]
}
}
]
functions = json.dumps(function_spec, indent=4)
messages = [
{'role': 'system', 'content': 'You are a helpful assistant with access to functions. Use them if required.'},
{'role': 'user', 'content': 'Hi, can you tell me the current stock price of google and netflix?'}
]
now = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
model_inputs = tokenizer.apply_chat_template(messages, functions=functions, datetime=now, return_tensors="pt").to(model.device)
generated_ids = model.generate(model_inputs, max_new_tokens=128)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])
Resources
Runs of mav23 llama-3-firefunction-v2-GGUF on huggingface.co
126
Total runs
0
24-hour runs
-2
3-day runs
-31
7-day runs
79
30-day runs
More Information About llama-3-firefunction-v2-GGUF huggingface.co Model
Url of llama-3-firefunction-v2-GGUF
llama-3-firefunction-v2-GGUF huggingface.co Url
Provider of llama-3-firefunction-v2-GGUF huggingface.co
Other API from mav23
Updated:October 11 2024
Total runs:
2.2K
Run Growth:
1.7K
Growth Rate:
75.47%
Total runs:
1.6K
Run Growth:
1.4K
Growth Rate:
89.95%
Total runs:
1.4K
Run Growth:
1.4K
Growth Rate:
98.29%
Total runs:
1.0K
Run Growth:
858
Growth Rate:
82.03%
Total runs:
959
Run Growth:
17
Growth Rate:
1.77%
Total runs:
887
Run Growth:
595
Growth Rate:
67.08%
Total runs:
677
Run Growth:
484
Growth Rate:
71.49%
Total runs:
625
Run Growth:
538
Growth Rate:
86.08%
Total runs:
463
Run Growth:
441
Growth Rate:
95.25%
Total runs:
410
Run Growth:
173
Growth Rate:
42.20%
Total runs:
367
Run Growth:
99
Growth Rate:
26.98%
Total runs:
332
Run Growth:
157
Growth Rate:
47.29%
Total runs:
322
Run Growth:
-122
Growth Rate:
-37.89%
Total runs:
279
Run Growth:
-1.5K
Growth Rate:
-552.69%
Total runs:
274
Run Growth:
147
Growth Rate:
53.65%
Total runs:
266
Run Growth:
200
Growth Rate:
75.19%
Total runs:
260
Run Growth:
-13
Growth Rate:
-5.00%
Total runs:
254
Run Growth:
1
Growth Rate:
0.39%
Total runs:
253
Run Growth:
138
Growth Rate:
54.55%
Total runs:
252
Run Growth:
-22
Growth Rate:
-8.73%
Total runs:
247
Run Growth:
27
Growth Rate:
10.93%
Total runs:
246
Run Growth:
172
Growth Rate:
69.92%
Total runs:
243
Run Growth:
213
Growth Rate:
87.65%
Total runs:
236
Run Growth:
-334
Growth Rate:
-141.53%
Total runs:
234
Run Growth:
186
Growth Rate:
79.49%
Total runs:
233
Run Growth:
128
Growth Rate:
54.94%
Total runs:
222
Run Growth:
97
Growth Rate:
43.69%
Total runs:
203
Run Growth:
-67
Growth Rate:
-33.00%
Total runs:
199
Run Growth:
93
Growth Rate:
46.73%
Total runs:
195
Run Growth:
125
Growth Rate:
64.10%
Total runs:
193
Run Growth:
-47
Growth Rate:
-24.35%
Total runs:
193
Run Growth:
-5
Growth Rate:
-2.59%
Total runs:
190
Run Growth:
153
Growth Rate:
80.53%
Total runs:
189
Run Growth:
20
Growth Rate:
10.58%
Total runs:
188
Run Growth:
-1
Growth Rate:
-0.53%
Total runs:
180
Run Growth:
74
Growth Rate:
41.11%
Total runs:
177
Run Growth:
14
Growth Rate:
7.91%
Total runs:
177
Run Growth:
79
Growth Rate:
44.63%
Total runs:
176
Run Growth:
150
Growth Rate:
85.23%
Total runs:
173
Run Growth:
166
Growth Rate:
95.95%
Total runs:
170
Run Growth:
52
Growth Rate:
30.59%
Total runs:
166
Run Growth:
43
Growth Rate:
25.90%
Total runs:
162
Run Growth:
19
Growth Rate:
11.73%
Total runs:
160
Run Growth:
-214
Growth Rate:
-133.75%
Total runs:
157
Run Growth:
43
Growth Rate:
27.39%
Total runs:
156
Run Growth:
19
Growth Rate:
12.18%
Total runs:
152
Run Growth:
-251
Growth Rate:
-165.13%
Total runs:
152
Run Growth:
-2
Growth Rate:
-1.32%
Total runs:
151
Run Growth:
35
Growth Rate:
23.18%
Total runs:
148
Run Growth:
-91
Growth Rate:
-61.49%
Total runs:
146
Run Growth:
105
Growth Rate:
71.92%
Total runs:
139
Run Growth:
105
Growth Rate:
75.54%
Total runs:
138
Run Growth:
-12
Growth Rate:
-8.70%
Total runs:
138
Run Growth:
103
Growth Rate:
74.64%
Total runs:
135
Run Growth:
89
Growth Rate:
65.93%
Total runs:
134
Run Growth:
13
Growth Rate:
9.70%
Total runs:
131
Run Growth:
-133
Growth Rate:
-101.53%
Total runs:
131
Run Growth:
13
Growth Rate:
9.92%
Total runs:
128
Run Growth:
-124
Growth Rate:
-96.88%
Total runs:
127
Run Growth:
22
Growth Rate:
17.32%
Total runs:
127
Run Growth:
6
Growth Rate:
4.72%
Total runs:
126
Run Growth:
38
Growth Rate:
30.16%
Total runs:
126
Run Growth:
26
Growth Rate:
20.63%
Total runs:
122
Run Growth:
47
Growth Rate:
38.52%
Total runs:
121
Run Growth:
-1.1K
Growth Rate:
-909.92%
Total runs:
121
Run Growth:
88
Growth Rate:
72.73%
Total runs:
117
Run Growth:
58
Growth Rate:
49.57%
Total runs:
114
Run Growth:
-3.5K
Growth Rate:
-3107.02%
Total runs:
113
Run Growth:
0
Growth Rate:
0.00%
Total runs:
111
Run Growth:
103
Growth Rate:
92.79%
Total runs:
111
Run Growth:
39
Growth Rate:
35.14%
Total runs:
110
Run Growth:
-112
Growth Rate:
-101.82%
Total runs:
110
Run Growth:
-6
Growth Rate:
-5.45%
Total runs:
110
Run Growth:
-397
Growth Rate:
-360.91%
Total runs:
110
Run Growth:
-88
Growth Rate:
-80.00%
Total runs:
110
Run Growth:
68
Growth Rate:
61.82%
Total runs:
109
Run Growth:
-20
Growth Rate:
-18.35%
Total runs:
109
Run Growth:
88
Growth Rate:
80.73%
Total runs:
108
Run Growth:
-15
Growth Rate:
-13.89%
Total runs:
107
Run Growth:
3
Growth Rate:
2.80%
Total runs:
107
Run Growth:
-109
Growth Rate:
-101.87%