Update README.md
Browse files
README.md
CHANGED
|
@@ -18,14 +18,7 @@ FireFunction is a state-of-the-art function calling model with a commercially vi
|
|
| 18 |
|
| 19 |
💡 Hosted on the [Fireworks](https://fireworks.ai/models/fireworks/firefunction-v2) platform
|
| 20 |
|
| 21 |
-
##
|
| 22 |
-
* [Fireworks discord with function calling channel](https://discord.gg/mMqQxvFD9A)
|
| 23 |
-
* [Documentation](https://readme.fireworks.ai/docs/function-calling)
|
| 24 |
-
* [UI Demo app](https://functional-chat.vercel.app/)
|
| 25 |
-
* [Try in Fireworks prompt playground UI](https://fireworks.ai/models/fireworks/firefunction-v2)
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
# Intended Use and Limitations
|
| 29 |
|
| 30 |
### Supported usecases
|
| 31 |
The model was tuned to perfom well on a range of usecases including:
|
|
@@ -40,6 +33,18 @@ The model was not optimized for the following use cases:
|
|
| 40 |
* 100+ function specs
|
| 41 |
* nested function calling
|
| 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
## Example Usage
|
| 44 |
|
| 45 |
See [documentation](https://readme.fireworks.ai/docs/function-calling) for more detail.
|
|
@@ -110,6 +115,8 @@ decoded = tokenizer.batch_decode(generated_ids)
|
|
| 110 |
print(decoded[0])
|
| 111 |
```
|
| 112 |
|
| 113 |
-
##
|
| 114 |
-
|
| 115 |
-
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
💡 Hosted on the [Fireworks](https://fireworks.ai/models/fireworks/firefunction-v2) platform
|
| 20 |
|
| 21 |
+
## Intended Use and Limitations
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
### Supported usecases
|
| 24 |
The model was tuned to perfom well on a range of usecases including:
|
|
|
|
| 33 |
* 100+ function specs
|
| 34 |
* nested function calling
|
| 35 |
|
| 36 |
+
## Metrics
|
| 37 |
+
|
| 38 |
+
| | Firefunction v1 | Firefunction v2 | Llama 3 70b Instruct | Gpt-4o |
|
| 39 |
+
|------------------------------------|-----------------|-----------------|----------------------|--------|
|
| 40 |
+
| Gorilla simple | 0.91 | 0.94 | 0.925 | 0.88 |
|
| 41 |
+
| Gorilla multiple_function | 0.92 | 0.91 | 0.86 | 0.91 |
|
| 42 |
+
| Gorilla parallel_function | 0 | 0.9 | 0.86 | 0.89 |
|
| 43 |
+
| Gorilla parallel_multiple_function | 0 | 0.8 | 0.615 | 0.72 |
|
| 44 |
+
| Nexus parallel | 0.38 | 0.53 | 0.3 | 0.47 |
|
| 45 |
+
| Mtbench | 0.73 | 0.84 | 0.89 | 0.93 |
|
| 46 |
+
| Average | 0.49 | 0.82 | 0.74 | 0.8 |
|
| 47 |
+
|
| 48 |
## Example Usage
|
| 49 |
|
| 50 |
See [documentation](https://readme.fireworks.ai/docs/function-calling) for more detail.
|
|
|
|
| 115 |
print(decoded[0])
|
| 116 |
```
|
| 117 |
|
| 118 |
+
## Resources
|
| 119 |
+
* [Fireworks discord with function calling channel](https://discord.gg/mMqQxvFD9A)
|
| 120 |
+
* [Documentation](https://readme.fireworks.ai/docs/function-calling)
|
| 121 |
+
* [Demo app](https://functional-chat.vercel.app/)
|
| 122 |
+
* [Try in Fireworks prompt playground UI](https://fireworks.ai/models/fireworks/firefunction-v2)
|