Sponsored by
APIMart
ibivibiv / athena-120b
Model's Last Updated: January 05 2024
text-generation
Introduction of athena-120b
Model Details of athena-120b
Athena 120b
An auto-regressive causal LM created by combining 3x finetuned models into one via passthrough merging slices in a stacked order. This is my first early prototype attempt at expanding on existing work and simply attempts to take the creative RP focused model of Venus-120 and switch out the Euryale-1.3-L2-70B for WinterGoddess-1.4x-70B-L2 in an attempt to inherit more task focus and instruct as is mentioned in WinterGoddess. My goals are much more work and task focused so I wanted a model that was creative but less focused on RP and more focused on writing and documentation tasks.
Prompting Format
Both Vicuna and Alpaca will work, but due the final layers belonging primarily to Xwin.
Benchmarks
Coming soon.
Acknowledgements
@chargoddard - mergekit . @migtissera - for Tess-XL which inspired me to believe that open models can compete on logic tasks with the big commercial models. @alpindale - for Goliath-120B that started this crazy endeavor for us all @nsfwthrowitaway69 - for sharing the merge config for Venus-120B and getting me off the starting block with some questions on mergekit and tokenizers
Keep it open and keep sharing everyone! With Mixtral and MOE changes to mergekit coupled with these larger merged models? I think the sky is the limit for us all. I can only imagine what will happen if we took a group of these 120 models, fin tuned them each a bit and applied the MOE Mixtral merge method to them? I would also point out that if a clever VC came along and funded that work? You have the people you need right here on huggingface and all they need is the equipment to do it on.
Runs of ibivibiv athena-120b on huggingface.co
27
Total runs
0
24-hour runs
-2
3-day runs
-3
7-day runs
-2
30-day runs
More Information About athena-120b huggingface.co Model
Url of athena-120b
athena-120b huggingface.co Url
Provider of athena-120b huggingface.co
Other API from ibivibiv
Total runs:
8.6K
Run Growth:
33
Growth Rate:
0.39%
Total runs:
85
Run Growth:
2
Growth Rate:
2.25%
Total runs:
83
Run Growth:
26
Growth Rate:
30.59%
Total runs:
73
Run Growth:
-16
Growth Rate:
-21.33%
Total runs:
72
Run Growth:
-84
Growth Rate:
-115.07%
Total runs:
70
Run Growth:
-12
Growth Rate:
-16.90%
Total runs:
66
Run Growth:
-3
Growth Rate:
-4.41%
Total runs:
63
Run Growth:
-12
Growth Rate:
-18.46%
Total runs:
57
Run Growth:
-27
Growth Rate:
-50.00%
Total runs:
42
Run Growth:
11
Growth Rate:
26.19%
Total runs:
30
Run Growth:
6
Growth Rate:
20.00%
Total runs:
23
Run Growth:
2
Growth Rate:
9.09%
Total runs:
22
Run Growth:
-4
Growth Rate:
-18.18%
Total runs:
22
Run Growth:
1
Growth Rate:
4.76%
Total runs:
20
Run Growth:
3
Growth Rate:
15.00%
Total runs:
15
Run Growth:
-5
Growth Rate:
-33.33%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
-4
Growth Rate:
-28.57%
Total runs:
13
Run Growth:
-6
Growth Rate:
-46.15%
Total runs:
13
Run Growth:
-4
Growth Rate:
-30.77%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
13
Run Growth:
-13
Growth Rate:
-92.86%
Total runs:
12
Run Growth:
-13
Growth Rate:
-108.33%
Total runs:
12
Run Growth:
-4
Growth Rate:
-33.33%
Total runs:
11
Run Growth:
-10
Growth Rate:
-90.91%
Total runs:
11
Run Growth:
-11
Growth Rate:
-100.00%
Total runs:
10
Run Growth:
-10
Growth Rate:
-100.00%
Total runs:
9
Run Growth:
-3
Growth Rate:
-33.33%
Total runs:
9
Run Growth:
-8
Growth Rate:
-88.89%
Total runs:
9
Run Growth:
-3
Growth Rate:
-37.50%
Total runs:
9
Run Growth:
4
Growth Rate:
44.44%
Total runs:
9
Run Growth:
-2
Growth Rate:
-22.22%
Total runs:
8
Run Growth:
1
Growth Rate:
11.11%
Total runs:
8
Run Growth:
2
Growth Rate:
22.22%