Sponsored by
APIMart
jamesdborin / LeoLM-hessianai-7b
Model's Last Updated: October 16 2023
text-generation
Introduction of LeoLM-hessianai-7b
Model Details of LeoLM-hessianai-7b
LAION LeoLM: L inguistically E nhanced O pen L anguage M odel
Meet LeoLM, the first open and commercially available German Foundation Language Model built on Llama-2.
Our models extend Llama-2's capabilities into German through continued pretraining on a large corpus of German-language and mostly locality specific text.
Thanks to a compute grant at HessianAI's new supercomputer
42
, we release two foundation models trained with 8k context length,
LeoLM/leo-hessianai-7b
and
LeoLM/leo-hessianai-13b
under the
Llama-2 community license
(70b also coming soon! 👀).
With this release, we hope to bring a new wave of opportunities to German open-source and commercial LLM research and accelerate adoption.
Read our
blog post
or our paper (preprint coming soon) for more details!
A project by Björn Plüster and Christoph Schuhmann in collaboration with LAION and HessianAI.
Model Details
- Finetuned from: meta-llama/Llama-2-7b-hf
- Model type: Causal decoder-only transformer language model
- Language: English and German
- License: LLAMA 2 COMMUNITY LICENSE AGREEMENT
- Contact: LAION Discord or Björn Plüster
Use in 🤗Transformers
First install direct dependencies:
pip install transformers torch sentencepiece
If you want faster inference using flash-attention2, you need to install these dependencies:
pip install packaging ninja
pip install flash-attn==v2.1.1 --no-build-isolation
pip install git+https://github.com/HazyResearch/[email protected]#subdirectory=csrc/rotary
Then load the model in transformers:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
model="LeoLM/leo-hessianai-7b",
device_map="auto",
torch_dtype=torch.float16,
trust_remote_code=True # True for flash-attn2 else False
)
Training parameters
Benchmarks
Runs of jamesdborin LeoLM-hessianai-7b on huggingface.co
30
Total runs
0
24-hour runs
3
3-day runs
3
7-day runs
15
30-day runs
More Information About LeoLM-hessianai-7b huggingface.co Model
Url of LeoLM-hessianai-7b
LeoLM-hessianai-7b huggingface.co Url
Provider of LeoLM-hessianai-7b huggingface.co
Other API from jamesdborin
Total runs:
74
Run Growth:
-14
Growth Rate:
-20.90%
Total runs:
67
Run Growth:
-601
Growth Rate:
-871.01%
Total runs:
67
Run Growth:
6
Growth Rate:
9.52%
Total runs:
49
Run Growth:
-45
Growth Rate:
-109.76%
Total runs:
49
Run Growth:
1
Growth Rate:
2.04%
Total runs:
47
Run Growth:
1
Growth Rate:
2.13%
Total runs:
46
Run Growth:
-55
Growth Rate:
-134.15%
Total runs:
44
Run Growth:
7
Growth Rate:
15.91%
Total runs:
42
Run Growth:
22
Growth Rate:
52.38%
Total runs:
36
Run Growth:
22
Growth Rate:
61.11%
Total runs:
35
Run Growth:
20
Growth Rate:
57.14%
Total runs:
35
Run Growth:
20
Growth Rate:
57.14%
Total runs:
35
Run Growth:
19
Growth Rate:
54.29%
Total runs:
35
Run Growth:
-45
Growth Rate:
-150.00%
Total runs:
34
Run Growth:
23
Growth Rate:
67.65%
Total runs:
34
Run Growth:
17
Growth Rate:
50.00%
Total runs:
34
Run Growth:
17
Growth Rate:
50.00%
Total runs:
34
Run Growth:
22
Growth Rate:
64.71%
Total runs:
34
Run Growth:
22
Growth Rate:
64.71%
Total runs:
34
Run Growth:
-11
Growth Rate:
-32.35%
Total runs:
33
Run Growth:
19
Growth Rate:
57.58%
Total runs:
33
Run Growth:
19
Growth Rate:
57.58%
Total runs:
32
Run Growth:
-10
Growth Rate:
-31.25%
Total runs:
32
Run Growth:
20
Growth Rate:
62.50%
Total runs:
32
Run Growth:
13
Growth Rate:
40.63%
Total runs:
31
Run Growth:
16
Growth Rate:
51.61%
Total runs:
31
Run Growth:
21
Growth Rate:
67.74%
Total runs:
31
Run Growth:
17
Growth Rate:
54.84%
Total runs:
31
Run Growth:
-14
Growth Rate:
-45.16%
Total runs:
31
Run Growth:
20
Growth Rate:
64.52%
Total runs:
31
Run Growth:
16
Growth Rate:
51.61%
Total runs:
30
Run Growth:
17
Growth Rate:
56.67%
Total runs:
30
Run Growth:
9
Growth Rate:
30.00%
Total runs:
30
Run Growth:
19
Growth Rate:
63.33%
Total runs:
30
Run Growth:
17
Growth Rate:
56.67%
Total runs:
30
Run Growth:
4
Growth Rate:
13.33%
Total runs:
30
Run Growth:
12
Growth Rate:
40.00%
Total runs:
30
Run Growth:
15
Growth Rate:
50.00%
Total runs:
30
Run Growth:
18
Growth Rate:
60.00%
Total runs:
28
Run Growth:
13
Growth Rate:
46.43%
Total runs:
28
Run Growth:
4
Growth Rate:
14.29%
Total runs:
28
Run Growth:
15
Growth Rate:
53.57%
Total runs:
27
Run Growth:
17
Growth Rate:
62.96%
Total runs:
27
Run Growth:
10
Growth Rate:
37.04%
Total runs:
27
Run Growth:
-6
Growth Rate:
-22.22%
Total runs:
27
Run Growth:
12
Growth Rate:
44.44%
Total runs:
26
Run Growth:
17
Growth Rate:
65.38%
Total runs:
26
Run Growth:
14
Growth Rate:
53.85%
Total runs:
27
Run Growth:
13
Growth Rate:
50.00%
Total runs:
25
Run Growth:
13
Growth Rate:
52.00%
Total runs:
25
Run Growth:
12
Growth Rate:
48.00%
Total runs:
25
Run Growth:
10
Growth Rate:
40.00%
Total runs:
22
Run Growth:
11
Growth Rate:
50.00%
Total runs:
20
Run Growth:
1
Growth Rate:
5.00%
Total runs:
19
Run Growth:
10
Growth Rate:
52.63%
Total runs:
17
Run Growth:
3
Growth Rate:
23.08%
Total runs:
16
Run Growth:
0
Growth Rate:
0.00%
Total runs:
14
Run Growth:
1
Growth Rate:
10.00%
Total runs:
13
Run Growth:
5
Growth Rate:
38.46%
Total runs:
11
Run Growth:
2
Growth Rate:
18.18%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%