Total runs:
265.7K
Run Growth:
-83.6K
Growth Rate:
-31.28%
Sponsored by
APIMart
mrm8488 / longformer-base-4096-finetuned-squadv2
Model's Last Updated: December 05 2022
question-answering
Introduction of longformer-base-4096-finetuned-squadv2
Model Details of longformer-base-4096-finetuned-squadv2
Longformer-base-4096 fine-tuned on SQuAD v2
Longformer-base-4096 model fine-tuned on SQuAD v2 for Q&A downstream task.
Longformer-base-4096
Longformer is a transformer model for long documents.
longformer-base-4096
is a BERT-like model started from the RoBERTa checkpoint and pretrained for MLM on long documents. It supports sequences of length up to 4,096.
Longformer uses a combination of a sliding window (local) attention and global attention. Global attention is user-configured based on the task to allow the model to learn task-specific representations.
Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓
Dataset ID:
squad_v2
from
HuggingFace/Datasets
| Dataset | Split | # samples |
|---|---|---|
| squad_v2 | train | 130319 |
| squad_v2 | valid | 11873 |
How to load it from datasets
!pip install datasets
from datasets import load_dataset
dataset = load_dataset('squad_v2')
Check out more about this dataset and others in Datasets Viewer
Model fine-tuning 🏋️
The training script is a slightly modified version of this one
Model in Action 🚀
import torch
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
ckpt = "mrm8488/longformer-base-4096-finetuned-squadv2"
tokenizer = AutoTokenizer.from_pretrained(ckpt)
model = AutoModelForQuestionAnswering.from_pretrained(ckpt)
text = "Huggingface has democratized NLP. Huge thanks to Huggingface for this."
question = "What has Huggingface done ?"
encoding = tokenizer(question, text, return_tensors="pt")
input_ids = encoding["input_ids"]
# default is local attention everywhere
# the forward method will automatically set global attention on question tokens
attention_mask = encoding["attention_mask"]
start_scores, end_scores = model(input_ids, attention_mask=attention_mask)
all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0].tolist())
answer_tokens = all_tokens[torch.argmax(start_scores) :torch.argmax(end_scores)+1]
answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))
# output => democratized NLP
Usage with HF
pipleine
from transformers import AutoTokenizer, AutoModelForQuestionAnswering, pipeline
ckpt = "mrm8488/longformer-base-4096-finetuned-squadv2"
tokenizer = AutoTokenizer.from_pretrained(ckpt)
model = AutoModelForQuestionAnswering.from_pretrained(ckpt)
qa = pipeline("question-answering", model=model, tokenizer=tokenizer)
text = "Huggingface has democratized NLP. Huge thanks to Huggingface for this."
question = "What has Huggingface done?"
qa({"question": question, "context": text})
If given the same context we ask something that is not there, the output for
no answer
will be
<s>
Created by Manuel Romero/@mrm8488 | LinkedIn
Made with ♥ in Spain
Runs of mrm8488 longformer-base-4096-finetuned-squadv2 on huggingface.co
300
Total runs
0
24-hour runs
0
3-day runs
67
7-day runs
-48
30-day runs
More Information About longformer-base-4096-finetuned-squadv2 huggingface.co Model
Url of longformer-base-4096-finetuned-squadv2
longformer-base-4096-finetuned-squadv2 huggingface.co Url
Provider of longformer-base-4096-finetuned-squadv2 huggingface.co
Other API from mrm8488
Updated:January 21 2024
Total runs:
65.7K
Run Growth:
49.0K
Growth Rate:
74.57%
Total runs:
33.2K
Run Growth:
353
Growth Rate:
1.08%
Total runs:
12.8K
Run Growth:
-577
Growth Rate:
-4.50%
Total runs:
9.5K
Run Growth:
4.9K
Growth Rate:
50.91%
Total runs:
8.7K
Run Growth:
356
Growth Rate:
4.12%
Total runs:
8.7K
Run Growth:
5.7K
Growth Rate:
65.22%
Total runs:
7.6K
Run Growth:
-4.7K
Growth Rate:
-62.21%
Total runs:
5.9K
Run Growth:
-55
Growth Rate:
-0.94%
Total runs:
4.4K
Run Growth:
1.6K
Growth Rate:
35.76%
Total runs:
3.7K
Run Growth:
-870
Growth Rate:
-22.77%
Total runs:
2.4K
Run Growth:
-1.3K
Growth Rate:
-53.98%
Total runs:
1.5K
Run Growth:
142
Growth Rate:
9.65%
Total runs:
1.4K
Run Growth:
662
Growth Rate:
45.06%
Total runs:
1.1K
Run Growth:
673
Growth Rate:
60.79%
Total runs:
987
Run Growth:
59
Growth Rate:
6.10%
Total runs:
973
Run Growth:
-1.3K
Growth Rate:
-133.57%
Total runs:
805
Run Growth:
-14
Growth Rate:
-1.75%
Total runs:
779
Run Growth:
59
Growth Rate:
7.44%
Total runs:
682
Run Growth:
335
Growth Rate:
49.12%
Total runs:
651
Run Growth:
-309
Growth Rate:
-47.47%
Total runs:
647
Run Growth:
-219
Growth Rate:
-31.06%
Total runs:
634
Run Growth:
-1.3K
Growth Rate:
-213.56%
Total runs:
617
Run Growth:
75
Growth Rate:
12.16%
Total runs:
475
Run Growth:
-907
Growth Rate:
-187.78%
Total runs:
464
Run Growth:
366
Growth Rate:
78.88%
Total runs:
462
Run Growth:
145
Growth Rate:
31.87%
Total runs:
376
Run Growth:
358
Growth Rate:
95.21%
Total runs:
366
Run Growth:
57
Growth Rate:
15.57%
Total runs:
363
Run Growth:
-189
Growth Rate:
-54.94%
Total runs:
336
Run Growth:
-50
Growth Rate:
-15.24%
Total runs:
326
Run Growth:
88
Growth Rate:
26.99%
Total runs:
324
Run Growth:
197
Growth Rate:
61.76%
Total runs:
269
Run Growth:
24
Growth Rate:
8.92%
Total runs:
260
Run Growth:
255
Growth Rate:
98.08%
Total runs:
250
Run Growth:
-122
Growth Rate:
-48.80%
Total runs:
242
Run Growth:
-17
Growth Rate:
-7.02%
Total runs:
227
Run Growth:
-197
Growth Rate:
-86.78%
Total runs:
197
Run Growth:
39
Growth Rate:
19.80%
Total runs:
190
Run Growth:
154
Growth Rate:
83.24%
Total runs:
175
Run Growth:
140
Growth Rate:
80.92%
Total runs:
168
Run Growth:
93
Growth Rate:
57.41%
Total runs:
165
Run Growth:
90
Growth Rate:
43.06%
Total runs:
159
Run Growth:
-105
Growth Rate:
-70.47%
Total runs:
154
Run Growth:
120
Growth Rate:
77.92%
Total runs:
150
Run Growth:
-314
Growth Rate:
-205.23%
Total runs:
131
Run Growth:
62
Growth Rate:
46.97%
Total runs:
128
Run Growth:
-39
Growth Rate:
-36.79%
Total runs:
127
Run Growth:
55
Growth Rate:
44.00%
Total runs:
125
Run Growth:
42
Growth Rate:
33.60%
Total runs:
124
Run Growth:
-225
Growth Rate:
-181.45%
Total runs:
122
Run Growth:
-54
Growth Rate:
-44.26%
Total runs:
118
Run Growth:
10
Growth Rate:
9.80%
Total runs:
116
Run Growth:
38
Growth Rate:
32.76%
Total runs:
115
Run Growth:
85
Growth Rate:
73.91%
Total runs:
114
Run Growth:
52
Growth Rate:
45.61%
Total runs:
113
Run Growth:
19
Growth Rate:
16.96%
Total runs:
112
Run Growth:
108
Growth Rate:
89.26%
Total runs:
111
Run Growth:
26
Growth Rate:
24.30%
Total runs:
110
Run Growth:
56
Growth Rate:
50.45%
Total runs:
109
Run Growth:
29
Growth Rate:
26.61%
Total runs:
107
Run Growth:
74
Growth Rate:
69.81%
Total runs:
105
Run Growth:
64
Growth Rate:
62.14%
Total runs:
105
Run Growth:
70
Growth Rate:
66.67%
Total runs:
104
Run Growth:
63
Growth Rate:
60.58%
Total runs:
99
Run Growth:
-107
Growth Rate:
-108.08%
Total runs:
94
Run Growth:
-18
Growth Rate:
-19.15%
Total runs:
92
Run Growth:
68
Growth Rate:
73.91%
Total runs:
87
Run Growth:
-71
Growth Rate:
-81.61%
Total runs:
85
Run Growth:
-76
Growth Rate:
-89.41%
Total runs:
84
Run Growth:
-8
Growth Rate:
-9.52%
Total runs:
82
Run Growth:
-40
Growth Rate:
-48.78%
Total runs:
82
Run Growth:
57
Growth Rate:
69.51%
Total runs:
81
Run Growth:
73
Growth Rate:
90.12%
Total runs:
79
Run Growth:
51
Growth Rate:
64.56%
Total runs:
78
Run Growth:
-23
Growth Rate:
-36.51%
Total runs:
76
Run Growth:
47
Growth Rate:
61.84%
Total runs:
73
Run Growth:
31
Growth Rate:
42.47%
Total runs:
66
Run Growth:
48
Growth Rate:
77.42%
Total runs:
65
Run Growth:
24
Growth Rate:
36.92%
Total runs:
63
Run Growth:
31
Growth Rate:
50.82%
Total runs:
63
Run Growth:
22
Growth Rate:
34.92%
Total runs:
60
Run Growth:
30
Growth Rate:
50.85%
Total runs:
58
Run Growth:
7
Growth Rate:
12.50%
Total runs:
58
Run Growth:
-104
Growth Rate:
-179.31%
Total runs:
57
Run Growth:
-57
Growth Rate:
-98.28%
Total runs:
57
Run Growth:
-27
Growth Rate:
-47.37%
Total runs:
56
Run Growth:
41
Growth Rate:
73.21%
Total runs:
56
Run Growth:
0
Growth Rate:
0.00%
Total runs:
56
Run Growth:
29
Growth Rate:
51.79%
Total runs:
55
Run Growth:
47
Growth Rate:
85.45%
Total runs:
54
Run Growth:
31
Growth Rate:
62.00%