This model is a fine tune of the excellent
tasksource/ModernBERT-large-nli
, using the
dleemiller/FineCat-NLI
dataset. This dataset is a compilation of several high quality sources of NLI data, with quality screening and
reduction of easy samples in the train split.
This model and dataset specifically targets improving NLI, through high quality sources. The tasksource models
are the best checkpoints to start from, although training from ModernBERT is also competitive.
NLI Evaluation Results
F1-Micro scores (equivalent to accuracy) for each dataset.
Model
finecat
mnli
mnli_mismatched
snli
anli_r1
anli_r2
anli_r3
wanli
lingnli
dleemiller/finecat-nli-l
0.8152
0.9088
0.9217
0.9259
0.7400
0.5230
0.5150
0.7424
0.8689
tasksource/ModernBERT-large-nli
0.7959
0.8983
0.9229
0.9188
0.7260
0.5110
0.4925
0.6978
0.8504
dleemiller/ModernCE-large-nli
0.7811
0.9088
0.9205
0.9273
0.6630
0.4860
0.4408
0.6576
0.8566
tasksource/ModernBERT-base-nli
0.7595
0.8685
0.8979
0.8915
0.6300
0.4820
0.4192
0.6632
0.8118
dleemiller/ModernCE-base-nli
0.7533
0.8923
0.9035
0.9187
0.5240
0.3950
0.3333
0.6464
0.8282
dleemiller/EttinX-nli-s
0.7251
0.8765
0.8798
0.9128
0.3360
0.2790
0.3083
0.6234
0.8012
dleemiller/EttinX-nli-xs
0.7013
0.8376
0.8380
0.8979
0.2780
0.2840
0.2800
0.5838
0.7521
dleemiller/EttinX-nli-xxs
0.6842
0.7988
0.8047
0.8851
0.2590
0.3060
0.2992
0.5426
0.7018
Usage
Label Map:
entailment
: 0
neutral
: 1
contradiction
: 2
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import CrossEncoder
# Download from the 🤗 Hub
model = CrossEncoder("dleemiller/finecat-nli-l")
# Get scores for pairs of texts
pairs = [
['She was quite young, not more than eighteen.', 'The girls was at least eighteen years old, but not much older. '],
['Two men working on something in a shop.', 'Two boys are milking a milk cart.'],
['A man in goggle swimming.', 'A man swimming in the water with goggles.'],
['The Argentine Grand Prix (Spanish: "Gran Premio de Argentina") was a round of the Formula One championship, held intermittently from to , all at the same autodrome in the Argentine national capital of Buenos Aires. Argentine president Juan Perón was the driving force behind the creation of the circuit, after seeing the success of the country\'s own Juan Manuel Fangio.', 'The Argentine Grand Prix involves the racing of automobiles with four wheels'],
['Because of limited resources, local legal services programs are forced to turn away tens of thousands of people with critical legal problems.', 'If the resources were there, it would be unthinkable for these programs to be turning so many people away.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5, 3)
Runs of dleemiller finecat-nli-l on huggingface.co
2.1K
Total runs
-90
24-hour runs
-222
3-day runs
-524
7-day runs
-279
30-day runs
More Information About finecat-nli-l huggingface.co Model
finecat-nli-l huggingface.co
finecat-nli-l huggingface.co is an AI model on huggingface.co that provides finecat-nli-l's model effect (), which can be used instantly with this dleemiller finecat-nli-l model. huggingface.co supports a free trial of the finecat-nli-l model, and also provides paid use of the finecat-nli-l. Support call finecat-nli-l model through api, including Node.js, Python, http.
finecat-nli-l huggingface.co is an online trial and call api platform, which integrates finecat-nli-l's modeling effects, including api services, and provides a free online trial of finecat-nli-l, you can try finecat-nli-l online for free by clicking the link below.
dleemiller finecat-nli-l online free url in huggingface.co:
finecat-nli-l is an open source model from GitHub that offers a free installation service, and any user can find finecat-nli-l on GitHub to install. At the same time, huggingface.co provides the effect of finecat-nli-l install, users can directly use finecat-nli-l installed effect in huggingface.co for debugging and trial. It also supports api for free installation.