tomaarsen / span-marker-bert-base-fewnerd-fine-super

SpanMarker with bert-base-cased on FewNERD

This is a SpanMarker model trained on the FewNERD dataset that can be used for Named Entity Recognition. This SpanMarker model uses bert-base-cased as the underlying encoder.

Model Details

Model Description

• Model Type: SpanMarker
• Encoder: bert-base-cased
• Maximum Sequence Length: 256 tokens
• Maximum Entity Length: 8 words
• Training Dataset: FewNERD
• Language: en
• License: cc-by-sa-4.0

Model Sources

• Repository: SpanMarker on GitHub
• Thesis: SpanMarker For Named Entity Recognition

Model Labels

Uses

Direct Use

from span_marker import SpanMarkerModel

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-fewnerd-fine-super")
# Run inference
entities = model.predict("Amelia Earhart flew her single engine Lockheed Vega 5B across the Atlantic to Paris.")

Downstream Use

You can finetune this model on your own dataset.

from span_marker import SpanMarkerModel, Trainer

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-fewnerd-fine-super")

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("tomaarsen/span-marker-bert-base-fewnerd-fine-super-finetuned")

Training Details

Training Set Metrics

Training Hyperparameters

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3

Training Hardware

• On Cloud : No
• GPU Model : 1 x NVIDIA GeForce RTX 3090
• CPU Model : 13th Gen Intel(R) Core(TM) i7-13700K
• RAM Size : 31.78 GB

Framework Versions

• Python: 3.9.16
• SpanMarker: 1.3.1.dev
• Transformers : 4.29.2
• PyTorch: 2.0.1+cu118
• Datasets: 2.14.3
• Tokenizers: 0.13.2