agentlans / flan-t5-small-ner

flan-t5-small-ner

This model is a fine-tuned version of google/flan-t5-small on the pile_ner dataset . It achieves a loss of 0.5738 on the evaluation set and has processed 199 397 272 input tokens during training.

Model Description

flan-t5-small-ner can extract entities of specific types or definitions from text such as person, company, school, technology, and many more. It builds upon the FLAN-T5 architecture, which has strong performance across natural language processing tasks.

Example:

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
import torch

model_path = "agentlans/flan-t5-small-ner"
model = AutoModelForSeq2SeqLM.from_pretrained(model_path).to("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = AutoTokenizer.from_pretrained(model_path)

def find_entities(input_text, entity_type):
    txt = entity_type + "[SEP]" + input_text
    inputs = tokenizer(txt, return_tensors="pt").to(model.device)
    outputs = model.generate(**inputs, max_new_tokens=100)
    decoded = tokenizer.decode(outputs[0], skip_special_tokens=False)
    raw_list = decoded.split("[SEP]")
    clean_list = [item.replace("<pad>", "").replace("[END]", "").replace("</s>", "").strip() for item in raw_list]
    return [item for item in clean_list if item]

# Example usage
input_text = "In the bustling metropolis of New York City, Apple Inc. sponsored a conference where Dr. Elena Rodriguez presented groundbreaking research."
print(find_entities(input_text, "person")) # ['Dr. Elena Rodriguez']
print(find_entities(input_text, "company")) # ['Apple Inc.']
print(find_entities(input_text, "fruit")) # []

Limitations

• False positives and negatives are possible.
• May struggle with specialized knowledge or fine distinctions.
• Performance may vary for very short or long texts.
• English language only.
• Consider privacy when processing sensitive text.

Training Procedure

Training Hyperparameters

• Learning rate: 5e-05
• Train batch size: 8
• Eval batch size: 8
• Seed: 42
• Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
• LR scheduler type: linear
• Number of epochs: 5.0

Training Results

Framework Versions

• Transformers: 4.46.3
• PyTorch: 2.5.1+cu124
• Datasets: 3.2.0
• Tokenizers: 0.20.3