project-aps / finbert-finetune

📊 FinBERT Fine-Tuned on Financial News/Texts

A fine-tuned version of ProsusAI/finbert trained for financial sentiment analysis on financial news texts and headlines. This fine-tuned model achieves a significant improvement over the original finbert, outperforming it by over 38% in accuracy on financial sentiment classification tasks.

---

🔧 Model Objective

The goal of this model is to detect positive , neutral , or negative sentiment on financial texts and headlines.

---

🗂️ Training Dataset

Primary Dataset : fingpt-sentiment-train (~60,000 examples)

• Labeled financial text samples (positive / neutral / negative)
• Includes earnings statements, market commentary, and financial news headlines
• Only included neutral , positive and negative texts.

---

🧪 Benchmark Evaluation

The model was evaluated against three benchmark datasets :

• Financial PhraseBank (All Agree and All Combined)
• FiQA + PhraseBank Kaggle Merge
• fingpt-sentiment-train (test split)

Metrics used:

• Accuracy
• F1 Score
• Precision
• Recall

We benchmarked this model against the original ProsusAI/finbert on multiple financial datasets:

Note: All metrics represent classification performance improvements after fine-tuning FinBERT on respective financial sentiment datasets. Metrics in parentheses represent relative improvement over base FinBERT performance.

---

🧠 Text-Level Comparison: FinBERT vs FinBERT-Finetuned (Ours)

🔴 FinBERT Failed Texts (as per discussed in its Paper ) (Correctly Predicted by Ours)

🟡 FinBERT Incorrect, Ours Corrected It

⚪ Out-of-Context Texts (FinBERT Misclassified, Ours Handled Properly)

Note : These examples demonstrate improvements in real-world understanding, context handling, and sentiment differentiation with our FinBERT-finetuned model. Values in parentheses (e.g., 0.9485 ) indicate the model’s confidence score for its predicted sentiment.

---

⚠️ Limitations & Failure Cases

While the model outperformed the base FinBERT across benchmarks, some failure cases were observed in statements involving fine-grained numerical reasoning , particularly when numerical comparison semantics are complex or subtle.

Note : Values in parentheses (e.g., 0.9485 ) indicate the model’s confidence score for its predicted sentiment.

This suggests that explicit numerical comparison reasoning still remains challenging without targeted pretraining or numerical reasoning augmentation.

---

Hyperparameters

During fine-tuning, the following hyperparameters were used to optimize model performance:

• Learning Rate: 2e-5
• Batch Size: 32
• Number of Epochs: 3
• Max Sequence Length: 128 tokens
• Optimizer: AdamW
• Weight Decay: 0.01
• Evaluation Strategy: Evaluation performed after each epoch

Note : These settings were chosen to balance training efficiency and accuracy for financial news sentiment classification.

---

💡 Summary

✅ Better generalization than FinBERT on both benchmark and noisy real-world samples
✅ Strong accuracy and F1 scores
⚠️ Room to improve on numerical reasoning comparisons — potential for integration with numerical-aware transformers or contrastive fine-tuning

---

Usage

Pipeline Approach

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline
import torch

model_name = "project-aps/finbert-finetune"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Override the config's id2label and label2id
label_map = {0: "neutral", 1: "negative", 2: "positive"}
model.config.id2label = label_map
model.config.label2id = {v: k for k, v in label_map.items()}

pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)

text = "Earnings smashed expectations AAPL posts $0.89 EPS vs $0.78 est. Bullish momentum incoming! #EarningsSeason"
print(pipe(text)) #Output: [{'label': 'positive', 'score': 0.9997484087944031}]

Simple Approach

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "project-aps/finbert-finetune"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "Earnings smashed expectations AAPL posts $0.89 EPS vs $0.78 est. Bullish momentum incoming! #EarningsSeason"
inputs = tokenizer(text, return_tensors="pt", truncation=True)
outputs = model(**inputs)
predicted_class = torch.argmax(outputs.logits, dim=1).item()

label_map = {0: "neutral", 1: "negative", 2: "positive"}
print(f"Text : {text}")
print(f"Sentiment: {label_map[predicted_class]}")

---

Acknowledgements

We gratefully acknowledge the creators and maintainers of the resources used in this project:

• ProsusAI/FinBERT – A pre-trained BERT model specifically designed for financial sentiment analysis, which served as the foundation for our fine-tuning efforts.

• FinGPT Sentiment Train Dataset – The dataset used for fine-tuning, containing a large collection of finance-related news headlines and sentiment annotations.

• Financial PhraseBank Dataset – A widely used benchmark dataset for financial sentiment classification, including the All Agree and All Combined subsets.

• FiQA + PhraseBank Kaggle Merged Dataset – A merged dataset combining FiQA and Financial PhraseBank entries, used for broader benchmarking of sentiment performance.

We thank these contributors for making their models and datasets publicly available, enabling high-quality research and development in financial NLP.

---