SIRIS-Lab / specter2-fapesp-area-multiclass

📗 SPECTER2–FAPESP Knowledge Area (Multiclass Classification on FAPESP Area do Conhecimento (Level 1))

This model is a fine-tuned version of allenai/specter2_base on the FAPESP dataset. It achieves the following results on the evaluation set:

• Loss: 1.3826
• Accuracy: 0.6926
• Precision Micro: 0.6926
• Precision Macro: 0.6941
• Recall Micro: 0.6926
• Recall Macro: 0.6794
• F1 Micro: 0.6926
• F1 Macro: 0.6765

Model description

This model is a fine-tuned version of SPECTER2 ( allenai/specter2_base ) adapted for multiclass classification across the 76 Áreas do Conhecimento of FAPESP .

The model accepts the title, abstract, or title + abstract of a research projects and assigns it to exactly one of the Areas (e.g., Veterinary Medicine, Dentistry, Physiotherapy and Occupational Therapy, Philosophy).

Key characteristics:

• Base model: allenai/specter2_base
• Task: multiclass document classification
• Labels: 76 Knowledge Areas
• Activation: softmax
• Loss: CrossEntropyLoss
• Output: single best-matching FAPESP's Knowledge Area category

FAPESP's Knowledge Areas represents broad disciplinary domains designed for high-level categorization of R&I documents.

Intended uses & limitations

This multiclass model is suitable for:

• Assigning publications to top-level scientific disciplines
• Enriching metadata in:
  • repositories
  • research output systems
  • funding and project datasets
  • bibliometric dashboards
• Supporting scientometric analyses such as:
  • broad-discipline portfolio mapping
  • domain-level clustering
  • modeling research diversification
• Classifying documents when only title/abstract is available

The model supports inputs such as:

• title only
• abstract only
• title + abstract (recommended)

Limitations

• Documents spanning multiple fields must be forced into one label—an inherent limitation of multiclass classification.
• The training labels come from FAPESP funded projects , not manual expert annotation.
• Not suitable for:
  • downstream tasks requiring multilabel outputs
  • WoS Categories or ASJC Areas (use separate models)
  • clinical or regulatory decision-making

Predictions should be treated as field-level disciplinary metadata .

Training and evaluation data

The training and evaluation dataset was constructed from publicly available FAPESP (Fundação de Amparo à Pesquisa do Estado de São Paulo) research project records. These records cover funded research projects and scholarships across all scientific domains in Brazil.

The dataset was assembled using the following CSV downloads provided by FAPESP:

• Auxílios em andamento (ongoing research grants)
• Auxílios concluídos (completed research grants)
• Bolsas no Brasil em andamento (ongoing domestic scholarships)
• Bolsas no Brasil concluídas (completed domestic scholarships)
• Bolsas no exterior em andamento (ongoing international scholarships)
• Bolsas no exterior concluídas (completed international scholarships)

Each record contains metadata such as project titles, abstracts, funding type, and scientific classifications.
From these files, the following fields were extracted and standardized:

• Title (English)
• Abstract (English)
• Grande Área do Conhecimento (major scientific domain)
• Área do Conhecimento (field of study)

Only entries containing at least one English component (title or abstract) were retained.
Scientific areas were normalized and mapped to a controlled English taxonomy to ensure consistency and comparability across records.

The final dataset consists of labeled scientific text samples distributed across multiple domains, providing a balanced corpus for supervised classification.

Training procedure

Preprocessing

• Input text constructed as:
  abstract
• Tokenization using the SPECTER2 tokenizer
• Maximum sequence length: 512 tokens

Model

• Base model: allenai/specter2_base
• Classification head: linear layer → softmax
• Loss: CrossEntropyLoss

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Evaluation results

Framework versions

• Transformers 4.57.1
• Pytorch 2.8.0+cu126
• Datasets 3.6.0
• Tokenizers 0.22.1