mnavas / roberta-finetuned-CPV_Spanish

roberta-finetuned-CPV_Spanish

This model is a fine-tuned version of PlanTL-GOB-ES/roberta-base-bne on a dataset derived from Spanish Public Procurement documents from 2019. The whole fine-tuning process is available in the following Kaggle notebook .

It achieves the following results on the evaluation set:

• Loss: 0.0465
• F1: 0.7918
• Roc Auc: 0.8860
• Accuracy: 0.7376
• Coverage Error: 10.2744
• Label Ranking Average Precision Score: 0.7973

Intended uses & limitations

This model only predicts the first two digits of the CPV codes. The list of divisions CPV codes is the following:

Training and evaluation data

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Framework versions

• Transformers 4.16.2
• Pytorch 1.9.1
• Datasets 1.18.4
• Tokenizers 0.11.6

Aknowledgments

This work has been supported by NextProcurement European Action (grant agreement INEA/CEF/ICT/A2020/2373713-Action 2020-ES-IA-0255) and the Madrid Government (Comunidad de Madrid-Spain) under the Multiannual Agreement with Universidad Politécnica de Madrid in the line Support for R&D projects for Beatriz Galindo researchers, in the context of the V PRICIT (Regional Programme of Research and Technological Innovation). We also acknowledge the participation of Jennifer Tabita for the preparation of the initial set of notebooks, and the AI4Gov master students from the first cohort for their validation of the approach. Source of the data: Ministerio de Hacienda.