Total runs:
44.8K
Run Growth:
44.7K
Growth Rate:
99.86%
Sponsored by
iMean AI Trip Planner
aieng-lab / math_pretrained_bert-mamut
Model's Last Updated: May 26 2025
Introduction of math_pretrained_bert-mamut
Model Details of math_pretrained_bert-mamut
MAMUT-MPBert (Math Mutator Math-Pretrained-BERT)
MAMUT-MPBERT is a pretrained language model based on AnReu/math_pretrained_bert , further pretrained on mathematical texts and formulas. It was introduced in MAMUT: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model Training .
Despite its base model is already a mathematical model, our training aims to improve the mathematical understanding even further, as shown in our paper.
Model Details
Overview
MAMUT-MPBERT was pretrained on four math-specific tasks across four datasets.
- Mathematical Formulas (MF) : A Masked Language Modeling (MLM) task on math formulas written in LaTeX.
- Mathematical Texts (MT) : An MLM task on natural language text containing inline LaTeX math ( mathematical texts ). The masking probability was biased toward mathematical tokens (inside math environment $...$) and domain-specific terms (e.g., sum , one , ...)
- Named Math Formulas (NMF) : A Next-Sentence-Prediction (NSP)-style task: given a formula and the name of a mathematical identity (e.g., Pythagorean Theorem), classify whether they match.
- Math Formula Retrieval (MFR) : Another NSP-style task to decide if two formulas describe the same mathematical identity or concept.
Model Sources
- Base Model: AnReu/math_pretrained_bert (whose base model is bert-base-cased )
- Pretraining Code: aieng-lab/transformer-math-pretraining
- MAMUT Repository: aieng-lab/math-mutator
- Paper: MAMUT: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model Training
Uses
MAMUT-MPBERT is intended for downstream tasks that require improved mathematical understanding, such as:
- Formula classification
- Retrieval of semantically similar formulas
- Math-related question answering
Note: This model was saved without the MLM or NSP heads and requires fine-tuning before use in downstream tasks.
Similarly trained models are
MAMUT-BERT based on
bert-base-cased
and
MAMUT-MathBERT based on
tbs17/MathBERT
. However, this model (MAMUT-MPBert) performed best according to our evalution.
Training Details
Training configurations are described in Appendix C of the MAMUT paper .
Evaluation
The model is evaluated in Section 7 and Appendix C.4 of the MAMUT paper (MAMUT-MPBERT).
Environmental Impact
- Hardware Type: 8xA100
- Hours used: 48
- Compute Region: Germany
Citation
BibTeX:
@misc{drechsel2025mamutnovelframeworkmodifying,
title={{MAMUT}: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model Training},
author={Jonathan Drechsel and Anja Reusch and Steffen Herbold},
year={2025},
eprint={2502.20855},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2502.20855},
}
Runs of aieng-lab math_pretrained_bert-mamut on huggingface.co
19
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About math_pretrained_bert-mamut huggingface.co Model
Url of math_pretrained_bert-mamut
math_pretrained_bert-mamut huggingface.co Url
Provider of math_pretrained_bert-mamut huggingface.co
Other API from aieng-lab
Updated:July 08 2025
Total runs:
132
Run Growth:
95
Growth Rate:
71.97%
Total runs:
123
Run Growth:
66
Growth Rate:
53.66%
Total runs:
59
Run Growth:
23
Growth Rate:
38.98%
Total runs:
48
Run Growth:
-4
Growth Rate:
-8.33%
Total runs:
42
Run Growth:
-4
Growth Rate:
-9.52%
Total runs:
40
Run Growth:
38
Growth Rate:
95.00%
Total runs:
34
Run Growth:
29
Growth Rate:
85.29%
Total runs:
33
Run Growth:
-9
Growth Rate:
-27.27%
Total runs:
31
Run Growth:
-14
Growth Rate:
-45.16%
Total runs:
30
Run Growth:
26
Growth Rate:
86.67%
Total runs:
29
Run Growth:
27
Growth Rate:
90.00%
Total runs:
29
Run Growth:
26
Growth Rate:
89.66%
Total runs:
28
Run Growth:
24
Growth Rate:
85.71%
Total runs:
27
Run Growth:
25
Growth Rate:
92.59%
Total runs:
27
Run Growth:
23
Growth Rate:
85.19%
Total runs:
27
Run Growth:
24
Growth Rate:
88.89%
Total runs:
27
Run Growth:
23
Growth Rate:
85.19%
Total runs:
26
Run Growth:
15
Growth Rate:
57.69%
Total runs:
26
Run Growth:
22
Growth Rate:
88.00%
Total runs:
26
Run Growth:
21
Growth Rate:
80.77%
Total runs:
26
Run Growth:
23
Growth Rate:
88.46%
Total runs:
26
Run Growth:
24
Growth Rate:
92.31%
Total runs:
26
Run Growth:
21
Growth Rate:
80.77%
Total runs:
25
Run Growth:
21
Growth Rate:
84.00%
Total runs:
25
Run Growth:
19
Growth Rate:
76.00%
Total runs:
25
Run Growth:
21
Growth Rate:
87.50%
Total runs:
25
Run Growth:
22
Growth Rate:
88.00%
Total runs:
25
Run Growth:
22
Growth Rate:
88.00%
Total runs:
24
Run Growth:
21
Growth Rate:
87.50%
Total runs:
24
Run Growth:
19
Growth Rate:
79.17%
Total runs:
24
Run Growth:
20
Growth Rate:
83.33%
Total runs:
23
Run Growth:
22
Growth Rate:
95.65%
Total runs:
23
Run Growth:
18
Growth Rate:
78.26%
Total runs:
23
Run Growth:
18
Growth Rate:
78.26%
Total runs:
23
Run Growth:
19
Growth Rate:
82.61%
Total runs:
23
Run Growth:
19
Growth Rate:
82.61%
Total runs:
23
Run Growth:
21
Growth Rate:
91.30%
Total runs:
22
Run Growth:
18
Growth Rate:
81.82%
Total runs:
22
Run Growth:
19
Growth Rate:
86.36%
Total runs:
22
Run Growth:
18
Growth Rate:
78.26%
Total runs:
22
Run Growth:
15
Growth Rate:
68.18%
Total runs:
22
Run Growth:
-29
Growth Rate:
-131.82%
Total runs:
22
Run Growth:
19
Growth Rate:
86.36%
Total runs:
22
Run Growth:
19
Growth Rate:
86.36%
Total runs:
22
Run Growth:
19
Growth Rate:
86.36%
Total runs:
22
Run Growth:
19
Growth Rate:
82.61%
Total runs:
21
Run Growth:
17
Growth Rate:
80.95%
Total runs:
21
Run Growth:
18
Growth Rate:
85.71%
Total runs:
21
Run Growth:
17
Growth Rate:
85.00%
Total runs:
21
Run Growth:
17
Growth Rate:
80.95%
Total runs:
21
Run Growth:
16
Growth Rate:
76.19%
Total runs:
21
Run Growth:
18
Growth Rate:
85.71%
Total runs:
21
Run Growth:
17
Growth Rate:
80.95%
Total runs:
21
Run Growth:
17
Growth Rate:
85.00%
Total runs:
20
Run Growth:
16
Growth Rate:
80.00%
Total runs:
20
Run Growth:
16
Growth Rate:
80.00%
Total runs:
20
Run Growth:
17
Growth Rate:
85.00%
Total runs:
20
Run Growth:
17
Growth Rate:
85.00%
Total runs:
20
Run Growth:
17
Growth Rate:
85.00%
Total runs:
20
Run Growth:
16
Growth Rate:
80.00%
Total runs:
20
Run Growth:
18
Growth Rate:
85.71%
Total runs:
20
Run Growth:
16
Growth Rate:
80.00%
Total runs:
20
Run Growth:
17
Growth Rate:
85.00%
Total runs:
19
Run Growth:
15
Growth Rate:
78.95%
Total runs:
19
Run Growth:
15
Growth Rate:
78.95%
Total runs:
19
Run Growth:
17
Growth Rate:
89.47%
Total runs:
19
Run Growth:
16
Growth Rate:
84.21%
Total runs:
19
Run Growth:
16
Growth Rate:
84.21%
Total runs:
19
Run Growth:
16
Growth Rate:
84.21%
Total runs:
19
Run Growth:
16
Growth Rate:
84.21%
Total runs:
19
Run Growth:
15
Growth Rate:
78.95%
Total runs:
18
Run Growth:
14
Growth Rate:
77.78%
Total runs:
18
Run Growth:
14
Growth Rate:
77.78%
Total runs:
18
Run Growth:
14
Growth Rate:
77.78%
Total runs:
18
Run Growth:
16
Growth Rate:
88.89%
Total runs:
18
Run Growth:
14
Growth Rate:
77.78%
Total runs:
18
Run Growth:
15
Growth Rate:
83.33%
Total runs:
18
Run Growth:
5
Growth Rate:
27.78%
Total runs:
18
Run Growth:
13
Growth Rate:
72.22%
Total runs:
18
Run Growth:
15
Growth Rate:
83.33%
Total runs:
18
Run Growth:
13
Growth Rate:
76.47%
Total runs:
18
Run Growth:
17
Growth Rate:
89.47%
Total runs:
18
Run Growth:
15
Growth Rate:
83.33%
Total runs:
18
Run Growth:
15
Growth Rate:
83.33%
Total runs:
17
Run Growth:
14
Growth Rate:
82.35%
Total runs:
17
Run Growth:
14
Growth Rate:
82.35%
Total runs:
17
Run Growth:
13
Growth Rate:
76.47%
Total runs:
17
Run Growth:
14
Growth Rate:
82.35%
Total runs:
17
Run Growth:
11
Growth Rate:
64.71%
Total runs:
17
Run Growth:
13
Growth Rate:
76.47%
Total runs:
17
Run Growth:
14
Growth Rate:
82.35%
Total runs:
17
Run Growth:
13
Growth Rate:
72.22%
Total runs:
17
Run Growth:
13
Growth Rate:
76.47%
Total runs:
17
Run Growth:
13
Growth Rate:
76.47%
Total runs:
17
Run Growth:
12
Growth Rate:
70.59%