FlameF0X / arXivGPT

FlameF0X/arXivGPT

A small GPT-2 language model trained from scratch with Auto-PreTrain. It is a standard GPT2LMHeadModel , so no custom code or trust_remote_code is needed.

Usage

from transformers import pipeline

generator = pipeline("text-generation", model="FlameF0X/arXivGPT")
print(generator("Once upon a time", max_new_tokens=50)[0]["generated_text"])

Model

Training data

FlameF0X/arXiv-AI-ML , split train , first 100,000 rows. 4,682 train, 247 eval examples (~630,912 tokens), packed into full-length blocks.

Training setup

Results

Example output

Prompt: Once upon a time

Once upon a time-of-art dataset, allowing the target model's number of the proposed method with the model's training-to-step and a single-training framework for our method. AB-based approach is an multi-based approach that generates a comprehensive

Limitations

This is a tiny model trained on a small data sample. Expect limited coherence and factual accuracy, and it may reproduce biases present in the training data. It is intended for experimentation and learning, not production use.