c-bone / CrystaLLM-pi_Chili100K-XRD

Model Card for CrystaLLM-pi_Chili100K-XRD

Model Details

Model Description

CrystaLLM-pi_Chili100K-XRD is a conditional generative model designed for the recovery of crystal structures from X-ray Diffraction (XRD) data. It is a fine-tuned version of the CrystaLLM-pi framework, utilizing a GPT-2 decoder-only architecture. This model employs a Residual Attention (Slider) mechanism to condition the generation of Crystallographic Information Files (CIFs) on heterogeneous X-ray diffraction data.

The model generates crystal structures based on an XRD pattern input vector consisting of the 20 most intense peaks:

1. Peak Positions ($2\theta$)
2. Peak Intensities

The Chili-100K XRD dataset the model is fine-tuned on contains experimentally determined structures sourced from Chili-100K, which is an inorganic experimental nanomaterials curated and filtered subset of the Crystallographic Open Database (COD). Notably, this model features an extended context window of 1536 tokens , enabling the generation of larger and more complex unit cells containing up to ~100 atoms.

• Developed by: Bone et al. (University College London)
• Model type: Autoregressive Transformer with Residual Attention Conditioning
• Language(s): CIF (Crystallographic Information File) syntax
• License: MIT
• Finetuned from model: c-bone/CrystaLLM-pi_Mattergen-XRD

Model Sources

• Repository: GitHub: CrystaLLM-pi
• Paper: Discovery and recovery of crystalline materials with property-conditioned transformers (arXiv:2511.21299)
• Dataset: HuggingFace: c-bone/mattergen_XRD (Stage 1), HuggingFace: c-bone/chili100k_strat (Stage 2)

Uses

Direct Use

The model is intended for structure solution and recovery from powder XRD data. Researchers can input a list of peak positions and intensities derived from experimental diffraction patterns to generate candidate crystal structures that match the experimental signature.

Out-of-Scope Use

• Disordered Systems: The model does not natively handle partial occupancies or significant disorder.
• Organic/MOFs: The training data was strictly filtered for inorganic nanomaterials as per the Chili-100K dataset methdology.
• Extremely Large Unit Cells: While the context window is expanded to 1536 tokens, structures with high numbers of atoms per unit cell may face or degradation in generation quality.

Bias, Risks, and Limitations

• Experimental Noise: Performance relies on the quality of the input peak extraction and rarity of material.
• Missing Data: The "Slider" mechanism handles missing peaks (padded with -100), but significant data loss degrades recovery rates.
• Polymorphs: In cases of strong structural similarity, the model may bias towards the polymorph most prevalent in the Chili-100K distribution.

How to Get Started with the Model

For instructions on loading and running generation, refer to the _load_and_generate.py script in the CrystaLLM-pi GitHub Repository . This script handles XRD vector tokenization and normalization.

Training Details

Training Data

The model underwent a two-stage fine-tuning process:

1. MatterGen XRD: Theoretical XRD patterns generated from the MatterGen dataset.
2. Chili-100K XRD ( c-bone/chili100k_strat ): An experimentally determined, curated, and filtered subset of inorganic nanomaterials from the COD (accessed April 2026). After deduplication, this comprises ~14K materials derived from ~21K CIFs.

Dataset Splitting (Chili-100K):

• Train:Val:Test Ratio: 78.6:10.7:10.7
• Leakage-Aware Test Set: The test set was strictly stratified to evaluate generalization:
  • 500 materials: Fully seen during training (LeMaterial, MatterGen XRD, or Chili-100K train/val).
  • 500 materials: Reduced formula seen during training, but the specific structure was unseen (measured via Structure Novelty metric).
  • 500 materials: Neither reduced formula nor structure seen in any training phase.

Training Procedure

• Architecture: GPT-2 with Residual Attention (Slider) layers. (~47.7M parameters)
• Mechanism: The Slider mechanism computes a parallel attention score for the conditioning vector, dynamically weighting it against base self-attention to robustly handle heterogeneous/missing diffraction data.

Evaluation

Metrics

The model is evaluated on the leakage-aware test splits using:

1. Match Rate: Percentage of ground truth structures successfully recovered.
2. RMS-d: Root Mean Square distance between ground truth and generated structures.
3. Lattice Parameter and Volume MAE: Mean Absolute Error of predicted unit cell dimensions.
4. N atoms match : The average amount of atoms in the unit cell of matched material in the test set.

Citation

Primary Model Paper:

@misc{bone2025discoveryrecoverycrystallinematerials,
      title={Discovery and recovery of crystalline materials with property-conditioned transformers}, 
      author={Cyprien Bone and Matthew Walker and Kuangdai Leng and Luis M. Antunes and Ricardo Grau-Crespo and Amil Aligayev and Javier Dominguez and Keith T. Butler},
      year={2025},
      eprint={2511.21299},
      archivePrefix={arXiv},
      primaryClass={cond-mat.mtrl-sci},
      url={https://arxiv.org/abs/2511.21299}, 
}

CHILI Dataset:

@inproceedings{10.1145/3637528.3671538,
      author = {Friis-Jensen, Ulrik and Johansen, Frederik L. and Anker, Andy S. and Dam, Erik B. and Jensen, Kirsten M. \O{}. and Selvan, Raghavendra},
      title = {CHILI: Chemically-Informed Large-scale Inorganic Nanomaterials Dataset for Advancing Graph Machine Learning},
      year = {2024},
      isbn = {9798400704901},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3637528.3671538},
      doi = {10.1145/3637528.3671538},
      pages = {4962–4973},
      numpages = {12},
      keywords = {atomic structure, chemistry, datasets, deep learning, graph neural network, graphs, machine learning, nanomaterials, neutron, scattering, x-ray},
      location = {Barcelona, Spain},
      series = {KDD '24}
}