fondress / PDeepPP_ACE

PDeepPP model

PDeepPP is a hybrid protein language model designed to predict post-translational modification (PTM) sites and extract biologically relevant features from protein sequences. By leveraging pretrained embeddings from ESM and incorporating both transformer and convolutional neural network (CNN) architectures, PDeepPP provides a robust framework for analyzing protein sequences in various contexts.

Model description

PDeepPP is a flexible model architecture that integrates the power of transformer-based self-attention mechanisms with convolutional operations for capturing local and global sequence features. The model consists of:

1. A Self-Attention Global Features module for capturing long-range dependencies.
2. A TransConv1d module , combining transformers and convolutional layers.
3. A PosCNN module , which applies position-aware convolutional operations for feature extraction.

The model is trained with a loss function that combines classification loss and additional regularization terms to enhance generalization and interpretability. It is compatible with Hugging Face's transformers library, allowing seamless integration with other tools and workflows.

Intended uses

PDeepPP is designed for two primary tasks:

1. PTM site prediction : Identifying post-translational modification sites (e.g., phosphorylation) in protein sequences, focusing on serine (S), threonine (T), and tyrosine (Y) residues.
2. Biologically active sequence analysis (BPS) : Extracting biologically relevant regions from protein sequences for downstream analysis.

The model processes protein sequences and outputs:

• Embedded representations of the sequences, which can be used for various downstream tasks.
• Predicted probabilities for PTM or other sequence-specific features.

Key features:

• PTM mode : Focuses on sequences centered around specific residues (S, T, Y) to predict PTM activity.
• BPS mode : Analyzes overlapping or non-overlapping subsequences of a protein for broader biological insights.

How to use

To use PDeepPP , you need to install the required dependencies, including torch and transformers :

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install transformers

Here is an example of how to use PDeepPP to process protein sequences and obtain predictions:

import torch
from transformers import AutoModel
from processing_pdeeppp import PDeepPPProcessor

# Load `PDeepPP` model
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using {device} device")
model = AutoModel.from_pretrained("fondress/PDeepPP_ACE", trust_remote_code=True)
model.to(device)

# Example protein sequences
protein_sequences = ["MKVSTYSTQ", "MSRSTYV"]

# Preprocess sequences
processor = PDeepPPProcessor(pad_char="X", target_length=33)
inputs = processor(sequences=protein_sequences, ptm_mode=True, return_tensors="pt")  # Set ptm_mode=True for PTM processing

# Make predictions
model.eval()
outputs = model(**inputs)
print(outputs["logits"])

Training and customization

PDeepPP supports fine-tuning on custom datasets. The model uses a configuration class ( PDeepPPConfig ) to specify hyperparameters such as:

• Number of transformer layers
• Hidden layer size
• Dropout rate
• PTM type and other task-specific parameters

Refer to PDeepPPConfig for details.

Citation

If you use PDeepPP in your research, please cite the associated paper or repository:

@article{your_reference,
  title={`PDeepPP`: A Hybrid Model for Protein Sequence Analysis},
  author={Author Name},
  journal={Journal Name},
  year={2025}
}