ethicalabs / Echo-DSRN-v0.1.3-Embed-Exp

Echo-DSRN Embedding Model (Echo-DSRN-v0.1.3-Embed-Exp)

This is a high-performance experimental sentence embedding model based on the recurrent-hybrid Echo-DSRN architecture.

It scales linearly ($O(N)$) with sequence length, offering extreme efficiency and sub-millisecond latency on both CPU and GPU.

🚀 Model Details

• Developer: ethicalabs
• Base Model: ethicalabs/Echo-DSRN-114M-v0.1.2
• Architecture: Echo-DSRN Recurrent-Hybrid
• Parameters: 98.26M (98,264,064)
• Embedding Dimension: 2048

📊 Evaluation Results (MTEB STS)

Spearman Rank Correlation scores on Semantic Textual Similarity (STS) benchmark tasks:

⚡ Efficiency and Systems Scaling Profile

Inference performance (latency and peak VRAM allocation) on GPU and CPU configurations across different sequence lengths:

GPU Latency & VRAM Benchmark

CPU Latency Benchmark

Note: 'N/A (OOR)' indicates sequence length exceeds model's maximum position embedding range.

🏗️ Architecture Details

📊 Parameter Breakdown

💻 Usage

You can load and use this model directly via sentence-transformers :

from sentence_transformers import SentenceTransformer

# Load model with auto-mapping enabled
model = SentenceTransformer("ethicalabs/Echo-DSRN-v0.1.3-Embed-Exp", trust_remote_code=True)

# Encode text to get 2048-dimensional embeddings
sentences = ["The recurrent slow state contains the aligned sequence representations.", "Echo-DSRN has linear complexity."]
embeddings = model.encode(sentences)
print(embeddings.shape) # (2, 2048)

🛠️ Training Procedure

The model was trained in three sequential phases:

1. Contrastive Pre-training : Representation space alignment using natural language inference datasets.
2. Fine-grained Similarity Tuning : Fine-tuning using semantic textual similarity benchmarks to calibrate similarity scores.
3. Multi-Task Generalization Tuning : Training on NLI retrieval and STS semantic similarity.

This model card was automatically generated by scripts/generate_model_card.py .