Vela Base is a multilingual encoder foundation for specialized routing models.
307M parameters · Input capacity: 32,768 tokens, including special tokens.
Evaluation
Masked-token negative log-likelihood on the same multilingual development set, compared with
the original mmBERT32K Base
. Lower is better.
Context length
Original mmBERT
Vela
512 tokens · 156 windows
1.441
1.387
8K tokens · 54 windows
1.226
1.212
16K tokens · 54 windows
1.206
1.168
32K tokens · 54 windows
1.075
1.018
The 318 windows cover English, Chinese, German, French, Japanese and Arabic. Both models use FP32, identical fixed 15% masked-token targets, and complete inputs without truncation. This development set informed Vela checkpoint selection; it is not an independent test set.
Quick start
With PyTorch and Transformers 4.57.6 or 5.17.0:
from transformers import pipeline
model_id = "llm-semantic-router/Vela-1.0-Encoder-307M"
model = pipeline("fill-mask", model=model_id, device=-1)
print(model(f"The capital of France is {model.tokenizer.mask_token}."))
Vela-1.0-Encoder-307M huggingface.co is an AI model on huggingface.co that provides Vela-1.0-Encoder-307M's model effect (), which can be used instantly with this llm-semantic-router Vela-1.0-Encoder-307M model. huggingface.co supports a free trial of the Vela-1.0-Encoder-307M model, and also provides paid use of the Vela-1.0-Encoder-307M. Support call Vela-1.0-Encoder-307M model through api, including Node.js, Python, http.
Vela-1.0-Encoder-307M huggingface.co is an online trial and call api platform, which integrates Vela-1.0-Encoder-307M's modeling effects, including api services, and provides a free online trial of Vela-1.0-Encoder-307M, you can try Vela-1.0-Encoder-307M online for free by clicking the link below.
llm-semantic-router Vela-1.0-Encoder-307M online free url in huggingface.co:
Vela-1.0-Encoder-307M is an open source model from GitHub that offers a free installation service, and any user can find Vela-1.0-Encoder-307M on GitHub to install. At the same time, huggingface.co provides the effect of Vela-1.0-Encoder-307M install, users can directly use Vela-1.0-Encoder-307M installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
Vela-1.0-Encoder-307M install url in huggingface.co: