Accio-Lab / occamy-1.0-FP8

Occamy 1.0 FP8

Blockwise FP8 conversion of Accio-Lab/occamy-1.0 . The tokenizer and vision weights are preserved. MTP is available separately.

In the paired evaluation below, FP8 increased token NLL by 0.3378% . Sampled mathematics accuracy was 62/64 versus 63/64 for BF16; science, code, JSON, multi-turn, long-input and tool-roundtrip scores matched the baseline. See the complete comparison below and the validation notes for per-case details.

Format

• FP8 E4M3FN weights, 128 × 128 blocks, FP32 inverse scales, dynamic activation quantization.
• 14 safetensors shards, 36.6 GB total. File size is not a runtime VRAM requirement.
• Conversion: Transformers 5.8.1 Fp8Quantize , PyTorch 2.11.0+cu130. No calibration data or training was used.
• Tensor selection follows the format metadata of Qwen/Qwen3.6-35B-A3B-FP8 . None of its weights are included.

Evaluation

SGLang 0.5.13.post1, PyTorch 2.11.0+cu130, FlashInfer 0.6.12. Identical prompts, greedy decoding, seed 42, thinking disabled, four concurrent requests, 16K configured context. Dataset revisions and fixed sample indices are in evaluation/provenance.json ; fixtures and raw scored responses are included.

FP8 NLL increased 0.3378% ; the perplexity ratio is 1.00748 . All scored log probabilities were finite, with matched token counts. No OOM or CUDA errors were observed.

These are sampled regression tests, not full benchmark scores. Per-case results, repeat checks, scoring details and coverage are in VALIDATION-NOTES.md and evaluation/ . This release does not include MTP or claim a measured throughput speedup.

Tested serving command

SGLANG_JIT_DEEPGEMM_PRECOMPILE=0 python -m sglang.launch_server \
  --model-path Accio-Lab/occamy-1.0-FP8 --served-model-name occamy \
  --host 127.0.0.1 --port 18400 --dtype bfloat16 \
  --context-length 16384 --max-running-requests 4 \
  --max-total-tokens 65536 --max-mamba-cache-size 16 \
  --mem-fraction-static 0.60 --attention-backend fa3 \
  --mm-attention-backend fa3 --mamba-backend triton \
  --disable-cuda-graph --random-seed 42 \
  --tool-call-parser qwen3_coder --enable-metrics

FP8 is detected from config.json ; --dtype bfloat16 applies to preserved weights. Memory settings depend on hardware. The original qwen tool parser does not parse this checkpoint's XML tool format correctly; the tested parser is qwen3_coder .

With the repository downloaded and the server ready:

python evaluation/evaluate.py http://127.0.0.1:18400 sglang-fp8
python evaluation/evaluate_nll.py http://127.0.0.1:18400 sglang-fp8

Reproduction settings and per-case results are in Technical details and evaluation/ .

vLLM compatibility

On a single H200, this checkpoint completed all 9 requests and passed all 9 semantic checks, covering text, code, strict JSON, tool calls and result follow-up, and images. The BF16 reference also passed 9/9. There were no request errors or truncated responses. MTP was disabled. This is a small compatibility check, not a broad quality or speed benchmark.

Attribution and license

Original model: Accio-Lab/Occamy-1.0, Apache-2.0; original license included in LICENSE . Format metadata reference: Qwen/Qwen3.6-35B-A3B-FP8. Evaluation datasets retain their own licenses and attribution; see evaluation/DATA-LICENSES.md .