Sponsored by AdsCreator.com
Toolify Toolify logo
AI Models
Qwen3-4B-LiteRT-gpu-composites

mlboydaisuke / Qwen3-4B-LiteRT-gpu-composites

Run on huggingface.co
huggingface.co
Total runs: 135
24-hour runs: 0
7-day runs: 3
30-day runs: 122
Model's Last Updated: September 22 2026
text-generation

Introduction of Qwen3-4B-LiteRT-gpu-composites

Model Details of Qwen3-4B-LiteRT-gpu-composites

Qwen3-4B — LiteRT-LM bundles exported with litert-torch's GPU composite flags

Test exports made for the LiteRT team's GPU-composite work (2026-09-10; re-exported 2026-09-11 with the team's fixed-prefill recipe, see Export). Not a release: on LiteRT-LM 0.17.0's macOS GPU paths the full-flag file does not produce correct output yet (see Status below). Weights int4 blockwise-32, activations fp32 ( dynamic_wi4b32_afp32 ).

Files
file flags bytes sha256
Qwen3-4B_dynamic_wi4b32_afp32_11flags.litertlm all eleven flags of the export command below 2,270,694,352 6edb82669203152517c92ec2506f1265af2b1071d01792277a37d7890804db8c
Qwen3-4B_dynamic_wi4b32_afp32_9flags.litertlm the same minus --use_sdpa_composite and --use_qkv_norm_rope_composite 2,270,290,496 c65779e9830ee8d1c93b72a9e66dce2420957f80814dce689074a20e7756adaa

Signatures: prefill_1024 (fixed) and decode ; cache 32771 (from --cache_length=32768 with --enable_gpu_dynamic_cache ; the exporter rounds GPU-dynamic sizes up to the next prime). Bundle metadata from LiteRT-LM's models/qwen3/LlmMetadataProto.pbtext ( max_num_tokens 4096, sampler TOP_P k=20 p=0.95 t=0.6, stop <|im_end|> ). The 2026-09-10 versions of these files ( prefill_1031 from --enable_gpu_dynamic_prefill , exporter-generated metadata with max_num_tokens 32771) are in this repo's history at commit 301fd8d1748ffbbd9f492cff86eb30defe0bd47c . Composites in the 11-flag graph: odml.qkv_norm_rope , odml.swiglu , odml.cache_update , odml.sdpa_transposed (decode), with fused QKV / gate-up projections and a bool mask.

Export

litert-torch upstream/main 6d4c622 (2026-09-08; use_swiglu_composite is not in the 0.9.4 wheel), litert-lm-builder 0.17.0 (main's builder needs it), ai-edge-litert 2.2.0, ai-edge-quantizer 0.9.0, torch 2.13.0, transformers 5.14.1. 

python -m litert_torch.generative.export_hf \
  --model=Qwen/Qwen3-4B --output_dir=out/p1024 \
  --quantization_recipe=dynamic_wi4b32_afp32 \
  --litert_lm_llm_metadata_override=<LiteRT-LM checkout>/models/qwen3/LlmMetadataProto.pbtext \
  --enable_gpu_dynamic_cache=True \
  --apply_gpu_composites=True --fuse_gate_up=True --fuse_qkv=True \
  --use_qkv_norm_rope_composite=True --use_swiglu_composite=True \
  --use_sdpa_composite=True --use_bool_mask=True \
  --bundle_litert_lm=True --prefill_lengths=1024 --cache_length=32768

The 9-flag file drops the two --use_*_composite flags named in the table.

Status on LiteRT-LM 0.17.0 (macOS, Mac Studio M4 Max)

The speeds below were measured 2026-09-10/11 on the previous versions of these files (dynamic prefill prefill_1031 , exporter-generated metadata; graphs otherwise identical). The current fixed-1024 files were re-checked 2026-09-11: 9-flag files 8/8 on the 8-question check, 11-flag failure modes unchanged, and the 0.6B A/B against its previous 9-flag file (three interleaved passes, same host) read the same speed within 3 %, and the 4B file's graph differs from its previous version in the same way.

file WebGPU path — PyPI litert-lm 0.17.0 wheel Metal path — litert_lm_main built from the v0.17.0 tag with --//runtime/executor:gpu_accelerator=metal , plus the release's prebuilt libLiteRtMetalAccelerator.dylib
11 flags does not run: odml.sdpa_transposed and odml.qkv_norm_rope fail WGSL validation, output is !! compiles, output wrong (blank); 72× GpuModelBuilder::UpdateOutputTensors(output of qkv_norm_rope) shape mismatch ({1,32,1024,1,128} vs {1,16,1024,1,128}) plus 36× the same for flash_decode_sdpa
9 flags decode 117 tok/s, prefill 1.41k tok/s ( litert-lm benchmark --backend gpu -p 1024 -d 128 --cache no , medians of 3 processes); 8-question sanity check 8/8 decode 103 tok/s, prefill 1.4k (996-token prompt, context 4096, decode to EOS, medians of 3); 8/8

Android and iOS: not measured.

Run (macOS) 
pip install litert-lm==0.17.0
litert-lm run Qwen3-4B_dynamic_wi4b32_afp32_9flags.litertlm --backend gpu --thinking false --cache no --prompt "What is the capital of Japan? Answer briefly."
litert-lm benchmark Qwen3-4B_dynamic_wi4b32_afp32_9flags.litertlm --backend gpu -p 1024 -d 128 --cache no

Runs of mlboydaisuke Qwen3-4B-LiteRT-gpu-composites on huggingface.co

135
Total runs
0
24-hour runs
0
3-day runs
3
7-day runs
122
30-day runs

More Information About Qwen3-4B-LiteRT-gpu-composites huggingface.co Model

More Qwen3-4B-LiteRT-gpu-composites license Visit here:

https://choosealicense.com/licenses/apache-2.0

Qwen3-4B-LiteRT-gpu-composites huggingface.co

Qwen3-4B-LiteRT-gpu-composites huggingface.co is an AI model on huggingface.co that provides Qwen3-4B-LiteRT-gpu-composites's model effect (), which can be used instantly with this mlboydaisuke Qwen3-4B-LiteRT-gpu-composites model. huggingface.co supports a free trial of the Qwen3-4B-LiteRT-gpu-composites model, and also provides paid use of the Qwen3-4B-LiteRT-gpu-composites. Support call Qwen3-4B-LiteRT-gpu-composites model through api, including Node.js, Python, http.

Qwen3-4B-LiteRT-gpu-composites huggingface.co Url

https://huggingface.co/mlboydaisuke/Qwen3-4B-LiteRT-gpu-composites

mlboydaisuke Qwen3-4B-LiteRT-gpu-composites online free

Qwen3-4B-LiteRT-gpu-composites huggingface.co is an online trial and call api platform, which integrates Qwen3-4B-LiteRT-gpu-composites's modeling effects, including api services, and provides a free online trial of Qwen3-4B-LiteRT-gpu-composites, you can try Qwen3-4B-LiteRT-gpu-composites online for free by clicking the link below.

mlboydaisuke Qwen3-4B-LiteRT-gpu-composites online free url in huggingface.co:

https://huggingface.co/mlboydaisuke/Qwen3-4B-LiteRT-gpu-composites

Qwen3-4B-LiteRT-gpu-composites install

Qwen3-4B-LiteRT-gpu-composites is an open source model from GitHub that offers a free installation service, and any user can find Qwen3-4B-LiteRT-gpu-composites on GitHub to install. At the same time, huggingface.co provides the effect of Qwen3-4B-LiteRT-gpu-composites install, users can directly use Qwen3-4B-LiteRT-gpu-composites installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Qwen3-4B-LiteRT-gpu-composites install url in huggingface.co:

https://huggingface.co/mlboydaisuke/Qwen3-4B-LiteRT-gpu-composites

Url of Qwen3-4B-LiteRT-gpu-composites

Qwen3-4B-LiteRT-gpu-composites huggingface.co Url

Provider of Qwen3-4B-LiteRT-gpu-composites huggingface.co

mlboydaisuke
ORGANIZATIONS
huggingface.co

Other API from mlboydaisuke