software-mansion / react-native-executorch-pp-doclayout-v3

Introduction

This repository hosts PaddleOCR PP-DocLayoutV3 , an RT-DETR-based document layout detector (~33M params), for the React Native ExecuTorch library, exported to .pte for the ExecuTorch runtime (XNNPACK, CoreML, Vulkan). It finds and classifies document regions — titles, paragraphs, tables, figures, formulas, headers/footers, etc. — and is a companion to react-native-executorch-paddleocr .

If you'd like to run these models in your own ExecuTorch runtime, refer to the official documentation for setup instructions.

The .pte is a pure tensor→tensor function; all pre/post-processing (resize, normalize, score threshold, box convert) is the client's job.

Output contract

A single static method forward , fixed input (no buckets):

in   [1, 3, 800, 800]          # RGB, ImageNet-normalized (x/255 - mean)/std
out  logits     [1, 300, 25]   # 25 layout classes, per query (apply sigmoid)
     pred_boxes [1, 300, 4]    # (cx, cy, w, h), normalized [0,1]

PP-DocLayoutV3 is a DETR set-prediction model → no NMS . Post-processing is just: score = sigmoid(logits) , keep queries above a threshold, convert (cx,cy,w,h) → (x1,y1,x2,y2) , scale to image size. Class names are in labels.json (index → label).

Classes (25)

abstract, algorithm, aside_text, chart, content, formula, doc_title, figure_title, footer, footnote, formula_number, header, image, number, paragraph_title, reference, reference_content, seal, table, text, vision_footnote (some indices map to the same display label; use labels.json as the authoritative index→label map).

Backends, sizes & latency (warm)

Vulkan is the recommended Android backend — ~2.4× faster than XNNPACK and half the size. It's mixed-delegate: most of RT-DETR runs fp16 on the GPU, while the box-head matmuls run on XNNPACK (they delegate as addmm → linear ). XNNPACK stays fp32 because RT-DETR's deformable attention feeds non-contiguous tensors that int8/portable paths mis-handle.

Compatibility

If you intend to use these models outside of React Native ExecuTorch, make sure your runtime is compatible with the ExecuTorch version used to export the .pte files. For more details, see the compatibility note in the ExecuTorch GitHub repository . If you work with React Native ExecuTorch, the library constants guarantee compatibility with the runtime used behind the scenes.