litert-community / DM-Count-Crowd-LiteRT

DM-Count — Crowd counting (LiteRT GPU)

On-device crowd counting running fully on the LiteRT CompiledModel GPU delegate (no CPU fallback). DM-Count (NeurIPS 2020) regresses a person density map whose sum is the crowd size — it counts hundreds of people where detector-based counting saturates.

• Architecture: VGG19 backbone + conv regression head — pure CNN.
• Weights: cvlab-stonybrook/DM-Count (UCF-QNRF) · MIT.
• Size: 86 MB.

Input (left) → density heatmap + count (right). Photo: Pexels (free license).

I/O

• Input: [1, 3, 512, 512] NCHW, RGB, ImageNet-normalized (mean [0.485,0.456,0.406] , std [0.229,0.224,0.225] ).
• Output: [1, 1, 64, 64] non-negative density map — sum(map) = estimated person count; normalize per frame for a heatmap overlay.

GPU conversion

DM-Count is a pure CNN (VGG19 + conv head). It converts fully GPU-compatible ( 30/30 nodes on the delegate, 1 partition ; Pixel 8a corr 0.9998–1.0 and count within 0.4% of PyTorch on real crowd images, ~79 ms/frame) with one exact rewrite : the mid-graph F.upsample_bilinear (align_corners=True RESIZE_BILINEAR , banned on the delegate) is a linear operator, re-authored as two constant-matrix multiplies — with the constant on the RHS (lowers to FULLY_CONNECTED ; the delegate rejects BATCH_MATMUL with a constant LHS). Desktop corr vs PyTorch is 1.000000 with an identical count.

Minimal usage

Kotlin (Android, LiteRT CompiledModel GPU)

val options = CompiledModel.Options(Accelerator.GPU)
val model = CompiledModel.create(context.assets, "dmcount.tflite", options, null)
val inBufs = model.createInputBuffers()
val outBufs = model.createOutputBuffers()

inBufs[0].writeFloat(inputNCHW)          // [1,3,512,512] RGB, ImageNet-norm
model.run(inBufs, outBufs)
val density = outBufs[0].readFloat()     // [64*64] density map
val count = density.sum()                // estimated number of people

Python (LiteRT CompiledModel API)

import numpy as np
from ai_edge_litert.compiled_model import CompiledModel

model = CompiledModel.from_file("dmcount.tflite")
inputs = model.create_input_buffers(0)
outputs = model.create_output_buffers(0)
inputs[0].write(np.ascontiguousarray(x, np.float32))  # [1,3,512,512] RGB, ImageNet-norm
model.run_by_index(0, inputs, outputs)
n = model.get_output_buffer_requirements(0, 0)["buffer_size"] // 4
density = outputs[0].read(n, np.float32).reshape(64, 64)
count = float(density.sum())

Conversion

Converted with litert-torch ( build_dmcount.py ): loads the MIT DM-Count (UCF-QNRF) weights and exports the raw density map. The UCF-QNRF checkpoint generalizes best across scenes; the upstream repo also bundles an NWPU-Crowd variant.

License

MIT (DM-Count / cvlab-stonybrook). Trained on UCF-QNRF.