Falconsai / ScopeNet

ScopeNet — woven architecture for airborne drone ID on Pi 5 + Hailo-8L

View in Model Surgeon

This card is generated from the surgical record itself; the package's lineage.intoto.jsonl is the signed source of truth (verify it free at the Surgeon's public verifier or with the bundled verify_attestation.py ).

Note on this revision. The sections Architecture through Compliance note are reproduced unchanged from the Model Surgeon record. Everything under Training record and Model details was added by the model owner from scopenet_trainer.ipynb and is not covered by the signed attestation.

Architecture

• Identification: Vision · Hybrid (98% confidence)
• Source format: pytorch · Intended task: not declared
• config.json : synthesized from the anatomy (no source config.json)
• Source license: not recorded (uploaded file)
• Lineage chain: 1 surgery (no prior attestation reachable)
• Post-surgery totals: 271,947 parameters · 274 tensors
• Compute estimate: 1.843853 GFLOPs (comparison metric, not a measurement)

What makes it a novel approach

1. Segment-Wise Re-ID Heads with Polarization (Leveraged in Falconsai/ScopeNet)

The Hybrid Head: The SegmentReID Head. Instead of projecting an object into one monolithic embedding space, this head splits the final embedding layer into distinct unit-norm segments (e.g., splitting a 96-d vector into three 32-d segments). Each segment is trained to independently match different physical traits: silhouette, kinematics (motion), and texture. It is supervised via a polarization repulsion loss to keep representation clusters strictly distinct. The Architecture: A Gated Mixture of "Scope Experts". The architecture processes the image using multiple independent convolutional stems (experts), each seeing the image at a different context ratio (e.g., 0.6x, 1.0x, 1.7x). A dense softmax gate dynamically mixes these experts based on the target's scale and sharpness. Current Approach: Edge-based vision systems currently use standard single-resolution feed-forward CNNs (like YOLO) coupled with basic Euclidean distance tracking. If an object rapidly changes scale or turns sideways, the holistic visual embedding changes entirely, and the system loses the track (an ID switch). The Novel Use Case: Real-time airborne drone pursuit and wildlife identification. Tracking erratic targets against noisy backgrounds where the object's aspect ratio changes violently. Why it is Substantially Better: If a drone turns sharply, its "silhouette" embedding segment changes drastically, but its "kinematics" and "texture" segments remain identical. The Re-ID head allows the model to maintain a continuous lock on the target despite physical transformations, drastically reducing ID switches without requiring a heavier model.

1. Woven Stateful/Stateless Distributed Architectures (The hardware philosophy behind ScopeNet)

The Hybrid Head/Constraint: An Edge-Distributed Tracking Graph. This isn't just a head, but a split of the model's brain based on hardware constraints. It uses a Motion Tokenizer to feed raw appearance data into the NPU, while utilizing an active Track Graph (using Graph Attention - GATv2) and a Wingbeat Spectral Branch on the CPU. The Architecture: A Woven Architecture. It resolves the tension between hardware limits and state-of-the-art physics tracking. It places the stateless feature-extraction layers (feed-forward CNN scope experts) onto an edge accelerator (like the Hailo-8L NPU), while weaving the stateful, memory-driven temporal tracking modules onto the host ARM CPU. Current Approach: Robotics engineers currently face a brutal tradeoff: either run lightweight, purely feed-forward models on edge NPUs (which drops accuracy by >10 mAP because they lack temporal memory), or try to run highly accurate recurrent/attention-based tracking models on weak CPUs (resulting in terrible frame rates and massive battery drain). The Novel Use Case: Micro-compute swarm robotics and edge ID. Fitting complex, high-accuracy multi-target tracking onto a low-power edge device like a Raspberry Pi 5. Why it is Substantially Better: By accepting the organizing constraint that NPUs are offline-compiled, stateless dataflow accelerators, this architecture achieves the "impossible." It honors the empirical superiority of recurrent temporal tracking (doing it on the CPU where tracks cost microseconds) while executing the heavy pixel-crunching appearance model at massive TOPS on the NPU. It gives heavy-compute accuracy on a micro-compute budget.

Target hardware: Raspberry Pi 5 with AI HAT+ (Hailo-8L, 13 TOPS), drone-mounted, real-time tracking and identification during variable-angle pursuit.

The organising constraint

Hailo-8L is a stateless dataflow accelerator with an offline compiler . It runs static-shape feed-forward CNNs well; it does not run recurrent layers, state-space models, spiking neurons, or dynamic shapes. But the strongest empirical result in the event-vision literature is that recurrent models beat sparse and feed-forward ones by more than 10 mAP.

The whole design follows from resolving that tension: stateless CNN on the NPU, stateful temporal model on the CPU. Per-frame appearance evidence is computed by the accelerator; everything with memory in it lives on the ARM cores, where a few dozen tracks cost microseconds.

Where each piece comes from

Provenance & operations

• Parents: scopenet.pt
• Operations performed: imaging×1, load×1
• Weight merges recorded: 0
• Quantized tensors (F32→F16): 0

Surgery Log (ordered)

1. load — scopenet.pt (1.2 MB, pytorch)
2. imaging — synthetic image 1x3x64x64 → [1, 2] [PASS]

Validation

• Tissue imaging: PASS
• Structural integrity is testable offline via the packaged load_and_test.py .

Compliance note

The signed attestation + this card together document model composition, modification history, and validation evidence — the record structure technical-documentation obligations (e.g. EU AI Act Annex IV) ask for. This is evidence, not legal advice.

---

Training record (owner-supplied, from scopenet_trainer.ipynb )

Summary

ScopeNet is a small (~0.26M learnable parameters) gated mixture of three convolutional "scope experts", each viewing the same image at a different context ratio, combined by a dense softmax gate. The architecture was designed for real-time airborne drone identification on a Raspberry Pi 5 with a Hailo-8L NPU, but this checkpoint was trained on a different task : binary image classification of normal vs sensitive images.

From the inference samples in the notebook, the sensitive class consists of technical and engineering imagery (mechanical drawings, exploded-view diagrams, floor plans, circuit schematics) and the normal class of everyday photographs and graphics. The owner should confirm this reading and state the intended use explicitly (see Intended use ).

Model details

The checkpoint also stores the architecture args, hyperparameters, preprocessing (class names, mean, std, resize, colour mode) and the validation metrics below. It can be loaded with load_scopenet() from the notebook.

The stage-B temporal modules defined in the notebook ( TrackGRU , TrackGATv2 , wingbeat_spectrum ) are not part of this checkpoint and were never trained.

Training data

• Folder-per-class dataset at dataset/images/train : 4,468 images, 2 classes.
• No separate validation folder was present, so the notebook made a per-class random split (15% validation, seed 0):
  • train 3,798 ( normal 1,712 / sensitive 2,086)
  • validation 670
• Of the training split, 35% (1,329 images) was held out for the gate ( V_hold ) and 65% (2,469) used to train the experts ( V_exp ). With all-wrong masking, 1,324 holdout images were used to fit the gate.
• Data source, collection method, licensing and labelling criteria are not recorded in the notebook.

Training procedure

• Stage A, experts: each expert trained independently on V_exp with AdamW (lr 3e-3, weight decay 0.05), 3-epoch linear warmup then cosine decay, up to 40 epochs, batch 64, label smoothing 0.1, gradient clipping at 5.0, EMA 0.999, early stopping with patience 12. Loss = cross-entropy + 0.1 × centroid polarization loss on the re-ID embedding. Augmentation: reflect-pad random shift (±8 px), horizontal flip, random erasing (p = 0.25).
• Stage A, gate: experts frozen; gate trained on the holdout features with AdamW (lr 1e-3), up to 300 epochs, patience 60.
• Environment: PyTorch 2.11.0+cu128 on CUDA, Python 3.14.4, Windows.
• Single seed (0).

Evaluation

Validation split, 670 images:

Mean gate weights by image sharpness quartile:

Other checks recorded in the notebook: save/reload agreement 1.0000; an 8-image spot check (4 per class) was 8/8 correct, but those images were drawn from the training set, so it is not evidence of generalization.

Limitations and caveats

• The gate does not earn its place on this task. Gated ScopeNet is 0.15 points below the best single expert, and its weights are almost identical on blurred and sharp images, so the scope-selection mechanism is not functioning. By the notebook's own criterion, a single expert would be the better deployment choice (same or better accuracy at one third of the NPU cost). At 670 validation images the difference is about one image and is within noise either way.

• Validation is optimistic. The split is random, not grouped by source. If the dataset contains near-duplicates (pages from the same document, crops of the same drawing), validation accuracy will overstate real performance. There is no held-out test set, and results come from a single seed.

• Wide scope uses reflect padding. The 1.7 context-ratio expert sees mirrored borders, not real surrounding content.

• Resolution mismatch in the Surgeon imaging test. The imaging check used a 64×64 input; the model was trained at 128×128. The PASS confirms the graph runs, not that 64×64 inputs are classified correctly.

• No edge artifact. ONNX export failed in the recorded run ( onnxscript not installed), so no Hailo-compiled or INT8-quantized version exists, and quantization loss has not been measured.

• Re-ID head is a weak proxy. It was trained with class labels standing in for track identities and has no meaningful use on this task.

How to load

net, preprocess, metrics = load_scopenet('scopenet.pt', dev='cpu')
# preprocess['classes'] -> ['normal', 'sensitive']
# resize to preprocess['resize'] (128), RGB, scale to [0,1],
# then (x - preprocess['mean']) / preprocess['std']
logits = net(x)            # x: [B, 3, 128, 128]

load_scopenet and the ScopeNet class definition are in scopenet_trainer.ipynb ; the checkpoint is not loadable without them.

---

Operated with Model Surgeon — verify this package at https://surgeon.falcons.ai/verify © 2026 FALCONS.AI — Model Surgeon record format. The model weights remain their owner's.