DJLougen / mpknet-v6.2-temporal

MPKNet V6.2 Temporal - Bio-Inspired Video Classification

An extension of MPKNet V6 that adds temporal processing to the M (Magnocellular) pathway for video understanding and action recognition. The M pathway processes 8 consecutive frames and computes inter-frame deltas to capture motion, while the P pathway sees only the current frame for spatial detail.

Architecture

Built on the three-pathway design with Fibonacci strides (2:3:5):

• P pathway (stride 2): Current frame only - fine spatial detail
• K pathway (stride 3): Current frame only - context and gating signals
• M pathway (stride 5): 8 consecutive frames - computes 7 inter-frame deltas for motion

The M pathway uses shared Conv2D weights across all frames, computing learned deltas between consecutive frame pairs. A temporal fusion module combines all 7 deltas into a single motion representation.

For static images, the model generates pseudo-frames via progressive scale and blur augmentation, teaching M to detect change even without real motion. This transfers to real video at inference.

Results

No pretraining. 0.58M parameters processing 8-frame sequences.

Usage

import torch
from mpknet_v6_2_temporal import BinocularMPKNetV6_2
from mpknet_components import count_params

model = BinocularMPKNetV6_2(num_classes=101, ch=48, use_stereo=True, num_frames=8)
state_dict = torch.load("mpknet_v6_2_ucf101_best.pt", map_location="cpu", weights_only=True)
model.load_state_dict(state_dict)
model.eval()

# Inference with video frames
current_frame = torch.randn(1, 3, 224, 224)
frame_sequence = torch.randn(1, 8, 3, 224, 224)

with torch.no_grad():
    logits = model(current_frame, frames=frame_sequence)
    pred = logits.argmax(dim=1)

# Or with a static image (auto-generates pseudo-frames)
with torch.no_grad():
    logits = model(current_frame)

Files

• mpknet_v6_2_ucf101_best.pt - Trained weights (UCF-101, 101 classes, 7.3MB)
• mpknet_v6_2_temporal.py - Model architecture with SequentialTemporalMPathway
• mpknet_components.py - Shared components

Citation

D.J. Lougen, "MPKNet: Bio-Inspired Visual Classification with Parallel LGN Pathways", 2025.

License

PolyForm Small Business License 1.0.0 - Free for organizations with less than 100K revenue, non-profits, and education.

Links

• GitHub