Arko007 / yoga-posture-models

Smart Yoga Posture Correction System (Project P05)

This repository hosts the model weights and label encoders for the Smart Yoga Posture Correction System (Final Year Project P05, RCC IIT Kolkata).

The system leverages a dual-model framework to classify and correct yoga poses:

1. ResMLP Model ( mlp_model.pth ) : A frame-level static posture classifier trained on 15 biomechanical joint angles, achieving 92.84% validation accuracy across 29 classes.
2. Sequence Flow Model ( stgcn_sequence_model.pth ) : A hybrid 1D Temporal Convolution + Stacked Residual GRU + Self-Attention model trained on 60-frame skeleton coordinate sequences, achieving 75.25% validation accuracy across 27 classes.

Both models incorporate class-weight smoothing and normalization techniques to resolve pose imbalance and coordinate noise.

---

Model Architectures & Training Logs

1. Static Pose Classifier (ResMLP)

Architecture

The ResMLP classifier processes 15 frame-level joint angles (computed from MediaPipe Pose landmarks):

• Input Layer : Linear(15 -> 256) followed by Batch Normalization and GELU activation.
• Residual blocks : 2 stacked residual blocks. Each block consists of:
  • Linear(256 -> 256) -> BatchNorm1d -> GELU -> Dropout(0.3)
  • Linear(256 -> 256) -> BatchNorm1d -> GELU -> Dropout(0.3)
  • Residual skip connection: x_out = x + block(x)
• Classification Head : Linear(256 -> 128) -> BatchNorm1d -> GELU -> Dropout(0.2) -> Linear(128 -> 29) .

Dataset & Preprocessing

• Dataset size : 654,488 frames in total.
  • Train size : 523,590 frames
  • Validation size : 130,898 frames
• Class Weights : Smoothed using the square-root count inverse function 1.0 / sqrt(count) to prevent major classes (such as transition/unknown and lunge_pose ) from dominating the gradients.

Training Performance & Curves

• Best Validation Loss : 0.1644 at Epoch 39.
• Final Epoch (40/40) :
  • Train Loss : 0.2238 | Train Acc : 90.78%
  • Val Loss : 0.1651 | Val Acc : 92.84%

Below is the training progress for selected epochs:

Static Pose Classification Report

                          precision    recall  f1-score   support

              chair_pose       0.56      0.94      0.70       366
              chaturanga       0.45      1.00      0.62         5
                   child       0.06      0.57      0.10         7
              child_pose       0.91      0.99      0.95      3260
              cobra_pose       0.90      0.96      0.93      5116
                  corpse       0.36      0.85      0.51        20
            downward_dog       0.90      0.95      0.92      4398
            halfway_lift       0.55      0.94      0.70       479
        imperfect_corpse       0.66      0.97      0.78       290
         imperfect_plank       0.86      0.96      0.91      1825
imperfect_seated_forward       0.87      0.99      0.92       938
      imperfect_triangle       0.87      0.96      0.91      2607
    imperfect_upward_dog       0.91      0.97      0.94      2556
              lunge_pose       0.97      0.93      0.95     19496
           mountain_pose       0.77      0.97      0.86      1233
                   plank       0.58      0.63      0.61       174
        seated_easy_pose       0.94      0.97      0.95     17465
          seated_forward       0.91      0.96      0.94        75
            seated_staff       0.80      0.94      0.86      1600
   standing_forward_fold       0.95      0.96      0.96      7907
           standing_pose       0.85      0.92      0.89      1405
               table_top       0.51      0.94      0.66       501
      transition/unknown       0.98      0.88      0.93     44781
               tree_pose       0.73      0.97      0.83      1474
                triangle       0.58      0.75      0.66       485
              upward_dog       0.42      0.60      0.49        67
           upward_salute       0.76      0.99      0.86       528
               warrior_1       0.94      0.98      0.96      4736
               warrior_2       0.88      0.95      0.91      7104

            weighted avg       0.94      0.93      0.93    130898
                accuracy                           0.93    130898

---

2. Sequence Flow Classifier (Hybrid Conv1D + Residual GRU + Attention)

Architecture

The sequence classifier processes 60-frame coordinate sequences (shape [batch_size, 60, 99] , representing 33 joints in 3D):

• Coordinate Normalization : Translates coordinate sequences to be pelvis-centered (using the midpoint between the left and right hip joints) and divides by hip-width. This guarantees absolute translation and scale invariance .
• 1D Temporal Convolution : Conv1d(in_channels=99, out_channels=128, kernel_size=5, padding=2) -> BatchNorm1d -> GELU -> Dropout(0.2) to smooth coordinate sequence noise.
• Stacked Residual GRU blocks : Two bidirectional GRU blocks with hidden dimension 128. Output is projected back from 256 to 128, normalized with LayerNorm, dropped out with 30% rate, and summed with input (residual connection).
• Self-Attention Pooling : Learns step importance weights dynamically and returns a weighted summary vector across the 60-frame window.
• Classification Head : Linear(128 -> 64) -> GELU -> Dropout(0.3) -> Linear(64 -> 27) .

Dataset & Preprocessing

• Total sequences : 18,165 (60-frame windows).
  • Train size : 14,532 sequences.
  • Validation size : 3,633 sequences.
• Training Hyperparameters :
  • Batch Size: 64
  • Optimizer: AdamW(lr=2e-3, weight_decay=1e-3)
  • LR Scheduler: CosineAnnealingLR (down to 1e-5 minimum LR)
  • Loss Function: CrossEntropy Loss with smoothed class weights and 0.1 label smoothing to handle majority-voted sequence label noise.
  • Checkpoint Metric: Monitored validation accuracy.
  • Early Stopping: Patience of 20 epochs based on Best Val Acc.

Training Performance & Curves

• Best Validation Accuracy : 75.25% at Epoch 90.
• Early Stopping : Triggered at Epoch 110.

Selected epochs during training:

Sequence Flow Classification Report

                          precision    recall  f1-score   support

              chair_pose       0.57      0.89      0.70         9
              child_pose       0.69      0.97      0.80        89
              cobra_pose       0.71      0.76      0.74       139
                  corpse       0.00      0.00      0.00         1
            downward_dog       0.67      0.72      0.70       122
            halfway_lift       0.38      0.67      0.48         9
        imperfect_corpse       0.50      0.86      0.63         7
         imperfect_plank       0.71      0.65      0.68        49
imperfect_seated_forward       0.70      0.84      0.76        25
      imperfect_triangle       0.81      0.81      0.81        70
    imperfect_upward_dog       0.54      0.94      0.69        70
              lunge_pose       0.75      0.66      0.70       549
           mountain_pose       0.45      0.52      0.48        33
                   plank       0.25      0.60      0.35         5
        seated_easy_pose       0.90      0.88      0.89       486
          seated_forward       0.40      1.00      0.57         2
            seated_staff       0.49      0.47      0.48        40
   standing_forward_fold       0.73      0.74      0.74       227
           standing_pose       0.39      0.70      0.50        37
               table_top       0.56      0.56      0.56         9
      transition/unknown       0.86      0.73      0.79      1270
               tree_pose       0.42      0.75      0.53        36
                triangle       0.25      0.29      0.27        14
              upward_dog       0.00      0.00      0.00         2
           upward_salute       0.65      0.87      0.74        15
               warrior_1       0.63      0.80      0.70       131
               warrior_2       0.71      0.79      0.74       187

            weighted avg       0.77      0.75      0.76      3633
                accuracy                           0.75      3633

---

Inference and Usage Guide

Both model state dicts and label encoder maps can be downloaded and loaded in Python as follows:

import numpy as np
import torch
import torch.nn as nn

# Load label encoders
mlp_classes = np.load("mlp_label_encoder.npy", allow_pickle=True)
stgcn_classes = np.load("stgcn_label_encoder.npy", allow_pickle=True)

# 1. Instantiate the ResMLP Model
# (Refer to the model definition above)
mlp_model = YogaMLP(input_dim=15, num_classes=len(mlp_classes))
mlp_model.load_state_dict(torch.load("mlp_model.pth", map_location="cpu"))
mlp_model.eval()

# 2. Instantiate the Sequence Model
# (Refer to the model definition above)
sequence_model = YogaSequenceLSTM(input_dim=99, hidden_dim=128, num_layers=2, num_classes=len(stgcn_classes))
sequence_model.load_state_dict(torch.load("stgcn_sequence_model.pth", map_location="cpu"))
sequence_model.eval()

Cooperative Prediction Protocol

For production deployment (e.g. FastAPI backend):

1. Extract frame joint coordinate sequences (shape [N, 60, 99] ) using MediaPipe.
2. If the current sequence is classified by stgcn_sequence_model.pth as transition/unknown , the backend falls back to using the static mlp_model.pth classifier on individual frames.
3. This cooperative approach minimizes false positives, provides real-time latency optimization, and ensures smooth transition tracking while practicing.

---

RCC Institute of Information Technology, Kolkata
Department of Computer Science & Engineering
Final Year Project 2026