Lite-SignVLM: An Efficiency-Oriented Framework for Sign Language Recognition with Temporal Storyboards
Sign language recognition (SLR) systems have made substantial progress in recent years; however, practical deployment remains challenging in real-world scenarios that require low latency, offline operation, and execution on resource-constrained devices. Although large transformer-based and visionlan...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , , |
| Format: | article |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/11073/33469 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Sign language recognition (SLR) systems have made substantial progress in recent years; however, practical deployment remains challenging in real-world scenarios that require low latency, offline operation, and execution on resource-constrained devices. Although large transformer-based and visionlanguage models achieve strong recognition accuracy, their reliance on computationally expensive backbones leads to high memory usage and inference cost. Additionally, critical efficiency metrics such as model size, GFLOPs, and inference latency are often unreported. In this work, we address these limitations by proposing a comprehensive, efficiency-oriented compression framework for SLR. Starting from a high-accuracy teacher model such as SignVLM, we distill knowledge into lightweight student models using three complementary strategies. First, we introduce a storyboard-based video representation that encodes the full temporal progression of a sign into a single structured RGB image. Second, we perform systematic architectural distillation by comparing CLIP and EfficientNet backbones under different decoder retention rates. Third, we conduct a detailed study of post-training FP16 casting and INT8 quantization. Experiments on the KArSL502 dataset under a signer-dependent evaluation protocol show that preserving full temporal input motion enables student models to match or exceed teacher accuracy despite significantly reduced complexity. Our best configuration achieves 99.34% Top-1 accuracy with only 5.38M parameters, 15.45 MB disk size, and 0.43 GFLOPs. While FP16 casting consistently preserves accuracy across architectures, INT8 quantization exhibits architecture-dependent behavior in which models with EfficientNet-based backbones maintain high recognition accuracy, whereas models with CLIP-based backbones suffer from severe recognition degradation due to outlier activation effects. Overall, this work provides a clear and deployable pathway for on-device SLR systems, bridging the gap between high recognition performance and practical efficiency constraints. |
|---|