WMSM: An Efficient Real-Time Framework for Hebrew Sign Language Recognition and Sentence-Level Translation

This paper introduces WMSM (Word Model, Sentence Mechanism), an innovative deep learning approach for real-time Hebrew Sign Language (HSL) recognition, optimized for mobile devices. WMSM features a novel word-level recognition model combined with a sentence-level prediction mechanism, offering an efficient alternative to traditional sequence-to-sequence (Seq2Seq) frameworks. Sign language recognition presents significant challenges, including limited dataset diversity, overfitting, and the difficulty of generalizing across different signers with varying signing styles. These issues are compounded by computational constraints, particularly when deploying models on mobile devices. To address these challenges, the paper details an augmentation pipeline that artificially expands the dataset, enhancing variability and enabling the model to generalize more effectively. Additionally, computational challenges such as packet loss and processing delays were mitigated through optimized model efficiency and the use of CPU-based workflows. The system demonstrates robust translation capabilities, producing coherent sentence-level outputs even under challenging conditions. This approach sets a new benchmark for efficient, high-performance real-time sign language recognition, addressing key challenges in dataset limitations, generalization, and mobile deployment.