Résumé: Focus on the machine terminals with the only monocular camera and low computational power without GPU on the Mobile Terminal, it is difficult to realize the real-time dynamic hand gesture when multiple fingertips move in a small range because of the spatiotemporal noise. In this paper, the KOA-CLSTM-based real-time dynamic hand gesture recognition method is proposed for Mobile Terminals. The method consists of three parts: Kernel Optimize Accumulation (KOA), Union Frame Difference (UDF), and CLSTM. First, KOA is put forward to realize gesture extraction under special situations like parallel fingers, juxtaposed fingertips, natural syndactyly, and curved fingertips. Second, UDF facilitates frame feature fusion in both temporal and spatial dimensions, effectively resolving spatiotemporal noise such as inter-frame noise, camera dithering, and background interference on the Mobile Terminal. For the lightweight network CLSTM, which has a 5-block 3D convolutional neural network, it is better than C3D in a similar situation with a classification accuracy of 0.9689 in 0.0267 and an identification accuracy of 0.9682 in less than 0.13 s. It means that our proposed architecture on the Mobile Terminal performs well on fingertip classification and hand gestural recognition.

Langue: Anglais