| 摘要: |
| 目的 为提高连续手语识别准确率，缓解听障人群与非听障人群的沟通障碍。方法 提出了基于全局注意力机制和LSTM的连续手语识别算法。通过帧间差分法对视频数据进行预处理，消除视频冗余帧，借助ResNet网络提取特征序列。通过注意力机制加权，获得全局手语状态特征，并利用LSTM进行时序分析，形成一种基于全局注意力机制和LSTM的连续手语识别算法，实现连续手语识别。结果 实验结果表明，该算法在中文连续手语数据集CSL上的平均识别率为90.08%，平均词错误率为41.2%，与5种算法相比，该方法在识别准确率与翻译性能上具有优势。结论 基于全局注意力机制和LSTM的连续手语识别算法实现了连续手语识别，并且具有较好的识别效果及翻译性能，对促进听障人群无障碍融入社会方面具有积极的意义。 |
| 关键词: 手语识别 特征提取 全局注意力机制 LSTM |
| DOI：10.19554/j.cnki.1001-3563.2022.08.004 |
| 分类号:TP18;TB472 |
| 基金项目:国家自然科学基金(62163007);贵州省科技计划项目(黔科合平台人才[2020]6007，黔科合支撑[2021]一般439，JXCX[2021]001) |
|
| Continuous Sign Language Recognition Algorithm Based on Global Attention Mechanism and LSTM |
|
YANG Guan-ci1,2,3, HAN Hai-fenga1, LIU Sai-sai2, JIANG Ya-wen2, LI Yang2
|
|
(1. School of Mechanical Engineering , Guiyang 550025, China;2. Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education , Guiyang 550025, China;3. State Key Laboratory of Public Big Data, Guizhou University, Guiyang 550025, China)
|
| Abstract: |
| To improve the continuous sign language recognition accuracy and alleviate the communication barrier between hearing-impaired people and non hearing-impaired people, this paper proposed the continuous sign language recognition algorithm based on Global Attention Mechanism and LSTM (CSLR-GAML). The video data is preprocessed by applying inter-frame difference to eliminate redundant video frames, and then the feature sequences of the key frames are extracted by using ResNet. After that, the attention mechanism is used to update the network parameters, which is capable of obtain the global feature of sign language, and then the LSTM is employed to finish the timing sequence analysis. Finally, use the Chinese continuous sign language data set CSL to check algorithm performance. And the experimental results show that the average recognition accuracy of the proposed algorithm is 90.08%, and the average word error rate is 41.2%. Compared CSLR-GAML with other Five algorithms, the proposed CSLR-GAML has advantages in recognition accuracy and translation performance. The sontinuous sign language recognition algorithm based on Global Attention Mechanism and LSTM realizes continuous sign language recognition, and has good recognition effect and translation performance. It is of positive significance to promote the barrier free integration of hearing-impaired people into society. |
| Key words: sign language recognition feature extraction global attention mechanism LSTM |
渝公网安备 50010702501716号