CAJ | 학술논문

在传统的视听双模态语音识别系统的研究中，经图像处理后的视觉特征往往具有数据量大、重要特征丢失等问题。针对这些问题，拟采用图像可听化技术对视频图像进行特征提取。以遗传算法优化的BP神经网络为融合模型，对视频、音频特征进行特征级融合。实验结果表明，经过图像可听化处理后视觉特征包含了一定的语音信息，在噪声环境下的识别效果比较稳定，神经网络的融合模型提高了系统的鲁棒性。
재전통적시은쌍모태어음식별계통적연구중，경도상처리후적시각특정왕왕구유수거량대、중요특정주실등문제。침대저사문제，의채용도상가은화기술대시빈도상진행특정제취。이유전산법우화적BP신경망락위융합모형，대시빈、음빈특정진행특정급융합。실험결과표명，경과도상가은화처리후시각특정포함료일정적어음신식，재조성배경하적식별효과비교은정，신경망락적융합모형제고료계통적로봉성。
While studying the traditional speech recognition system with audio-video dual mode , we found that the visual characteristics after image processing have the problems of large amount of data and important characteristics lost .Aiming at these problems , we plan to apply image sonification technology to extracting the characteristics of video image .By using BP neural network in genetic algorithm optimisation as the fusion model , we fuse the characteristics of audio and video at feature level .Experimental results show that , after being processed by the image sonification , the visual characteristics contain certain speech information , its recognition effect is stable in noise environment as well .The fusion model of neural network improves the robustness of the system .