CAJ | 학술논문

针对目前音乐专业学生钢琴即兴伴奏的应用能力较低的现状，提出一种结合MIDI键盘与Kinect深度传感器的方法来达到识别和弦进行与判断学生演奏时指法的目的，并实现该辅助教学系统。首先，获取彩色和深度图像的信息，构建和弦转位的状态转移图和和弦手势模板库，作为系统初始化条件。其次，利用传统的肤色建模和背景差分方法以及现在的深度数据，综合手势的表观特征，通过模板匹配实现手势识别。最后，判断和弦进行的正确性，并综合指法应用进行打分与评价。实验结果表明，该系统具有较高的鲁棒性，能够有效地应用于钢琴教学。
침대목전음악전업학생강금즉흥반주적응용능력교저적현상，제출일충결합MIDI건반여Kinect심도전감기적방법래체도식별화현진행여판단학생연주시지법적목적，병실현해보조교학계통。수선，획취채색화심도도상적신식，구건화현전위적상태전이도화화현수세모판고，작위계통초시화조건。기차，이용전통적부색건모화배경차분방법이급현재적심도수거，종합수세적표관특정，통과모판필배실현수세식별。최후，판단화현진행적정학성，병종합지법응용진행타분여평개。실험결과표명，해계통구유교고적로봉성，능구유효지응용우강금교학。
For the status quo of current students majoring music having low capability in impromptu piano accompaniment, we presented a method combining the MIDI keyboard with Kinect depth sensor to achieve the goals of recognising the chord progression and judging the fingering while playing, and to realise the assisted teaching system.First, the method gets the colour and deep image information, builds the state transition diagrams of chords inversion and chords gesture templates library, these are used as the initialising condition of the system. Secondly, it utilises traditional skin colour modelling, background subtraction method and current depth data, and integrates the appearance features of gesture to achieve gesture recognition through template matching.Finally, the system determines the correctness of chord progression, and marks and evaluates by integrating the figuring applications.Experimental results show that the system is highly robust and can be effectively applied to piano teaching.