心理学报
心理學報
심이학보
Acta Psychologica Sinica
2015年
5期
679~688
,共null页
罗照盛 喻晓锋 高椿雷 李喻骏 彭亚风 王睿 王钰彤
囉照盛 喻曉鋒 高椿雷 李喻駿 彭亞風 王睿 王鈺彤
라조성 유효봉 고춘뢰 리유준 팽아풍 왕예 왕옥동
认知诊断计算机化自适应测验 选题策略 属性掌握概率 属性掌握模式
認知診斷計算機化自適應測驗 選題策略 屬性掌握概率 屬性掌握模式
인지진단계산궤화자괄응측험 선제책략 속성장악개솔 속성장악모식
text CD-CAT; item selection strategies; attribute mastery probabilities; attribute mastery pattern
在认知诊断计算机化自适应测验(CD-CAT)中,被试对每个属性的掌握概率更直接地反映了被试能力的当前估计值。因此,基于被试的属性掌握概率来构建选题策略,选择最能改变被试属性掌握概率的题目作为下一个测验项目,这应该是一个值得尝试的方案。本文借鉴已有相关研究的数据生成模式进行探索,模拟实验结果表明:假设属性间相互独立,在定长(长度为16)、变长(长度为16或后验属性掌握模式概率达到0.8)以及短测验(长度分别为4、6、8、10)的情况下,基于属性掌握概率的选题策略PPWKL和PHKL有较好的分类准确率,在题目曝光率,题库使用均匀性等方面也有较好的表现;与研究较多的PWKL、HKL等策略相比,也略有优势;当属性间存在不同程度的相关时,在定长、变长以及较短的测验条件下,基于PHKL和MI的测验对知识状态估计精度较好,基于PPWKL和PHKL的测验综合表现占优。
在認知診斷計算機化自適應測驗(CD-CAT)中,被試對每箇屬性的掌握概率更直接地反映瞭被試能力的噹前估計值。因此,基于被試的屬性掌握概率來構建選題策略,選擇最能改變被試屬性掌握概率的題目作為下一箇測驗項目,這應該是一箇值得嘗試的方案。本文藉鑒已有相關研究的數據生成模式進行探索,模擬實驗結果錶明:假設屬性間相互獨立,在定長(長度為16)、變長(長度為16或後驗屬性掌握模式概率達到0.8)以及短測驗(長度分彆為4、6、8、10)的情況下,基于屬性掌握概率的選題策略PPWKL和PHKL有較好的分類準確率,在題目曝光率,題庫使用均勻性等方麵也有較好的錶現;與研究較多的PWKL、HKL等策略相比,也略有優勢;噹屬性間存在不同程度的相關時,在定長、變長以及較短的測驗條件下,基于PHKL和MI的測驗對知識狀態估計精度較好,基于PPWKL和PHKL的測驗綜閤錶現佔優。
재인지진단계산궤화자괄응측험(CD-CAT)중,피시대매개속성적장악개솔경직접지반영료피시능력적당전고계치。인차,기우피시적속성장악개솔래구건선제책략,선택최능개변피시속성장악개솔적제목작위하일개측험항목,저응해시일개치득상시적방안。본문차감이유상관연구적수거생성모식진행탐색,모의실험결과표명:가설속성간상호독립,재정장(장도위16)、변장(장도위16혹후험속성장악모식개솔체도0.8)이급단측험(장도분별위4、6、8、10)적정황하,기우속성장악개솔적선제책략PPWKL화PHKL유교호적분류준학솔,재제목폭광솔,제고사용균균성등방면야유교호적표현;여연구교다적PWKL、HKL등책략상비,야략유우세;당속성간존재불동정도적상관시,재정장、변장이급교단적측험조건하,기우PHKL화MI적측험대지식상태고계정도교호,기우PPWKL화PHKL적측험종합표현점우。
Cognitive diagnostic computerized adaptive testing(CD-CAT) is a popular mode of online testing of cognitive diagnostic assessment(CDA). The key to a CD-CAT system is the item selection strategies. Someof the popular strategies are developed based on Kullback-Leibler information(KL), Shannon entropy(SHE) toselect items in CD-CAT. Typically, during CD-CAT, thesefamiliar methods would use a cutoff point to transform the attribute mastery probabilities' provisional value to binary values, but at the initial stage, the cutoff point method may lead to a larger deviation. A method that can take advantage of the probabilistic information with regard to attributes may offer a better alternative. This paper proposed two item selection strategies based on the provisional value of the attribute mastery probabilities, as follows:(1) the first strategy, which is called as PPWKL(Posterior Probability WeightedKullback-Leibler), is based on the KL information, and it can lead to maximum difference of the sum of attribute mastery probabilities, and it is weighted bythe pattern's posterior probability as well asthe difference of the attribute mastery probabilities between the ?? and any possible latent state;(2) the PPWKL considers the fact that not all the patterns are equally likely, but overlooks the fact that the distances between different patterns and the current estimate are not all of equal importance. Therefore, the PPWKL can be weighed by the inverse of the distance between the ?? and any possible latent state, which is called as PHKL(Posterior Hybrid KullbackLeibler). Then, three simulation studies were carried out, one was the fixed length of CD-CAT, and the secondwas the variant length CD-CAT, and the last wasshort length CD-CAT.Variant item selection strategiesweretaken intoconsideration in these studies, including KL, SHE, PWKL, HKL, MI, PPWKL and PHWKL. The results were compared in terms of pattern or attribute classification correct rate, itemaverage exposure ratio, item maximum exposure ratio, item minimum exposure ratio, average test length, unused item number, number of items with exposure ratioover 20%, test overlap ratio.The simulation results indicate that:(1) the comprehensive performance of PPWKL and PHKLare better than other mentioned strategies in fixed and variant length CD-CAT; as to PHKL and MI, each has different strengths in short length CD-CAT;(2) PHKL and PPWKL can retain a good measurement accuracy, and also improve the utilization ratio of item pool.