心理学报
心理學報
심이학보
Acta Psychologica Sinica
2015年
5期
689~701
,共null页
詹沛达 李晓敏 王文中 边玉芳 王立君
詹沛達 李曉敏 王文中 邊玉芳 王立君
첨패체 리효민 왕문중 변옥방 왕립군
认知诊断 题组 项目反应理论 多维题组效应 Logistic题组框架 DINA
認知診斷 題組 項目反應理論 多維題組效應 Logistic題組框架 DINA
인지진단 제조 항목반응이론 다유제조효응 Logistic제조광가 DINA
cognitive diagnosis; testlet; item response theory; multidimensional testlet-effect; Logistic testlet framework; DINA
当前认知诊断领域还缺少对包含题组的测验进行诊断分析的研究,即已开发的认知诊断模型无法合理有效地处理含有题组效应的测验数据,且已开发的题组反应模型也不具有对被试知识结构或认知过程进行诊断的功能。针对该问题,本文尝试性地将多维题组效应向量参数引入线性Logistic模型中,同时开发了属性间具有补偿作用的和属性间具有非补偿作用的多维题组效应认知诊断模型。模拟研究结果显示新模型合理有效,与线性Logistic模型和DINA模型对比研究后表明:(1)作答数据含有题组效应时,忽略题组效应会导致项目参数的偏差估计并降低对目标属性的判准率;(2)新模型更具普适性,即便当作答数据不存在题组效应时,采用新模型进行测验分析亦能得到很好的项目参数估计结果且不影响对目标属性的判准率。整体来看,新模型既具有认知诊断功能又可有效处理题组效应。
噹前認知診斷領域還缺少對包含題組的測驗進行診斷分析的研究,即已開髮的認知診斷模型無法閤理有效地處理含有題組效應的測驗數據,且已開髮的題組反應模型也不具有對被試知識結構或認知過程進行診斷的功能。針對該問題,本文嘗試性地將多維題組效應嚮量參數引入線性Logistic模型中,同時開髮瞭屬性間具有補償作用的和屬性間具有非補償作用的多維題組效應認知診斷模型。模擬研究結果顯示新模型閤理有效,與線性Logistic模型和DINA模型對比研究後錶明:(1)作答數據含有題組效應時,忽略題組效應會導緻項目參數的偏差估計併降低對目標屬性的判準率;(2)新模型更具普適性,即便噹作答數據不存在題組效應時,採用新模型進行測驗分析亦能得到很好的項目參數估計結果且不影響對目標屬性的判準率。整體來看,新模型既具有認知診斷功能又可有效處理題組效應。
당전인지진단영역환결소대포함제조적측험진행진단분석적연구,즉이개발적인지진단모형무법합리유효지처리함유제조효응적측험수거,차이개발적제조반응모형야불구유대피시지식결구혹인지과정진행진단적공능。침대해문제,본문상시성지장다유제조효응향량삼수인입선성Logistic모형중,동시개발료속성간구유보상작용적화속성간구유비보상작용적다유제조효응인지진단모형。모의연구결과현시신모형합리유효,여선성Logistic모형화DINA모형대비연구후표명:(1)작답수거함유제조효응시,홀략제조효응회도치항목삼수적편차고계병강저대목표속성적판준솔;(2)신모형경구보괄성,즉편당작답수거불존재제조효응시,채용신모형진행측험분석역능득도흔호적항목삼수고계결과차불영향대목표속성적판준솔。정체래간,신모형기구유인지진단공능우가유효처리제조효응。
Cognitive diagnosis, which is also referred as skill assessment or skill profiling, utilizes latent class models to provide fine-grained information about students' strength and weakness in the learning process. The outcome of cognitive diagnostic models(CDMs) is a profile with binary element for each examinee to indicate the mastery/nonmastery status of every attribute/skill. Therefore, one major advantage of CDMs is the capacity to provide additional information about the instructional needs of students. In the past decades, extensive research has been conducted in the area of cognitive diagnosis and many statistical models based on a probabilistic approach have been proposed. Examples of CDMs include the deterministic inputs, noisy and gate(DINA) model(Junker Sijtsma, 2001), the deterministic input, noisy or gate(DINO) model(Templin Henson, 2006), and the linear Logistic model(LLM)(Maris, 1999). In educational measurement, one of the most commonly used formats is the testlet design, which is a cluster of items that share a common stimulus(e.g., a reading comprehension passage or a figure). Under the framework of item response theory(IRT), various testlet response models(TRM) have been proposed, such as the Rasch testlet model(Wang Wilson, 2005) and the multidimensional testlet-effect Rasch model(MTERM)(Zhan, Wang, Wang, Li, 2014). However, limited efforts have been contributed to the development of testlet models for CDMs. A question then naturally arises is the searching for a way to account for testlet effect under CDMs. To address this issue, this study proposed two testlet-CDMs. One followed the compensatory approach and the other followed the noncompensatory approach:(1) the compensatory multidimensional testlet-effect CDM(C-MTECDM) was based on the combination of LLM and MTERM, while(2) the noncompensatory multidimensional testlet-effect CDM(N-MTECDM) was based on the combination of(logit)DINA model and MTERM, respectively. Model parameters can be estimated by the Bayesian methods with Markov chain MonteCarlo(MCMC) algorithms, which have been implemented with the freeware Win BUGS. In study 1, a series of simulations were conducted to evaluate parameter recovery of two new models, and results showed that the model parameters could be recovered fairly well under all simulated conditions. In study 2, the two new models were compared with the LLM and the(logit)DINA model, respectively. Results showed that ignoring testlet effect would result in biased item parameter estimations and worse person classification rates. Additionally, fitting a more complicated model(i.e., MTECDM) to data with a simpler structure did litter harm on parameter recovery. In conclusion, the new models is feasible and flexible.