CAJ | 학술논문

矩阵取样测验包含多个题册，单个题册的总分不能直接作为匹配变量用于DIF检测。本研究首先基于模拟数据，同时采用IRT_△b法，以及用IRT模型估计的考生能力作为匹配变量修订后的LR法对矩阵取样测验进行DIF检测，分析二者进行DIF检测的有效性及其相关影响因素；并根据已有的LR法DIF判断标准划定出IRT_△b法分类标准；最后使用实证数据加以验证。结果显示：矩阵取样测验中，IRT_△b法和修正LR法均能较好地区分DIF量不同的题目；样本量、题册中DIF题目的比例和考生群体间真实能力的差异对两种方法的检验力、犯I类错误的概率和分类结果都有较大影响。
구진취양측험포함다개제책，단개제책적총분불능직접작위필배변량용우DIF검측。본연구수선기우모의수거，동시채용IRT_△b법，이급용IRT모형고계적고생능력작위필배변량수정후적LR법대구진취양측험진행DIF검측，분석이자진행DIF검측적유효성급기상관영향인소；병근거이유적LR법DIF판단표준화정출IRT_△b법분류표준；최후사용실증수거가이험증。결과현시：구진취양측험중，IRT_△b법화수정LR법균능교호지구분DIF량불동적제목；양본량、제책중DIF제목적비례화고생군체간진실능력적차이대량충방법적검험력、범I류착오적개솔화분류결과도유교대영향。
Matrix sampling is a useful technique widely used in large-scale educational assessments. In an assessment with matrix sampling design, each examinee takes one of the multiple booklets with partial items. A critical problem of detecting differential item functioning （DIF） in such scenario has gained a lot of attention in recent years, which is, it is not appropriate to take the observed total score obtained from individual booklet as the matching variable in detecting the DIF. Therefore, the traditional detecting methods, such as Mantel-Haenszel （MH）, SIBTEST, as well as Logistic Regression （LR） are not suitable. IRT_△b might be an alternative due to its abilities to provide valid matching variable. However, the DIF classification criterion of IRT_△b was not well established yet. Thus, the purpose of this study were： 1） to investigate the efficiency and robustness of using ability parameters obtained from Item Response Theory （IRT） model as the matching variable, comparing with the way using traditional observed raw total scores ; 2） to further identify what factors will influence the abilities in detecting DIF of two methods; 3） to propose a DIF classification criteria for IRT_△b. Simulated and empirical data were both employed in this study to explore the robustness and the efficiency of the two prevailing DIF detecting methods, which were the IRT_△b method and the adapted LR method with the estimation of group-level ability based on IRT model as the matching variable. In the Monte Carlo study, a matrix sampling test was generated, and various experimental conditions were simulated as follows： 1） different proportions of DIF items; 2） different actual examinee ability distributions; 3） different sample sizes; 4） different size of DIF. Two DIF detection methods were then applied and results were compared. In addition, power functions were established in order to derive DIF classification rule for IRT Ab based on current rules for LR. In the empirical study, through conducting a DIF analysis for American and Korean mathematics tests from Programme for International Student Assessment （PISA） 2003, the consistency of the classification rules between IRT Ab and LR were further examined. The results indicated that in the matrix sampling design, both IRT_△b method and adjusted LR method were sensitive to the diverse DIF magnitude. It was also found that the power, type I error, and the final classification of both methods were also influenced by the sample size, percentage of items with DIF, and ability differences between the focused group and the reference group. In conclusion, it was found that both the IRT_△b method and adjusted LR method can be used to detect DIF in matrix sampling tests. A classification rule for IRT_△b was proposed, which are： 0.85 between negligible DIF（A） and intermediate DIF（B）, 1.23 between intermediate DIF（B） and large DIF（C）. Meanwhile, it was suggested that researchers would take this rule as a tentative principle since the AR2 was limited between a narrow interval and the classification rule of LR was very flexible compared to classification rule of MH. Further studies could be conducted to take MH, IRT_△b as well as LR into consideration simultaneously to give more comparable and consistent classification rules for different methods.