CAJ | 학술논문

文本特征选择是在文本自动分类中最重要的一个环节。为了更好地解决维吾尔文文本分类中特征空间的高维性和文档表示向量的稀疏性问题, 提出一种基于特征的类别分布差异和信息熵的维吾尔文文本特征选择方法。该方法不仅要考虑特征在类别间的分布情况, 而且也要考虑特征在类别内的分布情况。采用本方法对维吾尔文文本语料进行了分类实验, 并与一些传统的特征选择方法进行了比较。从结果来看, 本方法在所选特征数更少的情况下, 达到了比其他方法更高的分类MacroF1值853%, 比传统的IG和CHI等方法在MacroF1值上分别高出了43%和61%。
문본특정선택시재문본자동분류중최중요적일개배절。위료경호지해결유오이문문본분류중특정공간적고유성화문당표시향량적희소성문제, 제출일충기우특정적유별분포차이화신식적적유오이문문본특정선택방법。해방법불부요고필특정재유별간적분포정황, 이차야요고필특정재유별내적분포정황。채용본방법대유오이문문본어료진행료분류실험, 병여일사전통적특정선택방법진행료비교。종결과래간, 본방법재소선특정수경소적정황하, 체도료비기타방법경고적분류MacroF1치853%, 비전통적IG화CHI등방법재MacroF1치상분별고출료43%화61%。
Text feature selection is the most important phase in automatic text categorization. In order to solve the high dimensionality and sparsness of text vector in Uyghur text categorization, this paper proposed the new Uyghur text feature selection method based on class distribution difference and term entropy. The propesed method not only considered the inter-class distribution of the term, but also considered the inner-class distribution of the term. It conducted the categorization experiments on the Uyghur text corpus using proposed method and compared with the traditional feature selection methods. The experimental results show that the categorization MacroF1 value is reached 853% and achieves the improvement of 43% and 61% respectivly comparing to IG and CHI.