CAJ | 학술논문

针对现有中文分词算法无法为移动搜索提供用户兴趣偏好信息的现状，提出一种改进的正向最大匹配中文分词算法。该算法基于逐字二分的分词词典机制，添加词分类信息，在词典中存储了每个词条的分类信息，分词时采用改进的次字区位码哈希非均匀分段机制进行正向最大匹配分词。实验结果表明，与逐字二分法相比，改进的分词算法其存储空间增加了13％，但时间效率提高了20％左右，且分词后可同时提取出词条的分类信息。
침대현유중문분사산법무법위이동수색제공용호흥취편호신식적현상，제출일충개진적정향최대필배중문분사산법。해산법기우축자이분적분사사전궤제，첨가사분류신식，재사전중존저료매개사조적분류신식，분사시채용개진적차자구위마합희비균균분단궤제진행정향최대필배분사。실험결과표명，여축자이분법상비，개진적분사산법기존저공간증가료13％，단시간효솔제고료20％좌우，차분사후가동시제취출사조적분류신식。
As existing Chinese word segmentation algorithm can’t provide user interest information for mobile search ,an improved FMM segmentation algorithm is proposed .Based on a new dictionary mechanism which contains words’ classified information , the algorithm performs Forward Maximum Matching by the improved second word area code hash non‐uniform segmentation mechanism . Experimental results show that compared with the Verbatim dichotomy ,the storage space of the improved algorithm is increased by 13% ,but the time efficiency is improved by about 20% ,and the words’ classified information is extracted simultaneously .