计算机工程与应用
計算機工程與應用
계산궤공정여응용
COMPUTER ENGINEERING AND APPLICATIONS
2010年
3期
125-127,130
,共4页
自动分词%分词算法%字典%歧义切分
自動分詞%分詞算法%字典%歧義切分
자동분사%분사산법%자전%기의절분
automatic segmentation%segmentation algorithm%dictionary%ambiguity segmentation
分析了中文分词词典的机制,提出了一种改进的整词分词字典结构,并针对机械分词算法的特点,将其与概率算法相结合,探讨了一种中文自动分词概率算法.采用哈希及二分法对词典进行分词匹配.实验表明,该算法具有较高的分词效率和准确率,对于消去歧义词也有较好的性能.
分析瞭中文分詞詞典的機製,提齣瞭一種改進的整詞分詞字典結構,併針對機械分詞算法的特點,將其與概率算法相結閤,探討瞭一種中文自動分詞概率算法.採用哈希及二分法對詞典進行分詞匹配.實驗錶明,該算法具有較高的分詞效率和準確率,對于消去歧義詞也有較好的性能.
분석료중문분사사전적궤제,제출료일충개진적정사분사자전결구,병침대궤계분사산법적특점,장기여개솔산법상결합,탐토료일충중문자동분사개솔산법.채용합희급이분법대사전진행분사필배.실험표명,해산법구유교고적분사효솔화준학솔,대우소거기의사야유교호적성능.
Chinese segmentation mechanism is analyzed.An improved structure of segmentation dictionary is presented,and in view of the characteristics of the mechanical Chinese word segmentation,combined with probabilistic algorithm,a Chinese Word Automatic Segmentation probabilistic algorithm is discussed.Hashing and binary search is used to segmentation match.Experiment indicates that the algorithm can greatly improve the speed of Chinese segmentation and precision,and strengthen the processing of dispelling ambiguity.
