CAJ | 학술논문

针对专利文献的特点,该文提出了一种基于统计和规则相结合的多策略分词方法.该方法利用文献中潜在的切分标记,结合切分文本的上下文信息进行最大概率分词,并利用术语前后缀规律进行后处理.该方法充分利用了从大规模语科中获取的全局信息和切分文本的上下文信息,有效地解决了专利分词中未登录词难以识别问题.实验结果表明,该文方法在封闭和开放测试下分别取得了较好的结果,对未登录词的识别也有很好的效果.
침대전리문헌적특점,해문제출료일충기우통계화규칙상결합적다책략분사방법.해방법이용문헌중잠재적절분표기,결합절분문본적상하문신식진행최대개솔분사,병이용술어전후철규률진행후처리.해방법충분이용료종대규모어과중획취적전국신식화절분문본적상하문신식,유효지해결료전리분사중미등록사난이식별문제.실험결과표명,해문방법재봉폐화개방측시하분별취득료교호적결과,대미등록사적식별야유흔호적효과.
According to the characteristics of the patent documents, this paper presents a multi-strategy approach for word segmentation based on statistics and rules. Our method takes advantage of the latent segmentation-marks in the document and employs the context information of the text in the a maximum probabilistic model of segmentation.Meanwhile, the term affix rules are applied in the post-processing. Making full use of the global information from a large scale corpus and the specific context information, this method effectively solves the problem of the out-of-vo-cabulary words difficult to identify in the patent segmentation. The experimental results indicate that this method achieves good results in the close and opening test, with improves on unknown words recognition as well.