CAJ | 학술논문

针对传统文本索引技术空间消耗大、分词不准确等问题，设计并实现了高性能文本索引系统。该系统采用压缩的全文自索引算法，节省了空间开销，避免了自然语言分词方法的影响，配合通配符搜索算法扩展了模糊搜索的功能，在众核CPU高性能处理器上可实现多线程并行处理，提高了处理速度，整个系统的实现是基于Web方式的，可以跨平台运行。实验结果表明，该系统将文本索引的空间消耗降为原文本的50％左右，具有较高的实用价值。
침대전통문본색인기술공간소모대、분사불준학등문제，설계병실현료고성능문본색인계통。해계통채용압축적전문자색인산법，절성료공간개소，피면료자연어언분사방법적영향，배합통배부수색산법확전료모호수색적공능，재음핵CPU고성능처리기상가실현다선정병행처리，제고료처리속도，정개계통적실현시기우Web방식적，가이과평태운행。실험결과표명，해계통장문본색인적공간소모강위원문본적50％좌우，구유교고적실용개치。
The traditional text index technology has problems like oversize space consumption,inaccurate word segmentation,etc. In order to solve these problems,a high-performance text index system is designed and implemented.This system employs com-pressed full-text self-index algorithm,which helps save space and avoid influence from natural language word segmentation.Com-bined with wildcard search algorithm,its search function is enriched.And applied on high-performance processors with many-core CPU,this system can execute in a parallel multi-thread way,which increases processing speed.The entire system is Web-based,so that it can be run in different operating systems.Experimental results show that this systemhas high practical value in thatit reduces space consumption to about 50%of primary text.