CAJ | 학술논문

随着互联网的飞速发展，网络上新闻信息越来越繁杂，采集有用数据过滤冗余数据变得十分重要，但目前市面上流行软件并不能过滤冗余新闻。采用网络爬虫、中文分词、向量空间模型、文本聚类等技术可设计一个能自动采集新闻并能将所得信息自动聚类的系统，并且通过真实新闻数据验证了该系统的有效性，证明其能帮助用户发现、过滤重复新闻、相似新闻，并能提取热点新闻，提高用户阅读新闻的效率。
수착호련망적비속발전，망락상신문신식월래월번잡，채집유용수거과려용여수거변득십분중요，단목전시면상류행연건병불능과려용여신문。채용망락파충、중문분사、향량공간모형、문본취류등기술가설계일개능자동채집신문병능장소득신식자동취류적계통，병차통과진실신문수거험증료해계통적유효성，증명기능방조용호발현、과려중복신문、상사신문，병능제취열점신문，제고용호열독신문적효솔。
With the rapid development of Internet, the news information resources on network are increasingly complicated. So it becomes very important to collect useful data and to filter redundant data, but the popular software can not do that. A system can automatically gather news and cluster obtained information by using technologies such as web crawler, Chinese segmentation, vec?tor space model and text clustering, which is proved to be an effective system through based on the real news data. And it can help users to find and filter overlapping news, extract the hot news as well as improve the efficiency to read the news.