CAJ | 학술논문

针对主题信息采集系统存在的采集效率低和可扩展性差等问题，研究分布式采集系统架构，设计一种基于局域网的多节点服务器架构主题采集系统，解决了客户／服务器模式在采集网页数据增多时服务器控制端性能下降的问题，弥补了自治模式下网络间传递信息量增加导致数据延迟的缺陷。通过分别构建单节点服务器架构系统和基于局域网的多节点服务器架构系统进行实验，分析采集速度、平均采集速度和准确率等评价指标，分析结果表明，多节点服务器架构性能明显优于单节点服务器架构。
침대주제신식채집계통존재적채집효솔저화가확전성차등문제，연구분포식채집계통가구，설계일충기우국역망적다절점복무기가구주제채집계통，해결료객호／복무기모식재채집망혈수거증다시복무기공제단성능하강적문제，미보료자치모식하망락간전체신식량증가도치수거연지적결함。통과분별구건단절점복무기가구계통화기우국역망적다절점복무기가구계통진행실험，분석채집속도、평균채집속도화준학솔등평개지표，분석결과표명，다절점복무기가구성능명현우우단절점복무기가구。
To address the problems of low collecting efficiency,poor scalability and other issues of the focused web crawler,the research of focused web crawler for distributed architecture was carried out.The focused web crawler of multi-node server archi-tecture based on distributed architecture was designed.Performance degradation problems of web server control terminal were solved when data collection increased in the client/server mode,and defects of data delay caused by the increasing the amount of transmission networks information in autonomous mode were overcome.Through constructing the single-node server architecture system and multi-node server architecture system,the evaluation including collecting speed,average speed and accuracy rate were compared.Results of the simulation indicate that the performance of multi-node server architecture is obviously superior to single-node server architecture.