CAJ | 학술논문

针对目前影响爬虫程序效率的诸多关键因素,在研究爬虫程序内部运行机理的基础上,进行架构优化,改进爬虫程序中的相关算法.在Linux网络环境下,通过对实现的爬虫程序运行进行检测,反馈出该解决方案和改进之处具有可行性,提高了页面抓取的效率和爬虫程序的整体性能.
침대목전영향파충정서효솔적제다관건인소,재연구파충정서내부운행궤리적기출상,진행가구우화,개진파충정서중적상관산법.재Linux망락배경하,통과대실현적파충정서운행진행검측,반궤출해해결방안화개진지처구유가행성,제고료혈면조취적효솔화파충정서적정체성능.
In view of current key aspects that affect the crawler system efficiency, through research of crawler system interior movement mechanism, this paper optimizes the overhead construction and improves its algorithm. In the Linux network environment, through movement examination of the crawler system, it may feed back several kinds of solutions and improvement place which are feasible, and it also enhances the efficiency and the crawler system overall performance.