CAJ | 학술논문

为了解决传统主题爬虫效率偏低的问题，传统主题爬虫会选择最有价值的链接进行访问，仅简单地计算链接的相关性，却忽视待分析URL之间的相关性关系，致使主题爬虫爬取效率较低。提出一种基于链接模型的相关性判别算法，综合利用有标种子URL和无标的待判别URL实现对无标URL的相关性判别，并推导出迭代初值选取对结果的不敏感性。实验结果表明，与传统的网络爬虫算法相关性判别方法相比，提出的方法效率更高。
위료해결전통주제파충효솔편저적문제，전통주제파충회선택최유개치적련접진행방문，부간단지계산련접적상관성，각홀시대분석URL지간적상관성관계，치사주제파충파취효솔교저。제출일충기우련접모형적상관성판별산법，종합이용유표충자URL화무표적대판별URL실현대무표URL적상관성판별，병추도출질대초치선취대결과적불민감성。실험결과표명，여전통적망락파충산법상관성판별방법상비，제출적방법효솔경고。
In order to solve the low efficiency problem of traditional focused crawler, web spider always selects the most valuable links to visit, so how to focus the search around a given topic is a key problem. The traditional method always only computes the relevance of the links, but ignores the relevance among the unlabeled URL, now it proposes the algorithm based on link model which combines the seed URL with unlabeled URL to compute the relevance of the other URL, and it deduces the point that initial iterative is insensitivity of the results. Compared with the methods based on traditional algorithm, experimental result proves the performance of the new algorithm is more efficient than the traditional ones.