CAJ | 학술논문

相似性连接，即利用相似函数度量数据之间的相似程度，满足条件后进行连接操作。MapReduce框架下已存在很多相似性连接算法，但仍然存在一些不足，如大量的索引加大时间、空间的开销；现有算法不能有效地完成增量式数据集的相似性连接等。针对海量增量式数据集进行了研究，采用抽样技术得到有效中枢，形成更为合理的分区，建立分区索引和分配原则，完成新增数据的相似性连接操作。实验证明，该算法能够有效地解决海量增量式数据集的相似性连接问题，验证了分区索引的建立，可以提高新增数据的相似性连接操作的效率。
상사성련접，즉이용상사함수도량수거지간적상사정도，만족조건후진행련접조작。MapReduce광가하이존재흔다상사성련접산법，단잉연존재일사불족，여대량적색인가대시간、공간적개소；현유산법불능유효지완성증량식수거집적상사성련접등。침대해량증량식수거집진행료연구，채용추양기술득도유효중추，형성경위합리적분구，건립분구색인화분배원칙，완성신증수거적상사성련접조작。실험증명，해산법능구유효지해결해량증량식수거집적상사성련접문제，험증료분구색인적건립，가이제고신증수거적상사성련접조작적효솔。
Similarity join was namely that using similar function to measure the similarity level of the data set,and then doing the join after meeting the condition.Many effective similarity join algorithms had been in mapreduce,but there were still some insufficiency,such as a lot of indexes increases the overhead of time and space;the existing algorithm couldn’t deal with the similarity computation of the incremental data set effectively,and so on.For massive incremental data set,this paper made use of sampling to get the valid pivots,which established partitions’indexes and distribution principle,then finished the similarity join operation of additional data.The experiments prove that the algorithm can solve the problem of the similarity join of the in-cremental data set effectively,and verify that through creating partitions’indexes,it can improve the efficiency of the similari-ty join operation of additional data.