CAJ | 학술논문

为了提高数据的可用性和集群的整体性能，目前的HDFS（ Hadoop Distributed File System）采用了副本数目固定的副本放置技术，然而由于文件热度存在较大差异，对那些具有较高热度文件的访问将影响作业的执行。为克服上述问题，本文提出一种基于预测的热点数据副本因子决策算法。根据数据的最近访问特征，基于灰色预测技术，采用马尔科夫预测模型修正因数据波动和突发访问造成的预测偏差，获取文件的未来访问热度，并基于预测值建立有限通道服务模型，寻找满足用户需求的最小副本因子。实验表明，较之现有的副本管理策略和基于实时热度调整副本因子策略，本策略可以有效减少热点数据的访问冲突，减少热点数据作业的执行时间和网络负载。
위료제고수거적가용성화집군적정체성능，목전적HDFS（ Hadoop Distributed File System）채용료부본수목고정적부본방치기술，연이유우문건열도존재교대차이，대나사구유교고열도문건적방문장영향작업적집행。위극복상술문제，본문제출일충기우예측적열점수거부본인자결책산법。근거수거적최근방문특정，기우회색예측기술，채용마이과부예측모형수정인수거파동화돌발방문조성적예측편차，획취문건적미래방문열도，병기우예측치건립유한통도복무모형，심조만족용호수구적최소부본인자。실험표명，교지현유적부본관리책략화기우실시열도조정부본인자책략，본책략가이유효감소열점수거적방문충돌，감소열점수거작업적집행시간화망락부재。
To improve data availability and performance of cluster, current HDFS adapt uniform data replication.However, dif-ferent files have different popularity and sometimes the disparity is enormous, access to high popular data may hurt job perform-ance.To address this problem, a dynamic replicas strategy based on predicted popularity is put forward.By making full use of the recent data popularity, based on grey prediction model, we use Markov prediction model to correct the predicted deviation be-cause of the burst access and shifting access, and get the accurate predicted popularity of file.After then, finite channel service model based on the predicted popularity is established to calculate the minimum replicas meeting user demand.Experimental re-sult shows that compared with default data replication, our strategy can more effectively avoid contentions, reduce the time consu-ming of job, and alleviated the network traffic.