计算机工程与应用
計算機工程與應用
계산궤공정여응용
COMPUTER ENGINEERING AND APPLICATIONS
2013年
19期
39-43,140
,共6页
MapReduce%推测执行%异构环境%K-means算法
MapReduce%推測執行%異構環境%K-means算法
MapReduce%추측집행%이구배경%K-means산법
MapReduce%speculative execution%heterogeneous environment%K-means algorithm
针对Hadoop默认调度算法和异构环境下LATE调度算法的不足,在SAMR调度算法的基础上提出了一种增强的自适应MapReduce调度算法。该算法记录了每个节点的历史信息,采用K-means聚类算法动态地调整阶段进度值以找到真正需要启动备份的落后任务。实验结果表明,增强自适应的MapReduce调度算法在提高任务执行时间的估算误差以及准确识别慢任务方面具有一定的有效性。
針對Hadoop默認調度算法和異構環境下LATE調度算法的不足,在SAMR調度算法的基礎上提齣瞭一種增彊的自適應MapReduce調度算法。該算法記錄瞭每箇節點的歷史信息,採用K-means聚類算法動態地調整階段進度值以找到真正需要啟動備份的落後任務。實驗結果錶明,增彊自適應的MapReduce調度算法在提高任務執行時間的估算誤差以及準確識彆慢任務方麵具有一定的有效性。
침대Hadoop묵인조도산법화이구배경하LATE조도산법적불족,재SAMR조도산법적기출상제출료일충증강적자괄응MapReduce조도산법。해산법기록료매개절점적역사신식,채용K-means취류산법동태지조정계단진도치이조도진정수요계동비빈적락후임무。실험결과표명,증강자괄응적MapReduce조도산법재제고임무집행시간적고산오차이급준학식별만임무방면구유일정적유효성。
Aiming at the shortage of Hadoop default scheduling algorithm and LATE scheduling algorithm of heterogeneous environment, this paper proposes an enhanced adaptive MapReduce scheduling algorithm on the basis of SAMR scheduling algorithm. The algorithm records the history information of each node, and uses K-means clustering algorithm to dynamically adjust the progress value, aims to find the slow tasks which are really need begin back-up. Finally, the experimental results show that the enhanced MapReduce scheduling algorithm has some validity in the aspect of improving the estimation error of the tasks’execution time and accurately identifying the slow tasks.