CAJ | 학술논문

针对Hadoop默认调度算法和异构环境下LATE调度算法的不足，在SAMR调度算法的基础上提出了一种增强的自适应MapReduce调度算法。该算法记录了每个节点的历史信息，采用K-means聚类算法动态地调整阶段进度值以找到真正需要启动备份的落后任务。实验结果表明，增强自适应的MapReduce调度算法在提高任务执行时间的估算误差以及准确识别慢任务方面具有一定的有效性。
침대Hadoop묵인조도산법화이구배경하LATE조도산법적불족，재SAMR조도산법적기출상제출료일충증강적자괄응MapReduce조도산법。해산법기록료매개절점적역사신식，채용K-means취류산법동태지조정계단진도치이조도진정수요계동비빈적락후임무。실험결과표명，증강자괄응적MapReduce조도산법재제고임무집행시간적고산오차이급준학식별만임무방면구유일정적유효성。
Aiming at the shortage of Hadoop default scheduling algorithm and LATE scheduling algorithm of heterogeneous environment, this paper proposes an enhanced adaptive MapReduce scheduling algorithm on the basis of SAMR scheduling algorithm. The algorithm records the history information of each node, and uses K-means clustering algorithm to dynamically adjust the progress value, aims to find the slow tasks which are really need begin back-up. Finally, the experimental results show that the enhanced MapReduce scheduling algorithm has some validity in the aspect of improving the estimation error of the tasks’execution time and accurately identifying the slow tasks.