CAJ | 학술논문

作为开源云计算平台的核心技术之一，MapReduce 作业处理框架及其作业调度算法，对整个系统的性能起着至关重要的作用，而数据本地性是衡量作业调度算法好坏的一个重要标准，首先本文介绍和分析了 MapReduce 基本原理，MapReduce 作业处理机制和 MapReduce 作业调度机制及其在数据本地性方面表现出的优缺点等相关内容。其次，针对原生作业调度算法在数据本地性考虑不周全的问题，结合数据预取技术的可行性与优势，通过引入资源预取技术设计并实现一种基于资源预取的Hadoop MapReduce作业调度算法，使作业执行效率更高。
작위개원운계산평태적핵심기술지일，MapReduce 작업처리광가급기작업조도산법，대정개계통적성능기착지관중요적작용，이수거본지성시형량작업조도산법호배적일개중요표준，수선본문개소화분석료 MapReduce 기본원리，MapReduce 작업처리궤제화 MapReduce 작업조도궤제급기재수거본지성방면표현출적우결점등상관내용。기차，침대원생작업조도산법재수거본지성고필불주전적문제，결합수거예취기술적가행성여우세，통과인입자원예취기술설계병실현일충기우자원예취적Hadoop MapReduce작업조도산법，사작업집행효솔경고。
As one of the core technologies in Hadoop,MapReduce framework and its job scheduling algorithm have a significant influence on performanceof the entire system,data locality determines the quality of scheduling algo-rithm.First of all,the introduction and analysis of the basic principles ,job processing mechanism , job scheduling me-chanism of MapReduce and the advantages and disadvantages about data locality on MapReduce are discussed and analyzed.Secondly,to address the problem that the existing Hadoop job scheduling algorithms cannot guarantee good data locality,combining with Data Prefetching technology,a Hadoop MapReduce job scheduling algorithm based on resource prefetching is proposed and implemented, the job executes more efficiently.