CAJ | 학술논문

针对云计算环境中大规模数据集的处理,MapReduce集群已成为一个强大的处理平台。文中提出了一种基于虚拟化平台动态资源重配置的资源评价和动态资源重新配置调度算法。该算法动态地评估作业在截止时间内完成所需要的Map和Reduce计算资源数量,并在不违反用户设定的时间目标的情况下,通过动态地增加或减少独立虚拟机的方式来调整CPU资源,以实现提高数据本地性,同时提高系统在运行作业时的资源利用率。仿真实验结果表明,该算法可以使集群上的MapReduce作业的吞吐率有明显的提高。
침대운계산배경중대규모수거집적처리,MapReduce집군이성위일개강대적처리평태。문중제출료일충기우허의화평태동태자원중배치적자원평개화동태자원중신배치조도산법。해산법동태지평고작업재절지시간내완성소수요적Map화Reduce계산자원수량,병재불위반용호설정적시간목표적정황하,통과동태지증가혹감소독립허의궤적방식래조정CPU자원,이실현제고수거본지성,동시제고계통재운행작업시적자원이용솔。방진실험결과표명,해산법가이사집군상적MapReduce작업적탄토솔유명현적제고。
Aiming at processing of large-scale data set in cloud computing environment,MapReduce has become a powerful processing platform. In this paper,propose a resource evaluation and dynamic resource reconfiguration and scheduling algorithm based on virtualiza-tion platform dynamic resource reconfiguration. It can dynamically evaluate the required number of Map/Reduce slots for every job to meet completion time guarantee and adjust the CPU resources while not violating completion time goals of the users by dynamically in-creasing or decreasing individual VMs to maximize data locality and also to maximize the use of resources within the system among the active jobs. Simulation results show that the algorithm can improve the throughput of MapReduce jobs on the cluster significantly.