CAJ | 학술논문

分析了 Cloudera 公司推出的 Impala 实时查询引擎原理与架构，并深入比较 Impala 与传统 MapReduce 的性能与特点，针对 Impala 进行复杂大数据处理方面的不足，提出了 MapReduce 与 Impala 结合的大数据处理方法，通过使用 MapReduce 对 Impala 的输入数据进行预处理，利用 MapReduce 在复杂作业处理方面的长处弥补了Impala 在这方面的不足。最后对电信手机上网日志进行大数据查询和分析计算实验，实验结果表明，在大数据查询性能方面，基于 MapReduce 与 Impala 结合的大数据处理速度比传统 MapReduce 快了一倍。特别地，在迭代查询实验中，基于 MapReduce 与 Impala 结合的处理方法超过传统 MapReduce 方法八倍以上。基于 MapReduce与 Impala 结合的处理方法在单次查询中的效率仍然高于传统 MapReduce；而在迭代查询中，MapReduce 与 Impala结合的处理方法远远地超过了 MapReduce。因此，MapReduce 与 Impala 结合的处理方法能够发挥 Impala 和 Ha-doop 各自的优点，让处理效率远超传统 MapReduce，对于复杂的大数据处理的能力高于 Impala。
분석료 Cloudera 공사추출적 Impala 실시사순인경원리여가구，병심입비교 Impala 여전통 MapReduce 적성능여특점，침대 Impala 진행복잡대수거처리방면적불족，제출료 MapReduce 여 Impala 결합적대수거처리방법，통과사용 MapReduce 대 Impala 적수입수거진행예처리，이용 MapReduce 재복잡작업처리방면적장처미보료Impala 재저방면적불족。최후대전신수궤상망일지진행대수거사순화분석계산실험，실험결과표명，재대수거사순성능방면，기우 MapReduce 여 Impala 결합적대수거처리속도비전통 MapReduce 쾌료일배。특별지，재질대사순실험중，기우 MapReduce 여 Impala 결합적처리방법초과전통 MapReduce 방법팔배이상。기우 MapReduce여 Impala 결합적처리방법재단차사순중적효솔잉연고우전통 MapReduce；이재질대사순중，MapReduce 여 Impala결합적처리방법원원지초과료 MapReduce。인차，MapReduce 여 Impala 결합적처리방법능구발휘 Impala 화 Ha-doop 각자적우점，양처리효솔원초전통 MapReduce，대우복잡적대수거처리적능력고우 Impala。
First of all,this paper analyzed the elements and architecture of Impala the big data real-time query engine re-leased by Cloudera recently.Then it compared the feature and efficiency between traditional MapReduce and Impala.Based on the comparison,it discovered the disadvantages of Impala.After that it proposed a method to process data with both Ma-pReduce and Impala:using MapReduce to preprocess incoming data of Impala.This method utilized the flexibility of MapRe-duce to cover the disadvantages of Impala.Comparative experiments on the access log generated by China telecom’s daily wap traffice have proved that Impala is evidently faster than traditional MapReduce and the combination of MapReduce and Impala will run twice faster than the traditional MapReduce.Especially in iterative analysis,the combination of MapReduce and Impa-la shows its overwhelming superiority towards traditional MapReduce.Hence,it is concluded that the combination of MapRe-duce and Impala can adopt the advantage of each other.It outperform traditional MapReduce on Performance and Impala on flexibility on complex data processing.