CAJ | 학술논문

为应对用电大数据的处理要求，基于 Hadoop 集群构建了面向用电信息采集系统大数据的处理架构和计算服务架构，对计算任务进行并行化处理，提高了用电大数据的计算效率；同时针对用电信息采集系统中大部分计算服务的计算规则复杂、存在数据依赖关系等特点，将同一计算服务内的 MapReduce 作业及及其依赖关系以有向无环图的形式进行组织，并利用依赖控制引擎进行作业运行状态的自动管理，实现作业提交时间的动态调整，达到优化作业执行效率和成功率的效果。实验结果证明，所提方法高效可行，计算服务性能高，能够适应用电大数据的实时处理要求。
위응대용전대수거적처리요구，기우 Hadoop 집군구건료면향용전신식채집계통대수거적처리가구화계산복무가구，대계산임무진행병행화처리，제고료용전대수거적계산효솔；동시침대용전신식채집계통중대부분계산복무적계산규칙복잡、존재수거의뢰관계등특점，장동일계산복무내적 MapReduce 작업급급기의뢰관계이유향무배도적형식진행조직，병이용의뢰공제인경진행작업운행상태적자동관리，실현작업제교시간적동태조정，체도우화작업집행효솔화성공솔적효과。실험결과증명，소제방법고효가행，계산복무성능고，능구괄응용전대수거적실시처리요구。
In response to processing requirements of large amounts of power data, processing and computing architectures are constructed on Hadoop for electricity big data. It uses parallel processing for computation tasks to improve calculation efficiency of power big data. And for complex calculation rules, data dependency, and other characteristics of computing services in electric energy data acquire system, MapReduce jobs and their dependencies in the same computation service are organized in directed acyclic graph form. Using dependency control engine, it performs automatic management of job operation state, and realizes dynamic adjustment of job submission time, to achieve optimization of job execution efficiency and success rate. Experimental results show that the proposed method is feasible and adaptive to real-time processing requirements of power big data with high computation service performance.