CAJ | 학술논문

随着分布式数据环境越来越复杂,ETL 工具要面临数据源多、分布地域广和海量数据等因素带来的挑战.原有的集中式 ETL 工作流优化理论不能满足现在复杂数据环境的要求.介绍了如何将基于置换的离散型粒子群算法应用到分布式 ETL 任务优化调度问题上,主要工作围绕 ETL 工作调度模型、算法编码设计、目标函数选择等内容来展开,给出了分布式 ETL 工作调度策略的实现过程和伪代码.理论分析和实验证明了实际应用的有效可行性.
수착분포식수거배경월래월복잡,ETL 공구요면림수거원다、분포지역엄화해량수거등인소대래적도전.원유적집중식 ETL 공작류우화이론불능만족현재복잡수거배경적요구.개소료여하장기우치환적리산형입자군산법응용도분포식 ETL 임무우화조도문제상,주요공작위요 ETL 공작조도모형、산법편마설계、목표함수선택등내용래전개,급출료분포식 ETL 공작조도책략적실현과정화위대마.이론분석화실험증명료실제응용적유효가행성.
With the increasing complexity of distributed data environment, ETL tools face the challenge of many data sources, geographic distribution, massive data and other factors. The original centralized ETL workflow optimization theory can not meet the demands of the environment of the complex data. This paper presents how the discrete particle swarm optimization based on replacement is used in task scheduling of the distributed ETL. The main contents include the abstraction of the ETL task schedul-ing model, design of the algorithm coding, selection of objective function and so on. The realization and pseudocode of distributed ETL job scheduling strategy are also mentioned. The theory and experiment prove it to be feasible and efficient.