CAJ | 학술논문

针对目前聚类算法对大数据处理效率较低的问题，研究云平台上高效并行化的聚类算法十分必要。在HDFS分布式文件系统基础上，设计一种并行聚类算法P-ISODATA，利用MapReduce编程框架的执行机制将传统ISODATA聚类算法并行化，并在Map阶段之后加入Combine阶段以减少网络传输开销，进一步提高执行效率。实验从著名UCI机器学习库上选取若干数据集作为测试数据，分析了新并行算法P-ISODATA性能，结果表明基于P-ISODATA算法具有优良的加速比、数据伸缩率和扩展率，可以有效地应用于大规模数据的处理。
침대목전취류산법대대수거처리효솔교저적문제，연구운평태상고효병행화적취류산법십분필요。재HDFS분포식문건계통기출상，설계일충병행취류산법P-ISODATA，이용MapReduce편정광가적집행궤제장전통ISODATA취류산법병행화，병재Map계단지후가입Combine계단이감소망락전수개소，진일보제고집행효솔。실험종저명UCI궤기학습고상선취약간수거집작위측시수거，분석료신병행산법P-ISODATA성능，결과표명기우P-ISODATA산법구유우량적가속비、수거신축솔화확전솔，가이유효지응용우대규모수거적처리。
In light of the problem that current clustering algorithms are inefficient in efficiency when processing big data, it is necessary to study efficient parallel clustering algorithms on cloud platform.We design a new parallel clustering algorithm P-ISODATA based on HDFS ( Hadoop distributed file system) , it parallelises traditional ISODATA clustering algorithm by utilising the execution mechanism of MapReduce programming framework, and introduces Combine phase following behind the Map phase to reduce network transmission overhead and to further improve the execution efficiency.In experiments we select different datasets from famous UCI machine learning repository as the testing data, and analyse the performance of P-ISODATA algorithm.Results show that P-ISODATA-based algorithm has good speed-up ratio, data scale-up rate and size-up ratio, therefore it can be effectively applied to large-scale data processing.