计算机工程
計算機工程
계산궤공정
COMPUTER ENGINEERING
2015年
2期
21-25
,共5页
陈洁%于永刚%刘明恒%潘盛合%徐克付
陳潔%于永剛%劉明恆%潘盛閤%徐剋付
진길%우영강%류명항%반성합%서극부
安全管理平台%云计算%Hadoop架构%海量日志%并行处理
安全管理平檯%雲計算%Hadoop架構%海量日誌%併行處理
안전관리평태%운계산%Hadoop가구%해량일지%병행처리
Security Management Platform ( SMP )%cloud computing%Hadoop architecture%massive log%parallel processing
安全管理平台( SMP)是实现安全管理工作常态化运行的技术支撑平台,在实际应用中需要实时处理来自安全设备所产生的海量日志信息。为解决现有SMP中海量日志查询效率低下的问题,设计基于云计算的SMP日志存储分析系统。基于Hive的任务转化模式,利用Hadoop架构的分布式文件系统和MapReduce并行编程模型,实现海量SMP日志的有效存储与查询。实验结果表明,与基于关系数据的多表关联查询方法相比,该系统使得SMP日志的平均查询效率提高约90%,并能加快SMP集中管控的整体响应速度。
安全管理平檯( SMP)是實現安全管理工作常態化運行的技術支撐平檯,在實際應用中需要實時處理來自安全設備所產生的海量日誌信息。為解決現有SMP中海量日誌查詢效率低下的問題,設計基于雲計算的SMP日誌存儲分析繫統。基于Hive的任務轉化模式,利用Hadoop架構的分佈式文件繫統和MapReduce併行編程模型,實現海量SMP日誌的有效存儲與查詢。實驗結果錶明,與基于關繫數據的多錶關聯查詢方法相比,該繫統使得SMP日誌的平均查詢效率提高約90%,併能加快SMP集中管控的整體響應速度。
안전관리평태( SMP)시실현안전관리공작상태화운행적기술지탱평태,재실제응용중수요실시처리래자안전설비소산생적해량일지신식。위해결현유SMP중해량일지사순효솔저하적문제,설계기우운계산적SMP일지존저분석계통。기우Hive적임무전화모식,이용Hadoop가구적분포식문건계통화MapReduce병행편정모형,실현해량SMP일지적유효존저여사순。실험결과표명,여기우관계수거적다표관련사순방법상비,해계통사득SMP일지적평균사순효솔제고약90%,병능가쾌SMP집중관공적정체향응속도。
With the development of cloud computing,massive data can be very easy to be stored and managed. Security Management Platform( SMP) is a support platform which realizes security management normalized operation. In a real application,this platform needs to process the massive information which generates from security device in real time. Considering the problem of low query efficiency,an efficient log analysis system based on the cloud computing for SMP is presented. It introduces the Hadoop distributed system infrastructure, and in the meantime, based on the study of transformation mission of the Hive,Hadoop Distributed File System(HDFS) and MapReduce are applied to effective storage and query of massive log. Experimental results show that,using proposed system can obtain a general increase in the query performance by about 90% compared with the existing Oracle storage method,and it can also further improve response speed of the SMP.