电网技术
電網技術
전망기술
Power System Technology
2015年
11期
3221-3227
,共7页
电力监测%云计算%容错%查询优化
電力鑑測%雲計算%容錯%查詢優化
전력감측%운계산%용착%사순우화
power monitoring%cloud computing%fault- tolerance%query optimization
针对电力系统监测中大量时间序列数据存储和高效查询的问题,利用云计算框架和HQL(Hive SQL)查询引擎,提出一种容错存储的分级分区查询优化方法。通过副本机制设计电力监测数据容错存储模式,综合运用了HQL查询计划生成、向Map/Reduce的转化和分区剪枝处理,进行了加载和查询优化测试。结果表明,当加载的监测数据记录超过200万条、查询数据记录超过380万条后,HQL的处理性能将远超SQL,数据量越大,优势越明显。分级分区查询测试结果表明,在查询耗时相近的条件下,分区查询的数据量可以扩大2个数量级,且二级分区比一级分区更高效,验证了查询优化技术可有效提高电力系统监测信息查询处理的效率,为大量电力监测数据处理提供了一种查询优化方法。
針對電力繫統鑑測中大量時間序列數據存儲和高效查詢的問題,利用雲計算框架和HQL(Hive SQL)查詢引擎,提齣一種容錯存儲的分級分區查詢優化方法。通過副本機製設計電力鑑測數據容錯存儲模式,綜閤運用瞭HQL查詢計劃生成、嚮Map/Reduce的轉化和分區剪枝處理,進行瞭加載和查詢優化測試。結果錶明,噹加載的鑑測數據記錄超過200萬條、查詢數據記錄超過380萬條後,HQL的處理性能將遠超SQL,數據量越大,優勢越明顯。分級分區查詢測試結果錶明,在查詢耗時相近的條件下,分區查詢的數據量可以擴大2箇數量級,且二級分區比一級分區更高效,驗證瞭查詢優化技術可有效提高電力繫統鑑測信息查詢處理的效率,為大量電力鑑測數據處理提供瞭一種查詢優化方法。
침대전력계통감측중대량시간서렬수거존저화고효사순적문제,이용운계산광가화HQL(Hive SQL)사순인경,제출일충용착존저적분급분구사순우화방법。통과부본궤제설계전력감측수거용착존저모식,종합운용료HQL사순계화생성、향Map/Reduce적전화화분구전지처리,진행료가재화사순우화측시。결과표명,당가재적감측수거기록초과200만조、사순수거기록초과380만조후,HQL적처이성능장원초SQL,수거량월대,우세월명현。분급분구사순측시결과표명,재사순모시상근적조건하,분구사순적수거량가이확대2개수량급,차이급분구비일급분구경고효,험증료사순우화기술가유효제고전력계통감측신식사순처리적효솔,위대량전력감측수거처리제공료일충사순우화방법。
In order to solve massive time series data storage and query problem in power system monitoring, a new classified partition query optimization method of fault tolerant storage is proposed using cloud computing framework and HQL query engine. Fault tolerant storage model of power system monitoring data is designed through redundancy replication mechanism, and loading test and query optimization test of monitoring data are carried out by coordinating HQL query plan generation, conversion from HQL to Map/Reduce and partition pruning process. Results show that HQL performance is better than SQL when amounts of loading data exceed two millions or amounts of query data exceed three hundred and eighty thousand, and the bigger the data volume is, the more obvious the performance advantage will be. Classified partition query results show that amounts of data for partition query can be expanded to two orders of magnitude under similar time conditions, and second-level partition is better than first-level partition. It is verified that query optimization can improve query effectiveness, and provide a query optimization method for power monitoring data processing.