CAJ | 학술논문

为满足气象实时资料传输中大量气象数据小文件的高时效传输需求，对其中的数据传输服务进行优化，提出一种基于实时网络状况的自适应数据传输优化方法。该方法采用优化网络传输协议并使用文件压缩技术，通过获取网络传输线路上的实时参数，实时调整压缩参数和网络传输参数以优化传输性能。在自适应压缩时，通过试验分析和归纳，确立了气象数据小文件标准为文件小于50 KB；根据网络实时状况，设计了基于实时网络状况自适应调整压缩等级的算法。在自适应传输参数调优中，研究了 TCP 缓冲区大小和 TCP 并发连接数在 GridFTP 协议中的重要性，针对实时网络状况，分别设计了自适应调整 TCP 缓冲区大小和 TCP 并发连接数的算法，算法提升传输性能65％。对以上提出各自适应参数调整算法进行试验验证表明，融合压缩和网络传输的自适应调优方法能显著提升气象小数据文件的传输性能近500倍。
위만족기상실시자료전수중대량기상수거소문건적고시효전수수구，대기중적수거전수복무진행우화，제출일충기우실시망락상황적자괄응수거전수우화방법。해방법채용우화망락전수협의병사용문건압축기술，통과획취망락전수선로상적실시삼수，실시조정압축삼수화망락전수삼수이우화전수성능。재자괄응압축시，통과시험분석화귀납，학립료기상수거소문건표준위문건소우50 KB；근거망락실시상황，설계료기우실시망락상황자괄응조정압축등급적산법。재자괄응전수삼수조우중，연구료 TCP 완충구대소화 TCP 병발련접수재 GridFTP 협의중적중요성，침대실시망락상황，분별설계료자괄응조정 TCP 완충구대소화 TCP 병발련접수적산법，산법제승전수성능65％。대이상제출각자괄응삼수조정산법진행시험험증표명，융합압축화망락전수적자괄응조우방법능현저제승기상소수거문건적전수성능근500배。
The data transfer and service architecture constructed by National Meteorological Information Center is the fundament for most meteorological data transmission.How to improve the timeliness of transmission of various data is a hot topic to enhance capabilities of meteorological services. <br> According to requirements of transmission performance of massive small files,transmission parame-ters are optimized.And a self-adapting data transmission method is proposed based on real-time network status,which emphasizes network transmission protocol and file compression.Compression parameters and network transmission parameters are adjusted in real-time operation. <br> Meteorological data include a great amount of heterogeneous small files,therefore compressing small files into a big file when being transformed will effectively reduce I/O accesses.First,50 KB is defined as the threshold for small meteorological data files through experiments.Then,by analyzing the file transfer time,the appropriate file amount in compressed packages is calculated to achieve the best transmission effi-ciency.Finally,considering the variability of network conditions and real-time network conditions,a self-adapting compression methods based on real-network is designed by means of real-time adjusting the com-pression level.This entire compression process is controlled by setting various parameters of lzop com-mands on the basis of the lzop algorithm library and the LZO algorithm.To achieve the goal of adjusting compression levels according to real-time network conditions,RTT (round trip time)is taken advantage of judging the current state of the network congestion.By comparing current RTT and previous RTT,chan-ging the compression level or not is decided. <br> In network transmission optimization,conclusions are made that TCP buffer and parallel transmission will consume memory resources according to experiments in Globus platform.At the same time,more par-allel streams and larger size of TCP buffers will result in network congestion.Then,the self-adapting ad-justment algorithm of TCP buffer size and the concurrent connection number algorithm of TCP based on real-network parameters are designed.Finally,the entire transmission framework of massive small files is designed by combining self-adapting compression method and transmission parameters optimization.Com-plete experiments are carried out based on the integration of self-adapting algorithm,showing that pro-posed optimization methods can improve the transmission performance sharply.