CAJ | 학술논문

随着互联网的普及，社交网络已经成为人们生活中至关重要的一部分。这种新媒体时代的潮流促进了信息的流动和传播，同时也带来了海量的媒体内容与用户数据。社交媒体分析是舆情监控系统的主要内容，舆情数据的分析、处理与监控是新媒体时代带来的新的技术问题之一。近年来的大数据处理计算技术提供了处理海量数据的成熟解决方案。大数据处理平台有很多种，其中Hadoop具有成熟的社区，其架构稳定且易于使用。在文本分类方面，LDA统计模型方法给文本分类问题带来了新的处理办法。因此，文章提出了一个基于成熟开源架构的舆情监控系统，系统基于Hadoop平台，以Nutch作为爬虫，使用Solr实现核心的索引查找功能。整个平台在海量数据处理方面展现了较高的分析处理效率，在应对海量数据带来的问题的同时还提供了智能的分析与统计功能。
수착호련망적보급，사교망락이경성위인문생활중지관중요적일부분。저충신매체시대적조류촉진료신식적류동화전파，동시야대래료해량적매체내용여용호수거。사교매체분석시여정감공계통적주요내용，여정수거적분석、처리여감공시신매체시대대래적신적기술문제지일。근년래적대수거처리계산기술제공료처리해량수거적성숙해결방안。대수거처리평태유흔다충，기중Hadoop구유성숙적사구，기가구은정차역우사용。재문본분류방면，LDA통계모형방법급문본분류문제대래료신적처리판법。인차，문장제출료일개기우성숙개원가구적여정감공계통，계통기우Hadoop평태，이Nutch작위파충，사용Solr실현핵심적색인사조공능。정개평태재해량수거처리방면전현료교고적분석처리효솔，재응대해량수거대래적문제적동시환제공료지능적분석여통계공능。
With the popularization of Internet, social network has become a vital part of people's lives. Social media promotes lfow and dissemination of information, but also brings a deluge of social media data and user data. Social media analysis is the main component of public opinion monitor system. Analysing and monitoring of public opinion data is one of the new technical problems caused by media in this era. In recent years, new technology such as big data processing provides proven solution to cope with the massive data . There are many big data processing platforms, in which Hadoop platform has a mature community and its structure is stable and easy to use. To text classiifcation problems, LDA statistical modeling brings a new approach. Therefore, this paper proposes a public opinion monitoring system based on proven open source architectures. The system bases on Hadoop platform, with Nutch as a crawler, using Solr to achieve the core index search function. The entire platform demonstrates its high efifciency in the mass data processing analysis. while providing intelligent analysis and statistical functions in response to the problems caused by massive amounts of data.