Design and Implementation of Log Analysis System Based on Hadoop/CloudBase/MySQL

ZHUO Haiyi

1

卓海藝(1987-)，男，碩士，主要研究方向：云計算、網絡安全

ZHAO Wenshen

1

趙文深(1954-)，男，副教授，主要研究方向：智能信息系統、云計算

LV Yuqin

1

呂玉琴，女，教授，主要研究方向：通信協議，網絡信息安全

HOU Bin

1

侯賓，男，講師，主要研究方向：云計算，網絡安全

1、School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876

Abstract：With the rapid development of SNS and mobile internet, the network log data of many internet companies and institutions is becoming bigger and bigger. The problem of how to processing large-scale log data quickly and deriving useful information for better serveice become a urgent problem which need to be solved. Cloud computing and data warehouse technology provides a way to solve this problem. Hadoop is open source framework which is developed by Apache and is widely used in massive data processing. CloudBase is a data warehouse software which is based on Hadoop. It can be used for large scale data analysis,. It is very convenient to query and analyse the large-scale log data by using the ANSI SQL. We have designed and implemented a distributed network log analysis system based on Hadoop and CloudBase and MySQL. It fetchs the network log data regularly from the backend server and processes it, then puts processing results into MySQL for diagram showing on the web page. Such a system can facilitate enterprises and institutions for understanding of user needs and then providing better services for customers. We demonstrate the effectiveness and the worth of our system by evaluating its performance.

總結

以上是生活随笔為你收集整理的mysql实现日志系统_基于Hadoop/CloudBase/MySQL的日志分析系统的设计与实现的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。