How to Detect System Problems More Efficiently? easylog – A Centralized Log Management System
easylog – A Centralized Log Management System
Author: Henry Chou
The system log records the events taking place in the system. If a problem occurs, you can search by keyword or debug. For a general MIS system with a limited system scope, this works just fine. But if the system architecture is getting larger and larger, the system log generated per day may be higher than ten terabytes. In the case of a distributed system architecture, system components are installed everywhere and include network facilities, AP platform, and the front-end platform as well as a database. That makes the collection and analysis of data much more challenging. For example, the IT departments handling online banking must monitor customer complaints and system problems daily. Especially in urgent matters like a wrong transaction in a customer’s account, finding and fixing the reason in the data mass within the log is getting difficult.
In response to this problem, ThinkPower Information created easylog according to the customer needs and system requirements. easylog integrates log files for distributed access problems and can so quickly respond to system errors and efficiently handle online problems. In the past, there were no set rules for the method and content of the log storage. Finding logs manually was very time consuming and the amount of data was huge. If problems occur, the IT staff is responsible and has a very high workload.
Datalog tracking can be executed in the following four steps: complete analysis, management requirements, query/analysis and fulfillment of future expansion requirements. Collecting all data sources, including Netcom equipment, the system, intermediary platform, AP server, and the database, etc. Management requirements also include authority control, AD/LDAP/Oauth2 management, group management, system monitoring requirements, and system alerts. The query and analytic features include system response performance, incident retrieval, full-text search, cross-analysis and export of results. For future expansion, it must meet the system needs and the storage capacity expansion and business growth requirements.
For a log management system, the following five features are very important: integration, urgency, normalization, centralization, and analysis needs. A comprehensive log management tool should be able to collect multiple sources and to provide real-time data collection. It is also necessary to analyze and integrate the log, but it must be noted, that the log management system should be normalized, so that the current status can be checked and analyzed by defined rules. The centralized log can manage all the massive data. Due to effective analysis tools, it can also provide multi-angle analysis, data retrieval, and cross-search.
With easylog, all machines and systems that generate logs are stored in a centralized management platform in accordance with standardized storage rules. It can integrate a variety of equipment and systems and can perform authority control based on AD account secrets. The information can be more comprehensive. A comprehensive log management platform can perform single centralized management, full-text search of keywords, big data analysis, analyze customer behavior and error trajectories, and effectively improve the availability of the system. Last but not least, a log management system can provide enhanced security measurements. Due to the requirements of personal information laws, system personnel must bear the burden of proof in case of customer data is leakage. A complete log analysis tool can provide this proof and close an important security gap.