Environment surrounding changing log analysis that cannot keep up with outsourced development

With the rapid spread of smart devices, content such as videos and music is becoming richer, and multi-device use is progressing rapidly. KDDI Corporation (hereafter, KDDI) is aiming to grow its domestic business with the "3M Strategy." We are contributing to the realization of a rich communication society by providing high-quality communication services and services with new social value, centered on our "global strategy" that aims to expand our overseas business.

The key to supporting the quality of such services is the analysis of a wide variety of logs output from networks, devices, cloud services, etc. Previously, KDDI utilized logs by introducing viewers and analysis tools that depended on outsourced individual development for each system. With such a method, there is a problem that the cost and period required for development swell, and even if a new problem occurs during operation, additional development cannot be done in time, and the system person in charge introduces a script for analysis independently. , It is said that there were many cases where it became inefficient due to individuality.

Satoshi Mizutani, assistant manager of Infrastructure Group 2, Platform Engineering Department, KDDI Platform Development Headquarters, says, ``This method is wasteful, such as incurring development costs and operational burdens, and it depends on the skills and experience of each person. There were major issues in terms of quality maintenance, time, and maintainability,” he recalls of the situation at the time.

In order to solve this problem, it was necessary to build a system that collects the logs of each system in one place, centrally visualizes and analyzes them, and develops an in-house analysis environment. It is said that this led to the birth of the common log platform.

The log common platform is designed to achieve scalability that can handle increasing logs through scale-out, cost efficiency and autonomy through batch construction and in-house development, and speedy and highly executable analysis and visualization. , based on the open-source distributed processing middleware "Hadoop", aiming to be easy to use by simply inputting logs into Hadoop without special skills.

Hadoop analysis platform "Hunk" that satisfies versatility, ease and flexibility

Although the common log platform was built in June 2012, an analysis tool suitable for its characteristics became essential for actual operation. There are three conditions for selection. The first is versatility. In order to perform log analysis of various systems, an almighty design that does not require designing for individual systems was required. The second is ease of use. It was important that anyone in charge of the system could use it easily, so that it could be developed with a sense of speed, even if it was in-house. The third is flexibility. The purpose of analysis often changed during operation, and it was necessary for the user to be able to assemble the analysis logic freely.

After comparing various tools based on these criteria, KDDI chose "Hunk," a Hadoop big data analysis platform from Splunk Inc., a US Macnica (hereinafter referred to as MNC). Users can directly access data stored in Hadoop, and intuitive operation and flexible searches via a web browser make work more efficient and faster. In addition, it offers a wide range of functions to support data analysis, such as analysis result alerts, reports, and dashboards.

Mr. Mizutani says that Hunk met three conditions. “No design is required, just putting logs into the Company enables flexible new analysis and development within the scope of roles (authorities) assigned to users. Since the operation feeling of Hunk is similar to that of Hunk, I felt that it was perfect for Splunk users in the company to use as it is, and it has a high affinity."

Introduced Hunk in October 2014. The first use was to build a system that could visualize the system resource status and demand forecast in real time in a large-scale system that generates terabytes of logs per day.

Mr. Mizutani first introduced Hunk into the verification environment and wrote a simple search sentence according to the tutorial manual. "It's easy to connect Hadoop and Hunk, and the operation method is the same as Splunk, so I felt that I could freely interact with raw log data on Hadoop."

After that, Mr. Mizutani launched a joint project with Splunk, MNC, and an in-house system manager, and studied Hunk configuration, search acceleration methods, Hunk server sizing, and so on. Also, using the professional service contract period with Splunk, we learned the know-how of efficient dashboard development, and created dashboards with in-house development. The number has reached nearly 100.

Hunk, which is suitable even for beginners in analysis, has the speed to keep up with changes.

In mid-December 2014, we released the production environment for the system resource/usage visualization system.

Looking back on the development, Mr. Mizutani said, "The intuitive Hunk is ideal for analysis beginners. In addition, we have developed a speed that can keep up with new systems for which problems have not yet been identified, and systems where the content to be analyzed changes moment by moment. Since it is a tool that I have, I recommend agile development while reviewing it," he says.

By introducing Hunk to the log common platform, it is possible to visualize the resource status and traffic status and avoid unforeseen system problems.

In addition, KDDI and MNC, as well as MITSUBISHI ELECTRIC INFORMATION SYSTEMS CORPORATION, Ltd., which provides the Hadoop distribution "MapR" adopted for the common log platform, aims to further improve the performance of the common log platform, By continuing to share information and know-how, we are pursuing best practices in combining Hunk and MapR.

“Based on that information, we will focus on developing human resources who can use Splunk and human resources who can use Hunk + Hadoop, create a hybrid environment with Hunk + Splunk, and aim to strengthen our analytical capabilities by complementing each other.”

In the past, the system side developed all analysis platforms and dashboards, but in the future, Hadoop and Hunk will be provided on the log common platform side, and the system side will develop dashboards and analysis methods. Mr. Mizutani sees that efficient analysis will be possible with

“I really like Hunk's 'hit-it-it-it-it-it' feel, which generates graphs and map plots right from the logs, and I'm looking forward to exploring the best ways to do deep analysis on large amounts of data.”

Together with Splunk, MNC will continue to fully support KDDI, which is said to be a leading company in using Hadoop in Japan, in its relentless pursuit.

User Profile

045-476-2010
月～金 8：45～17：30

inquiry
Please feel free to contact us

Free seminar
Click here for various seminars

Document download
Click here for various materials