Use Splunk Enterprise for everything from alert confirmation to resource monitoring

SB Payment Service Corp., Ltd., which provides various financial services such as settlement agency, operates a mechanism that links merchants such as EC operators and each payment backbone system. We operate a large number of front application servers and DB servers. Splunk Enterprise was introduced in 2010 as a platform for efficiently collecting and analyzing these server logs, and has been in operation for several years, says Mr. Akira Suzuki, Operation Planning Section, System Operation Section, System Headquarters. "We chose Splunk Enterprise with the de facto standard system in mind. At the time of the survey, information was stored in Splunk without collecting logs, and the necessary information could be quickly found using SPL, a powerful search command." We appreciated the accessibility," says Suzuki.

Initially, Splunk Enterprise was mainly used for checking alerts received from log monitoring, but we decided to extend it as a real-time monitoring tool for sources such as CPU. “All the resource information to be monitored was text, and it was difficult to visualize. Therefore, we decided to log the server resources, aggregate them in Splunk Enterprise, and graph them on the dashboard screen.” It has become easier to monitor work in the production environment, and it is evaluated that abnormalities are no longer overlooked. After that, in 2017, a project to visualize it started. “It is true that we have become able to understand resources, but we also need to understand events in the entire system, such as when services become unstable. ”, Suzuki recalls.

I want to break away from “personalization” by scratch development

Mr. Suzuki thought of creating an environment that would allow visualization and analysis of the entire service using Splunk Enterprise, which was used as a base for log collection. In fact, we initially considered developing it from scratch and actually worked on it. However, I am concerned that people who are familiar with Splunk will not know about it, and that it will be too difficult for members who are less familiar with it to use, and that it may become a matter of individual skill. At that time, Macnica, which provides implementation support for Splunk, introduced Splunk IT Service Intelligence, which enables monitoring and analysis that increases the visibility of IT services and business services as a whole. "After actually doing hands-on work, we realized that the functionality we wanted could be implemented simply by adding an add-on. We thought it would be cheaper to implement than developing it from scratch, and we could create a system that was easy for anyone to use. ” said Suzuki.

Powerful backup for installation with extensive experience

For Splunk IT Service Intelligence, we created a system map called Glass Table, which shows the relationships between servers collecting logs by setting thresholds for items to be monitored for each system, using a GUI. On the dashboard, the system status can be visualized in a color-coded form, and even non-experts can see at a glance where the problem is, which has succeeded in preventing personalization. Various logs are collected in real time by the Splunk forwarder, and about 100 GB of logs are retained for several days per day, and some are archived and then discarded sequentially. In the unlikely event that an anomaly occurs, it is possible to perform correlation analysis in time series by drilling down to Deep Dive where correlation analysis is possible. In addition, detailed analysis is possible from Splunk IT ServiceIntelligence by linking with other companies' products such as application performance management tools.

SplunkEnterprise itself is evaluated not only for its flexibility to disassemble and use data later even if it is input without prior processing, and the ease of use of SPL, a search processing language that is easy for general engineers to understand. SplunkIT Service Intelligence, which can grasp the state of the system visually even for those who are not familiar with the system, such as employees and management, is highly evaluated. Mr. Suzuki says that since it is possible to check in chronological order with Deep Dive, it is possible to communicate with others while sharing information, making it easier to look at each other. In addition, we cannot overlook the fact that advanced analysis, including time series analysis, can be performed in one hand.

For this project, Macnica, which handles many Splunk solutions, is providing support in a variety of ways, including proposals for Splunk IT Service Intelligence, holding hands-on sessions, support for on-site implementation, and operational support. "Macnica has many highly skilled engineers with extensive Splunk experience, and they were able to help us with version upgrades without any trouble. They answered detailed questions accurately, and we implemented Splunk IT Service Intelligence without any problems. I am grateful that I was able to do this," said Suzuki. Because of the company's close relationships with manufacturers, requests for improvements to Splunk are easily received, and in some cases it has actually led to improvements. "For the company as a whole, we are also relying on solutions other than Splunk. We are grateful that they are helping us with our challenges in a variety of ways," said Suzuki.

AI活用によってさらなる精度向上を目指す

In the future, he hopes to expand the range of logs collected to include network devices, so that even when troubles are caused by the network part, he wants to be able to look in detail. He also has high expectations for the machine learning functions of Splunk IT ServiceIntelligence, and hopes to successfully incorporate them into operations. "The KPIs set for each system are derived from our know-how, but if machine learning can be used to make fine adjustments, we should be able to further improve accuracy. We would like to further advance the visualization of the overall system status." That's right," he said about the future.

User Profile

045-476-2010
月～金 8：45～17：30

inquiry
Please feel free to contact us

Free seminar
Click here for various seminars

Document download
Click here for various materials