Splunk

Splunk

NEC Corporation

Adopted Splunk for targeted cyber attack countermeasures. Log collection that used to take a week can be done in just a few minutes, strongly supporting global monitoring and analysis work by CSIRT

Before
  • Extensive Log Analysis Required to Investigate Cyberattack Details
  • ログの回収に2日~1週間かかり多角的な分析のための思考が中断
  • Fragmented monitoring of logs for reactive decision-making and action
Arrow: Horizontal
Arrow: vertical
After
  • Realized a platform for high-speed analysis of a large number of types of logs
  • ログの検索が数分で完了し多角的な調査作業の効率が格段に向上
Mr. Takeo Tagami

NEC Corporation Management System Headquarters
and Cyber Security Strategy Headquarters
senior expert Mr. Takeo Tagami

Mr. Shota Kizukuri

NEC Corporation Management System Headquarters
and Cyber Security Strategy Headquarters
Mr. Shota Kizukuri

NEC Focuses on Social Solutions Based on Safety, Security, Efficiency, and Fairness

In 1977, in anticipation of the advent of the Internet age, NEC Corporation (hereinafter referred to as NEC) proposed the concept of "C&C" (The Integration of Computers and Communications), which means the fusion of computers and communications. Aiming to realize a rich and prosperous society, we have promoted numerous world-first and domestic first technologies and research and development, and have also worked earnestly on management reforms that make these innovations possible. In 2014, 37 years after C&C, the new brand statement "Orchestrating abrighter world" was launched, leveraging the ICT assets cultivated in the company's more than 100-year history to promote "safety," "reliability," "efficiency," and "fairness." We have declared that we will focus on the "social solutions business" that advances social infrastructure in order to realize the four values. In the social solutions business, strengthening the cyber security area was an unavoidable top priority for providing systems and services.

Focus on Splunk from 5 or more candidates for high-speed visualization of large volumes of logs

In 2011, when targeted attacks targeting the Japanese government and specific companies increased and attack methods began to diversify, NEC formulated a targeted cyberattack countermeasure policy consisting of 20 items for the entire NEC Group. Among them, the first gateway-level countermeasure we worked on was the operation of an unknown malware detection system. NEC adopted an unknown threat detection and analysis system, which was introduced as a web countermeasure in March 2012, and then additionally introduced in December 2012 as a countermeasure against targeted e-mail attacks.

“At first, we gave top priority to developing a mechanism to detect unknown attacks. Once we were able to actually detect unknown attacks, it became necessary to investigate the range of attacks and the existence of unauthorized communications. In order to do that, we had to deal with a large amount of logs, and we needed a system that could visualize them at high speed,” explains Tagami, senior expert at NEC's Management Systems Division and Cyber Security Strategy Division. Takeo.

Therefore, the Management System Headquarters collected a wide range of information on SIEM (security information management and event management) and integrated log management systems, made a list of five or more products, and compared and examined their functions. As a result, what attracted the most attention was the machine data analysis platform "Splunk," which enables the collection, search, analysis, and visualization of data generated from various IT systems.

“Splunk has been a hot topic among security people for a long time, so I was interested in it. SIEM products are effective in real-time detection, but our purpose is to retroactively attack. We wanted to be able to visualize the presence or absence of data and to be able to analyze a large number of different types of logs at high speed, so we decided to seriously consider whether Splunk lived up to its reputation for making it possible,” says Tagami. Mr. looks back.

  • Figure 1. Business overview of NEC's CSIRT team

Figure 1. NEC's CSIRT team work overview.
The main mission is to prevent damage and bring the incident to an early end.

Splunk Turns Weekly Proxy Log Collection into Minutes

We conducted POC (ProofOf Concept) for about three months from September 2014, and the results were astonishing.

At NEC, proxy logs alone generate 100-odd GB per day, and at that time, a huge amount of logs had to be searched every day. was also on the rise. Since the proxy logs are managed by the in-house NOC (Network Operation Center), the NOC collects the logs for the required period, formats them so that they can be easily searched, and then delivers them. It takes about 2 days to 1 week. After that, the CSIRT (incident response team that specializes in information security issues; have to wait until Targeted cyber-attacks take action in minutes to hours, so it was extremely difficult to respond to log investigations in the conventional way.

However, with Splunk, it was demonstrated that the CSIRT itself could retrieve proxy logs in as little as a few minutes. Mr. Shota Kizo of NEC's Management System Division and Cyber Security Strategy Division analyzes the changes as follows.

“The CSIRT security analyst members have advanced analytical skills, and if they are unable to obtain any results, they repeat the process of hypothesizing and verifying, flexibly searching from a different angle. With this method, we had to ask the NOC to extract the log every time we changed the angle, and our thinking tended to be divided. In addition, since the number of man-hours on the NOC side has been greatly reduced, we were also able to expect a significant change in the operation process itself.”

Mr. Tagami also agrees with Mr. Kizo's view. “CSIRT captures a variety of logs without restrictions and conducts investigations and analyzes in a multifaceted and flexible manner, so it has a wide range of openings that can capture any kind of log, and big data support functions that can quickly search and visualize large volumes of logs. Splunk is excellent, and it was exactly the ideal tool for the CSIRT to use on a daily basis.”

  • Figure 2. “Zero” waiting time for proxy log analysis start

Figure 2. “Zero” waiting time for proxy log analysis start

Sufficient effect on cyber risk minimization calculated by investigation speed and CSIRT process improvement

Based on the results of the POC, it was decided that the proposal would be approved in December 2014. There were no particular objections during the drafting and approval process.On the contrary, when considering the risk of corporate information leakage in the face of increasingly sophisticated threats, there was an opinion that Splunk should be introduced immediately.” Mr. Taue says.

On the other hand, along with the introduction of Splunk, utilization of the network forensics appliance "Blue Coat Security Analytics" (Blue Coat SA) was also considered. Blue Coat SA is an actual packet analysis product that captures and classifies packets from L2 to L7, grasps the content of communications in real time, identifies information leaks and malware specimens, and grasps the details of attacks. Many things can be analyzed with logs alone, but payloads and specimens cannot be understood unless the packets themselves are collected, so it is more effective to use log search (Splunk) and network forensics (Blue Coat SA) in conjunction. judged to be targeted. Therefore, in December 2014, it was decided to use Splunk and Blue Coat SA as a set.

  • Figure 3. Basic flow of gateway countermeasures. After detection by Box etc., log analysis by Splunk and actual packet investigation by Blue Coat SA

Figure 3. Basic flow of gateway countermeasures. After detection by Box etc., log analysis by Splunk and actual packet investigation by Blue Coat SA

Accumulating knowledge and know-how within the CSIRT to realize the ideal OODA loop

The NEC Group has approximately 100,000 employees and approximately 180,000 terminals on a consolidated basis. "Since around 2000, NEC's basic policy for security management has been to 'manage only what can be counted', and the number of vulnerable servers, the number of patches applied, and the number of infected terminals are visualized as numbers. Although it has become established, it was not possible to visualize it numerically only in the network field.In that sense, it can be said that visualization by Splunk was necessary,” says Mr. Tagami.

Specifically, logs from proxies, firewalls, unknown threat detection systems, mail servers, spam filters, anti-virus software, etc. can be monitored and compared with external data to determine which organization manages IP addresses. It is searchable and displayable using Splunk's Look Up function. You can easily create dashboards and reports to visualize organizations detected by the unknown threat detection system, web access trends for each organization, destinations of suspicious free emails, changes in the number of targeted attacks detected, etc. Mr. Kizo says he can. “Results analyzed using Splunk’s dashboard and reporting functions are shared among CSI RT members, leading to new insights in other analyses.”

More than 50 types of dashboards have been created so far, among which "monitoring emails with malware" and "monitoring attached files with passwords" are used on a daily basis.

After the introduction of Splunk, there was a big change when actually operating the incident investigation process. Before the introduction, only by fragmentary monitoring (Observe) of each log such as proxy and Box and judging the situation (Orient), it leads to reactive decision-making (Decide) and action (Act), and countermeasures are reactive. However, knowledge and know-how dissipated within the CSIRT.

However, with the introduction of Splunk, proactive decision-making and actions based on integrated monitoring and assessment of the situation have enabled anticipatory countermeasures. OODA loop" can now be configured.

“the Company regularly holds cyber-attack countermeasures review meetings, but in the past we would take questions and issues that arose from the agenda and take them home and investigate them, deferring conclusions until the next meeting. can now conduct surveys and analyzes from the Splunk screen on a laptop computer within the time of the meeting, and speeding up decision-making has greatly changed the decision-making process, greatly improving work." (Mr. Tagami)

  • Figure 4. After introducing Splunk, the incident investigation process has greatly improved, and an efficient OODA loop (monitoring: Observe → Situation judgment: Orient → Decision making: Decide → Action: Act cycle) has been realized.

Figure 4. After introducing Splunk, the incident investigation process has greatly improved, and an efficient OODA loop (monitoring: Observe → Situation judgment: Orient → Decision making: Decide → Action: Act cycle) has been realized.

Expanding the use of Splunk globally Plans to call attention to the blacklist

In April 2016, NEC appointed a CISO (Chief Information Security Officer) separately from the existing CIO (Chief Information Officer) to strengthen its cybersecurity risk management system. While holding information security strategy meetings, information security promotion meetings, promotion meetings for business partners, etc., the CSIRT is placed directly under the CISO so that it can directly respond to security incidents. Currently, the SOC (Security Operations Center), NOC, and CSIRT can examine the logs while viewing the Splunk screen, and it is possible to draw conclusions in a matter of minutes. It shows dramatic improvement in terms of early convergence of incidents.

In the future, the possibility of expanding log monitoring operations using Splunk at overseas subsidiaries has also become visible. We have already begun offering Splunk as a Service for internal users at some of our local subsidiaries, and we plan to expand this method to other regions. Also, in the past, proxy blacklists were managed on a separate DB, but now Splunk's KVStore is used to utilize MongoDB to register and delete the blacklist database on the Splunk side. As a result, it has become possible to visualize in a more diverse way than ever before. By applying this method to the proxy logs of global local subsidiaries and consolidating them all in Splunk, it will be possible to effectively manage blacklists as global knowledge. In the event that communications related to the blacklist are sent from a local subsidiary proxy, we are partially developing a mechanism to automatically call attention, and we plan to complete full-scale operation by fiscal 2016.

In addition, NEC plans to utilize the know-how cultivated in NEC's in-house cyber attack countermeasure system "CAPS" in conjunction with Splunk in the threat and vulnerability information management solution "NEC Cyber Security Platform" (NCSP) developed for external sales. He said he has a plan. The use of Splunk is becoming more and more widespread.

  • Expanding the use of Splunk globally Plans to call attention to the blacklist

Governance Audit of Powerful CSIRT's Splunk Usage

Looking back on the recent introduction of Splunk, Mr. Kizo said, ``When I first started using Splunk, I was unfamiliar with it and only used it for searching, but with the guidance from Macnica 's SE and learning at hands-on seminars, I was able to improve my skills considerably. "The training system for certified engineers is also extensive, and I feel that this quality is unique to Macnica."

Mr. Tagami added, "In addition to being able to search large volumes of logs from a wide variety of detection systems at high speed and precision with Splunk, we are also very satisfied with the other side effects. Since CSI RT has strong authority over logs, strict regulations have been created to prevent unauthorized searches, and it is possible to audit who, when, and what logs were searched, and Splunk is a powerful We are also trying to manage the use of the system in terms of governance,” he explained.

Speaking to Macnica, Mr. Tagami said, ``I hope that you will continue to inform Japan about Splunk's new features and examples of its use overseas.This will expand the base of Japanese users and create new This should also generate new knowledge."

Splunk and Macnica will continue to support NEC's cybersecurity risk management, which continues to provide safe, secure, and efficient social solutions, with their global-level knowledge.

User Profile

NEC Corporation
URLs

http://jpn.nec.com/

Founded in 1899 as Japan's first joint venture with foreign capital (then Western Electric Co. of the United States). He started developing computers in 1954, and in 1977 proposed the concept of "C&C," which means the fusion of computers and communications. In 2014, we announced a new brand statement, "Orchestrating a brighter world," and a "social solutions business" that advances social infrastructure to realize the four social values of "safety," "peace of mind," "efficiency," and "fairness." focus on. Its main businesses are "public business", "enterprise business", "telecom carrier business" and "system platform business".

Inquiry/Document request

In charge of Macnica Splunk Co., Ltd.

Mon-Fri 8:45-17:30