Introduction

This article explains the Security Data Pipeline Platform (hereinafter referred to as SDPP).
We will also introduce Cribl, a leading product in the SDPP industry.

Why has the SDPP field emerged?

First, let me explain why SDPP has emerged.
The rise of SDPP is influenced by the challenges of using SIEM.
SIEM has traditionally stored and analyzed all types of data within a company, playing an essential role in corporate security.
However, the amount of data stored and analyzed by SIEMs is increasing year by year, bringing to light issues such as "inflated SIEM costs" and "increasing complexity of analysis."
We also see many cases of SIEM vendor lock-in.

To address these challenges, SDPP products offer benefits such as "SIEM cost optimization," "data optimization for analysis," and "facilitated SIEM migration."

What is SDPP's leading product, Cribl?

Cribl is a leading SDPP product, offering multiple modules including Cribl Stream/Edge/Lake/Search.
This time, we will focus on Cribl Stream, Cribl's main feature.
Key features of Cribl Stream include:

• Data processing/filtering function
  • GUI-based data processing/filtering of unnecessary data
  • Parsing on the SDPP side into the format expected by the SIEM platform
  • Add a new field on the SDPP side
• Data routing function
  • Transfer data from multiple sources to multiple SIEMs/storages
  • A use case is to transfer data required for analysis to SIEM and raw logs to low-cost storage.
• Replay function from storage
  • Retransmitting raw logs from storage when data is lost to the analysis platform
• Vendor-free configuration
  • Vendor-free integration with multiple SIEM/storage products
  • Provides templates for linking with each product

Data Pipeline Challenges and Cribl's Benefits

Data pipelines are becoming increasingly complex for many companies today.
Pipelines are built and operated for each data source, which tends to increase costs and labor.
An image diagram is shown below.

If you introduce Cribl to address these issues, the configuration will look like this:

The advantages of this configuration are as follows:

• Each data pipeline can be managed by Cribl alone.
  • Pipeline configuration and linking with each source/destination is possible through GUI
  • Easily integrate with additional data sources
• Ability to route data to multiple destinations
  • Transfer only the data required for analysis to the SIEM, reducing costs
  • Transfer raw logs to low-cost storage and store them for a certain period of time
• Data processing/filtering on Cribl
  • Reduce SIEM costs by reducing unnecessary data
  • Transfer data in the format expected by your SIEM
  • SIEM analysis advancement

Summary

This time, I mainly explained the overview of Cribl, a leading product in the SDPP field.
Cribl has a variety of functions, so if you are interested, please feel free to contact us.