*This article is based on a lecture given at the Macnica Data・AI Forum 2024 Autumn held in October 2024.
Introduction
Recently, the evolution of generative AI has led to improved business efficiency and the creation of new value. However, ignoring the potential risks of AI can cause serious problems in terms of security and reliability. In this article, we will introduce the risks that may emerge when using generative AI and how to deal with them, based on the AI TRiSM (Trust, Risk, Security Management) framework. We will provide specific advice to help your business use AI technology safely and efficiently.
The risks of LLM
1. The risks of prompting
When using generative AI's LLM (large-scale language model), there is a risk that confidential information contained in the prompts may be unintentionally leaked to the outside. In particular, there have been reported cases where source code and internal API information have been leaked to the outside when sent to AI chatbots. In such cases, the information may be misused by malicious attackers, which may lead to legal trouble.
2. Prompt Injection
Prompt injection is an attack method that causes an AI model to behave unexpectedly by giving it unauthorized instructions. A specific example is a case where, despite settings to protect private key information, the restrictions are circumvented by changing the language of the prompt, resulting in information leakage.
3. The risks of fine-tuning
Fine-tuning is a retraining technique to improve the output for a specific domain, but it also comes with risks. If the training data contains personal information, that information may be leaked as the output of the AI chatbot. The risk of personal information leaks increases even more if the data is shared externally.
Risk countermeasures based on AI TRiSM
AI TRiSM is a framework proposed by Gartner, Inc., and is an important concept for increasing the reliability of AI. Based on this, we will introduce the following latest solutions as specific risk countermeasures.
3-1. Text Anonymization (Private AI)
Private AI is a text anonymization solution that automatically detects personal information contained in text data and replaces it with masked or pseudo data. Unlike conventional regular expression-based methods, it uses machine learning to identify personal information from the context and respond with high accuracy. As a result, personal information contained in the generative AI prompt is anonymized and sent to LLM in a secure state.
3-2. Security & Trust Platform (DeepKeep)
DeepKeep is a platform for solving security risks and reliability issues related to generative AI and computer vision models. Specifically, it has the following four functions:
- **risk assessment**
Assess models and data for vulnerabilities with automated penetration testing. - **prevention**
Based on the results of the risk assessment, specific measures to strengthen security are presented. - **detection**
Real-time threat detection and anomaly detection allow for immediate response. - **Mitigation**
The AI firewall quickly restricts access and issues alerts when threats are detected.
These functions help ensure that AI systems are resistant to external attacks and unauthorized use of data, and are operated safely.
Summary
This article explained the risks associated with using generative AI, the AI TRiSM framework proposed as a countermeasure, and specific solutions. As AI is increasingly being introduced, it is important to operate it while ensuring its reliability. In particular, by taking measures according to the risks, it is possible to establish the safety and reliability of AI systems and increase the competitiveness of your business. When promoting the use of AI in your company, we hope you will refer to the contents of this article and thoroughly manage risks.
Macnica, Inc.
Data & Application Division, Data & AI Platform Business Department, Section 2
Daichi Kakinuma
In the past, he worked as a sales engineer, support engineer, and training instructor for products, mainly data analysis platform products. He is currently engaged in sales activities in the fields of AI, machine learning, and data utilization.
Latest Information
Case Study
Blog
Document List