Goals

In this blog, I aim to share the following information:
・Brief background on what GAN is and its applications.
・While GAN is often used to generate authentic looking images, it can also be used for detecting outliers or anomalies.

Background

The idea of Generative Adversarial Network or GAN was first introduced by Goodfellow et al. in their paper, Generative Adversarial Networks. Since its introduction, GAN has been causing waves in the AI research world. Even one of the fathers of deep learning, Yann Lecun said that GAN is the most interesting idea in the last ten years of machine learning.

But first, what is GAN? Why the sudden interest in using this idea? Where can we use it? Let’s dive in.

GAN is composed of two sub-networks: the generator and discriminator.  A popular analogy for GAN is about a money counterfeiter (generator) trying to fool a police (discriminator) by creating fake money (assuming the counterfeiter doesn’t go to jail when he gets caught!). So the counterfeiter prints fake money, mix it with real money, and tries to use it. However, the police is able to detect the fake money so counterfeiter can’t use the fake money anymore. Counterfeiter learns from it and makes another batch of fake money. As the police gets better at detecting fake money, the counterfeiter also tries to do more sophisticated way to fool the police.  This goes on until the police can no longer distinguish between real and fake money.

In terms of neural network, the generator creates images and tries to fool the discriminator that detects the authenticity of the images. The generated and real images are shuffled and given to the discriminator. The network losses are then determined from the prediction of the discriminator. The two sub-networks essentially fight with each other through what is called adversarial training (that’s why it’s called Generative Adversarial Networks. Generative because it generates data). In real applications, we want the two sub-networks to learn from each other. If one sub-network is performing relatively bad, the other sub-network won’t be able to learn much.

Examples of Recent Progress and Application

I will mention here few applications (of many) of GAN.

In the picture above, can you say which one is a real image? Answer: None! These are all generated images by the model BigGAN. The model was able to imitate real images by learning from a large image dataset with varying contents. Now, there are models that can create realistic photos of humans, objects, and even anime characters. Have you used those apps that morph your face to look old, or make you smile, or change to the opposite gender? Most probably, the app uses some kind of GAN.

While transforming yourself into a pretty girl with glasses is fun, GAN can also be used for more useful applications. In one study, they used GAN to create synthetic CT-scan images of liver lesions. Then, they add it for training for liver lesion classification. This technique considerably improved the model’s performance.

GANs can also be applied for anomaly detection. This machine learning problem can be a difficult task because usually there’s a significant class imbalance between normal and anomalous data. Sometimes, there’s even no anomaly examples which makes it really hard to detect anomalies. In the next section, we will focus more on applying GAN to detect anomalies.

Using GAN for Anomaly Detection

A way of detecting anomalies is to train with normal data only. If the model is good, the difference between the predicted and actual output should be high, if the input is anomalous. For GAN, you can train the network with normal images only. So that when you input an anomalous data, you’ll be able to detect it because the reconstruction error of input and generated data will be high.

However, a standard GAN’s input is a random latent z, not an input image. In the first GAN-based anomaly detection paper, Schlegl et al iteratively searches the best latent z that best represents the input during prediction. While the performance of the model is good, the model is slow due to its method in finding the latent z.

Example Implementation: GANomaly

After that initial GAN-based anomaly detection paper, more GAN-based methods were researched on. To introduce one research, I will discuss this particular GAN-based anomaly detection framework that is both simple and powerful.

In a paper entitled “GANomaly: Semi-supervised Anomaly Detection via Adversarial Training”, Akcay et al. created a network that is composed of the following subnetworks: Bowtie Autoencoder as the generator, classifier as the discriminator, and another encoder that compresses the generated image. (see .gif below for the architecture and location of the sub-networks)

During training, only normal images are given to the generator. The generator compresses the image then decompresses it, generating a fake image. Both the input and fake image will be given to the discriminator. The discriminator will then try to distinguish whether the images are real or fake. The other encoder is used to compress the generated image, which will help the generator to encode normal images through the loss function. Using different loss functions, generator and discriminator will learn by trying to one up each other.

In the picture above, the model was tested on a dataset that contains normal baggage/parcels X-ray images and anomalous baggage with deadly weapon X-ray images. Because the model was only trained on normal baggage/parcels images, it wasn’t able to generate the deadly weapons on the images. Using the results, anomalous baggage can be found by comparing the input and generated image. This kind of model would greatly help in airport baggage surveillance!

The model was tested on different datasets. Maybe you can try it out too, the authors made their code available. Although, they currently don’t have classification task done yet. So you have to examine the generated images yourself.

Summary

In this post, I talked about what GAN and some of its applications. Then, took a deeper dive on using GAN for anomaly detection by examining one GAN-based implementation for anomaly detection. While GAN is very good in creating authentic looking images, it can also be used for non-obvious applications like anomaly detection. By training the GAN model on normal images, we can use its generated images to distinguish between normal and anomalous examples.

マクニカのARIH（AI Research & InnovationHub）では、最先端のAI研究・調査・実装による評価をした上で最もふさわしいAI技術を組み合わせた知見を提供し、企業課題に対する最適解に導く活動をしています。
詳細は下記よりご覧ください。

論文を活用した事例はこちら