[AI論文] ターゲット画像のみで画像修正を行う「Deep Image Prior」 - スマートマニュファクチャリング

This article is recommended for those who

I want to grasp the trend of the latest AI papers quickly!
I want to know about the AI paper that became a hot topic!
I want to know briefly what kind of method is Deep Image Prior, which has become a hot topic!

Time needed to finish reading this article

5 minutes

Introduction of useful methods for image processing

Hello, I'm Makky from Macnica AI Women's Club!

In the blink of an eye, half of 2019 is over. I feel that time passes quickly because I am fulfilling every day, but these days I can't help but think that it would be nice if the days were a little longer.

In the previous AI Joshibu tech blog, we covered D-GraphCNN, the latest method of applying CNN to data other than images.

This time, we will introduce an image restoration method using a neural network structure called "Deep Image Prior", which was published last year.

Deep learning method useful for image processing: CNN

As I mentioned in my previous blog, CNN is a useful method for images among deep learning. It was in 2012 that CNN first started to attract attention.

If you are studying deep learning, you may have heard about it, but in the 2012 image recognition contest "ILSVRC (ImageNet Large Scale Visual Recognition Challenge)", the University of Toronto team "SuperVision" was overwhelmingly good. It all started with winning the record.

The method used by “SuperVision” is CNN, and since this competition CNN has been used in a variety of image processing tasks.

Deep Image Prior without training data

Usually, analysis using deep learning such as CNN prepares a large amount of training data and performs image processing from a deep neural network (DNN) generated based on the training data.

However, "Deep Image Prior" shows that the problem of restoring the original image from a degraded image (hereafter referred to as "image restoration") can be solved using only the degraded image. It was.

In other words, there is no need for pre-trained DNNs or large image datasets that have been taken for granted until now.

General method of image restoration

There are generally two optimization goals in the field of image restoration, which are formulated as follows:

Retrieved from “Deep Image Prior” Main idea

Items shown in orange represent data dependent on restoration tasks such as denoising, super-resolution, and restoration. The item shown in blue represents the image data "Image Prior" learned from the teacher data.

For example, consider the case of restoring a clean image (right) from a blurred image as shown in the figure below (left).

By taking advantage of our past experience and the knowledge we have around the world, it may be possible to restore beautiful images without performing tasks such as noise reduction and super-resolution.

Our experience that supports restoration in this way, or “pre-learning,” is called “Prior” in the paper.

出典: “ Dmitry Ulyanov - Deep Image Prior,” YouTube video, 1:01:40, “Man AHL,” June 5, 2018,
https://www.youtube.com/watch?v=-g1NsTuP1_I&feature=youtu.be&t=258

A Novel Deep Image Prior Training Method

Deep Image Prior became a hot topic due to the novelty of simply preparing two things: the CNN structure and the image you want to restore.

Since there is no training data, the paper proposes to use the CNN itself as a prior for learning.

Therefore, instead of obtaining a prior distribution (image representation learned from training data) from the image space, we acquire a reconstructed image by learning one input image and optimizing the CNN parameter Θ with the gradient descent method. increase.

Ordinary image restoration (searching the image space)
Reconstruction using pre-training data

出典: “ Dmitry Ulyanov - Deep Image Prior,” YouTube video, 1:01:40, “Man AHL,” June 5, 2018,
https://www.youtube.com/watch?v=-g1NsTuP1_I&feature=youtu.be&t=975

Deep Image Prior Image Restoration (Exploring CNN Parameter Space)
Reconstruction by optimization of network parameters Θ using gradient descent

出典: “ Dmitry Ulyanov - Deep Image Prior,” YouTube video, 1:01:40, “Man AHL,” June 5, 2018,
https://www.youtube.com/watch?v=-g1NsTuP1_I&feature=youtu.be&t=1403

Deep Image PriorのGitHubページで、画像復元のデモを簡単に試すことができます。
（リンクは本ブログの最後にございます）

Why is the network structure itself useful?

Why is it possible to restore an image simply by learning the target image using a network?
Judging from the results, no clear answer has been revealed yet. However, it seems that the learning speed of the neural network has an effect.

When learning with a neural network, a large CNN network can reduce the error to 0 as much as possible. Therefore, in this case as well, the output of the network is considered to become closer to the target image as learning progresses.

The graph below shows the transition of the error introduced in the paper for each image type. The vertical axis represents the error (mean squared error: MSE), and the horizontal axis represents the iterations of learning.

Transition of Neural Network Error

Source: “Dmitry Ulyanov - Deep Image Prior,” YouTube video, 1:01:40, “Man AHL,” June 5, 2018,
https://www.youtube.com/watch?v=-g1NsTuP1_I&feature=youtu.be&t=1715

ノイズのようなランダムな画像（noise, Image shuffled）と比べると、グラフ中の「Image」で示すような自然な画像の方が誤差を小さくする速度、つまり学習速度が速く、綺麗な画像に早く収束できることがわかります。

Using this difference in learning speed, it seems that it is possible to output a beautiful image by stopping learning before it learns noise.

This result suggests that the CNN has a built-in bias for natural images, and it is possible to obtain beautiful images simply by giving the network time to repeat training.

We will continue to introduce AI papers in the future!

The "Deep Image Prior" introduced this time is a paper presented at CVPR 2018. The paper is available on the Deep Image Prior GitHub page.

We plan to introduce the AI papers presented at CVPR 2019 on the AI Women's Club tech blog in the future!
We will continue to introduce the latest AI papers, so please check it out!

■ 本記事で特集したAI論文の出典元／Reference Lists
　Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky, “ Deep Image Prior ” Web,
　Retrieved from https://dmitryulyanov.github.io/deep_image_prior

　Dmitry Ulyanov - Deep Image Prior.” YouTube video, 1:01:40.
　“Man AHL,” June 5, 2018.https://www.youtube.com/watch?v=-g1NsTuP1_I
　（著者が論文内容について説明している動画です）