I want to grasp the trend of the latest AI papers quickly!
I want to know about the AI paper that became a hot topic!
I want to know briefly what kind of method is Deep Image Prior, which has become a hot topic!
Time needed to finish reading this article
5 minutes
Introduction of useful methods for image processing
Hello, I'm Makky from Macnica AI Women's Club!
In the blink of an eye, half of 2019 is over. I feel that time passes quickly because I am fulfilling every day, but these days I can't help but think that it would be nice if the days were a little longer.
In the previous AI Joshibu tech blog, we covered D-GraphCNN, the latest method of applying CNN to data other than images.
This time, we will introduce an image restoration method using a neural network structure called "Deep Image Prior", which was published last year.
Deep learning method useful for image processing: CNN
As I mentioned in my previous blog, CNN is a useful method for images among deep learning. It was in 2012 that CNN first started to attract attention.
If you are studying deep learning, you may have heard about it, but in the 2012 image recognition contest "ILSVRC (ImageNet Large Scale Visual Recognition Challenge)", the University of Toronto team "SuperVision" was overwhelmingly good. It all started with winning the record.
The method used by “SuperVision” is CNN, and since this competition CNN has been used in a variety of image processing tasks.
Deep Image Prior without training data
Usually, analysis using deep learning such as CNN prepares a large amount of training data and performs image processing from a deep neural network (DNN) generated based on the training data.
However, "Deep Image Prior" shows that the problem of restoring the original image from a degraded image (hereafter referred to as "image restoration") can be solved using only the degraded image. It was.
In other words, there is no need for pre-trained DNNs or large image datasets that have been taken for granted until now.
General method of image restoration
There are generally two optimization goals in the field of image restoration, which are formulated as follows:
Items shown in orange represent data dependent on restoration tasks such as denoising, super-resolution, and restoration. The item shown in blue represents the image data "Image Prior" learned from the teacher data.
For example, consider the case of restoring a clean image (right) from a blurred image as shown in the figure below (left).
By taking advantage of our past experience and the knowledge we have around the world, it may be possible to restore beautiful images without performing tasks such as noise reduction and super-resolution.
Our experience that supports restoration in this way, or “pre-learning,” is called “Prior” in the paper.
Deep Image Prior became a hot topic due to the novelty of simply preparing two things: the CNN structure and the image you want to restore.
Since there is no training data, the paper proposes to use the CNN itself as a prior for learning.
Therefore, instead of obtaining a prior distribution (image representation learned from training data) from the image space, we acquire a reconstructed image by learning one input image and optimizing the CNN parameter Θ with the gradient descent method. increase.
Ordinary image restoration (searching the image space) Reconstruction using pre-training data
You can easily try an image restoration demo on the Deep Image Prior GitHub page. (link is at the end of this blog)
Why is the network structure itself useful?
Why is it possible to restore an image simply by learning the target image using a network? Judging from the results, no clear answer has been revealed yet. However, it seems that the learning speed of the neural network has an effect.
When learning with a neural network, a large CNN network can reduce the error to 0 as much as possible. Therefore, in this case as well, the output of the network is considered to become closer to the target image as learning progresses.
The graph below shows the transition of the error introduced in the paper for each image type. The vertical axis represents the error (mean squared error: MSE), and the horizontal axis represents the iterations of learning.
Compared to noise-like random images (noise, image shuffled), the speed at which natural images like "Image" in the graph reduce the error, that is, the learning speed is faster, and the images are cleaner. You can see that it can converge.
Using this difference in learning speed, it seems that it is possible to output a beautiful image by stopping learning before it learns noise.
This result suggests that the CNN has a built-in bias for natural images, and it is possible to obtain beautiful images simply by giving the network time to repeat training.
We will continue to introduce AI papers in the future!
The "Deep Image Prior" introduced this time is a paper presented at CVPR 2018. The paper is available on the Deep Image Prior GitHub page.
We plan to introduce the AI papers presented at CVPR 2019 on the AI Women's Club tech blog in the future! We will continue to introduce the latest AI papers, so please check it out!
■ Sources of AI papers featured in this article / Reference Lists Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky, “ Deep Image Prior ” Web, Retrieved from https://dmitryulyanov.github.io/deep_image_prior
Dmitry Ulyanov - Deep Image Prior.” YouTube video, 1:01:40. “Man AHL,” June 5, 2018. https://www.youtube.com/watch?v=-g1NsTuP1_I (This is a video in which the author explains the contents of the paper.)
Latest Information
Case Study
Blog
Document List