Deep learning for image super resolution
-
Upload
prudhvi-raj -
Category
Data & Analytics
-
view
29 -
download
0
Transcript of Deep learning for image super resolution
![Page 1: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/1.jpg)
DEEP LEARNING FOR IMAGE SUPER-RESOLUTIONCHAO DONG, CHEN CHANGE LOY, KAIMING HE, XIAOOU TANG
Presented By Prudhvi Raj DachapallyD. Prudhvi Raj
![Page 2: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/2.jpg)
AbstractUsing Deep Convolutional Networks,
the machine can learn end-to-end mapping between the low/high-resolution images. Unlike traditional methods, this method jointly optimizes all the layers of the image. A light-weight CNN structure is used, which is simple to implement and provides formidable trade-off from the existential methods.
![Page 3: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/3.jpg)
What is Deep Learning? A branch of Artificial Neural Networks and
Machine Learning that deals with more convolutional and realistic brain structures.
In the words of Dr. Andrew Ng, researcher at Stanford, Founder & CEO of Coursera, “Increased computing power has allowed us to map and process much larger neural networks than ever before.”
![Page 4: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/4.jpg)
Appealing Properties of the Proposed Model The name given for this model is Super – Resolution
Convolutional Neural Network (or) SRCNN. Structure is simple, but provides superior accuracy
compared to state-of-the-art methods. Since it is a fully feed-forward network, it is
unnecessary to solve the optimization problem. Restoration quality can be further improved with more
diverse data and/or more deeper network without changing the core structure of the network.
SRCNN model can also cope with channels of color images simultaneously with ease, which in turn can improve performance.
![Page 5: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/5.jpg)
Preliminaries Color Channel used – YCbCr
Y – Luminance Cb – Blue – difference Cr – Red – difference Cb and Cr are Chrominance components
First, we upscale the image to a desired size using bicubic interpolation method. This is just a pre-processing step.
![Page 6: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/6.jpg)
Structure of the Network
![Page 7: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/7.jpg)
Components in the Network Patch Extraction and Representation
Densely extracts patches and then represents them as a set of filters. This layer is expressed as a function F1, where
F 1(Y) = max(0, W1 * Y + B1)
This layer extracts a n1 –dimensional feature for each patch. Non – Linear Mapping
Maps each of the n1-dimensional vectors into an n2-dimensional one. This layer is expressed as a function F2, where
F 2(Y) = max(0, W2 * F1(Y) + B2)
It is possible to add more convolutional layers to this structure, but in perspective, increases the training time.
Reconstruction The predicted overlapping high-resolution patches are often averaged to
produce the final full image. This convolutional layer is defined as
F (Y) = W3 * F 2(Y) + B3
![Page 8: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/8.jpg)
Terms Used in the Formulations W1 = Corresponds to the n1 filters of size c
* f1 * f1, Where c is number of channels and f1 is the spatial size
of the filter. B1 = An n1-dimensional vector, whose each
element is associated with a filter. W2 = n2 filters of size n1 * f2 * f2. B2 = n2 dimensional vector. W3 = Corresponds to c filters of size n2 * f3 * f3 B3 = c- dimensional vector.
![Page 9: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/9.jpg)
Learning Process Estimation of network parameters can be
achieved through minimizing the loss between reconstructed images and the corresponding original high-resolution images. This is done by taking the Mean Squared Error (MSE).
Using MSE as a loss function, favors high PSNR( Peak Signal to Noise Ratio).
The loss is minimized by using stochastic gradient descent with regular back-propagation algorithm.
![Page 10: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/10.jpg)
Experiments Training Data
Very Large Data Set of 395, 909 images from 2013 ImageNet Competition.
Test Data A BSD200 Data Set with 200 images.
Basic Network Settings These are f1 = 9, f2 = 1, f3 = 5, n1 = 64 and
n2 = 32.
![Page 11: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/11.jpg)
Results
![Page 12: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/12.jpg)
Comparison Against the State-of-the-art Methods
![Page 13: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/13.jpg)
Real Time Results
![Page 14: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/14.jpg)
Expansion Scope Using Large Filters
Increasing the filter size can increase the PSNR value, but also increases the training time.
Using Deeper Networks This can sometimes be a contradiction to the
rule “More the layers, so is the accuracy.”
![Page 15: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/15.jpg)
Conclusion This approach, SRCNN, learns an end-to-end
mapping between low- and high-resolution images, with little extra pre/post-processing beyond the optimization. With a lightweight structure, the SRCNN achieves a superior performance than the state-of-the-art methods. Additional improvement in performance can be gained further by exploring more filters and different training strategies.
![Page 16: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/16.jpg)
ReferencesImages, tables and some of the text used in this presentation as taken from Chao Dong et.al. “Image Super-Resolution Using Deep Convolutional Networks”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 38, February 2016.
![Page 17: Deep learning for image super resolution](https://reader035.fdocuments.us/reader035/viewer/2022081520/589e23b01a28ab605b8b70a3/html5/thumbnails/17.jpg)
Thank You