1

Blind Image Quality Assessment

Based on Machine Learning

陈 欣 2014-12-29

2

Image Quality Assessment

Image quality assessment (IQA) aims to use computational models to measure the image quality consistently with subjective evaluations.

3

Image Quality Assessment

http://sse.tongji.edu.cn/linzhang/IQA/IQA.htm

Reference Image

Image QualityAssessment (IQA) MLA2013

Image Fidelity Metrics

Accumulate physical errors of the image

Mean Squared Error (MSE)Peak Signal to Noise Ratio (PSNR)

IQASignal Structure Metrics

Describe image degradation withperceived change in Structural

HVS Model Metrics

Simulate various aspects of the HVSperception property

Daly visible differences predictor (VDP)Perceptual Distortion Metric (PDM)

Machine Learning g Metrics

Utilize machine learning in differentaspects of image quality assessment

Structure Similarity (SSIM)Feature-Similarity (FSIM)

informationBlind Image Quality Index (BIQI)

Blind/Referenceless Image SpatialQUality Evaluator (BRISQUE)

VIPS Lab, Xidian University

Database Original images Distorted images Types of distortion

LIVEII 29 982 5

TID2008 25 1700 17

MICT 14 168 2

IVC 10 195 4

CSIQ 30 750 5

Evaluation criteria metrics

PLCC: Pearson linear correlation coefficient, provides the prediction accuracy.

SROCC: Spearman rank-order correlation coefficient, measures the predictionmonotonicity.

RMSE: Root mean square error.

MAE: Mean absolute error.

Databases and Evaluation Criteria

Blind Image Quality Assessment

PSNR PSNR PSNR PSNR

http://live.ece.utexas.edu/research/quality/ http://www.umiacs.umd.edu/~pengye/index.html

CVPR 2012,2013,2014

7

Blind Image Quality Assessment

NR-IQA

PSNR PSNR PSNR PSNR

Exploit Discriminant Features (transformation domain: wavelete transform, DCT transform)

Linear regression , SVM

Training (Codebook, Dictionary Learing)

8

Thank you !

Machine Learning and IQA

ML DL

9

CNN for BLIND IQA

CVPR 2014

10

CNN for BLIND IQA

Contributions

1 One of contributions is that they modified the network structure, such that it can learn image quality features more effectively.

2 It proposed a novel framework that allows learning and prediction of image quality on local regions.

3 The language of this paper is very good.

11

!

CNN for BLIND IQA

Local Normalization Pooling

12

CNN for BLIND IQA

Rectifield Linear Units Nonlinearity

13

CNN for BLIND IQA

14

CNN for BLIND IQA

…

…Image quality

Input

Blind IQA Using a General Regression Neural Network

OutputLayer

SummationLayer

FeaturesImageDatabase

Layer Patten LayerGeneral Regression Neural

[Li & Bovik, IEEE TNN, 2011]

Network (GRNN)

This quality measure based on several complementary and perceptually relevantimage features fed to a GRNN networkApproximating the functional relationship between these features and subjectivemean opinion scoresExperimental results show the method be closely with human subjective judgment

Similar methods for BLIND IQA

Similar methods for BLIND IQA

Unsupervised Feature Learning for No-reference IQA

Encoding Pooling

Features Image qualityTest Image Soft assignment Support vectorRegression (SVR)

K-means

Codebook Training set

Raw-image-patches are used to learn a codebook via K-means clusteringSoft-assignment coding with max pooling to obtain effective feature representationsImage features are projected to quality scores through support vector regression.

It is a general-purpose no-reference IQA method and can be adapted to differentdomains.

[P. Ye, D. Doermann, et al., CVPR, 2012]

Machine Learning and IQA

Learning without Human Scores for Blind IQA

Develop an effective BIQA without human

Distortion

Patch Extraction

scored images for training

Group patches into different groups, and QAC

is applied to each group to learn the quality-

aware centroids

Compare each patch with the centroids and

assign a score with the weighted average

Patch QualityEstimation Feature Extraction

Mapping

QualityAwareClustering

(QAC)

[Xue & Zhang, CVPR, 2013]

Function

Image Quality

18

The End

Happy new year!