ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images...
Transcript of ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images...
![Page 1: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/1.jpg)
ANR Prise 3D
Thesis : Noise detection in stereoscopic synthesis images
using machine learning
Jerome BUISINE
LISIC - Imap
![Page 2: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/2.jpg)
Table of contents
1. Noise in synthesis images
2. Image quality
3. Database
4. Current works
5. Conclusion
1
![Page 3: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/3.jpg)
Noise in synthesis images
![Page 4: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/4.jpg)
Output overview
(a) After 50 samples (b) After 300 samples (c) After 1200 samples
Figure 1: Preview of the images obtained by the Maxwell rendering engine of
the Cuisine01 (D) scene at different generation times
2
![Page 5: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/5.jpg)
Sub image overview
(a) After 50 samples (b) After 300 samples (c) After 1200 samples
Figure 2: Preview of the sub images obtained by the Maxwell render engine of
the Cuisine01 (D) scene at different generation times
Noise overview
As we can see after 50 minutes of generation, a perceptual noise is
generated due to the Monte-Carlo (stochastic) process.
3
![Page 6: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/6.jpg)
Problematic
• How to detect this perceptual noise ?
• How to quantify it ?
4
![Page 7: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/7.jpg)
Problematic
• How to detect this perceptual noise ?
• How to quantify it ?
4
![Page 8: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/8.jpg)
Image quality
![Page 9: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/9.jpg)
Image quality metrics
Image quality assessment (IQA) metrics can be divided in three
categories :
• Full reference (FR) metrics
• Reduced reference (RR) metrics
• No-reference (NR) metrics
5
![Page 10: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/10.jpg)
Image quality metrics
Image quality assessment (IQA) metrics can be divided in three
categories :
• Full reference (FR) metrics
• Reduced reference (RR) metrics
• No-reference (NR) metrics
5
![Page 11: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/11.jpg)
Image quality metrics
Image quality assessment (IQA) metrics can be divided in three
categories :
• Full reference (FR) metrics
• Reduced reference (RR) metrics
• No-reference (NR) metrics
5
![Page 12: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/12.jpg)
Image quality metrics
Image quality assessment (IQA) metrics can be divided in three
categories :
• Full reference (FR) metrics
• Reduced reference (RR) metrics
• No reference (NR) metrics
6
![Page 13: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/13.jpg)
Image quality metrics
• Full reference :
• Peak Signal to Noise Ratio (PSNR)
• Structural Similarity Index Metrix (SSIM) [Wang et al., 2004]
• Multi-Scale SSIM (MS-SSIM) [Wang et al., 2003]
• ...
• No-reference :
• Blind Image Quality Index (BIQI) [Moorthy and Bovik, 2010]
• Blind Referenceless Image Spatial Quality Evaluator (BRISQUE)
[Mittal et al., 2012]
• Perception-based Image QUality Evaluator (PIQUE) [N. et al., 2015]
• ...
7
![Page 14: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/14.jpg)
Image quality metrics Databases
Database of natural images are available with distortions applied on these
images.
A subjective score is then associated to these images.
• TID2008
• LIVE
• CSIQ
• ...
Model objective
Correlate as well as possible with the subjective scores.
8
![Page 15: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/15.jpg)
Image quality metrics Databases
Database of natural images are available with distortions applied on these
images.
A subjective score is then associated to these images.
• TID2008
• LIVE
• CSIQ
• ...
Model objective
Correlate as well as possible with the subjective scores.
8
![Page 16: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/16.jpg)
Image quality metrics Databases
Database of natural images are available with distortions applied on these
images.
A subjective score is then associated to these images.
• TID2008
• LIVE
• CSIQ
• ...
Synthesis images database
Currently in the literature, there is no database that identifies the noise
present into synthesis images.
9
![Page 17: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/17.jpg)
Database
![Page 18: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/18.jpg)
Scenes
ID Scene Indices (samples) Step Images
A Appart02 20 → 900 10 89
D Cuisine01 20 → 1200 10 119
G SdbCentre 20 → 950 10 94
H SdbDroite 20 → 950 10 94
Table 1: Reduced database information
10
![Page 19: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/19.jpg)
Database explanations
11
![Page 20: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/20.jpg)
Database explanations
12
![Page 21: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/21.jpg)
Database explanations
13
![Page 22: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/22.jpg)
Subjective perceptual threshold
14
![Page 23: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/23.jpg)
Subjective perceptual threshold
15
![Page 24: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/24.jpg)
Subjective perceptual threshold
16
![Page 25: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/25.jpg)
Subjective perceptual threshold
17
![Page 26: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/26.jpg)
Subjective perceptual threshold
18
![Page 27: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/27.jpg)
Subjective perceptual threshold
19
![Page 28: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/28.jpg)
Subjective perceptual threshold
20
![Page 29: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/29.jpg)
Subjective perceptual threshold
Information
The final threshold for a zone is the mean of all subjective thresholds
obtained.
21
![Page 30: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/30.jpg)
Subjective perceptual threshold
Information
The final threshold for a zone is the mean of all subjective thresholds
obtained.
21
![Page 31: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/31.jpg)
How sub-images are classified ?
22
![Page 32: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/32.jpg)
How sub-images are classified ?
23
![Page 33: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/33.jpg)
How sub-images are classified ?
24
![Page 34: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/34.jpg)
How sub-images are classified ?
25
![Page 35: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/35.jpg)
How sub-images are classified ?
26
![Page 36: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/36.jpg)
How sub-images are classified ?
27
![Page 37: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/37.jpg)
How sub-images are classified ?
28
![Page 38: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/38.jpg)
How sub-images are classified ?
Label attribution
For each zone, a label value (0 or 1) is associated to the sub-image
obtained at n samples during the generation.29
![Page 39: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/39.jpg)
Current works
![Page 40: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/40.jpg)
What we need ?
30
![Page 41: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/41.jpg)
What we need ?
31
![Page 42: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/42.jpg)
What we need ?
32
![Page 43: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/43.jpg)
What we need ?
33
![Page 44: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/44.jpg)
What we need ?
34
![Page 45: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/45.jpg)
What we need ?
• What kind of data model wants in order to classify as well as
possible sub-images
• The whole sub-image or reduced information ?
• What kind of data well described the perceived noise in image ?
35
![Page 46: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/46.jpg)
What we need ?
• What kind of data model wants in order to classify as well as
possible sub-images
• The whole sub-image or reduced information ?
• What kind of data well described the perceived noise in image ?
35
![Page 47: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/47.jpg)
What we need ?
• What kind of data model wants in order to classify as well as
possible sub-images
• The whole sub-image or reduced information ?
• What kind of data well described the perceived noise in image ?
35
![Page 48: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/48.jpg)
Expected model interactions
36
![Page 49: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/49.jpg)
Reduction of canals
• Using L (luminance) canal from L*a*b transformation.
• Using the Mean Substracted Contrast Normalized (MSCN, see Eq.
3) matrix.
37
![Page 50: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/50.jpg)
Reduction of canals
• Using L (luminance) canal from L*a*b transformation.
• Using the Mean Substracted Contrast Normalized (MSCN, see Eq.
3) matrix.
37
![Page 51: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/51.jpg)
Pool of final features
Hypothesis
Low bits values from images perhaps keep information about noise
low bits 3
1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
low bits 4 shifted 2
1 1 1 1 1 1 1 1 0 0 1 1 1 1 0 0
F = {lab,mscn, low bits 4 shifted 2}+ [low bits i ]
with i ∈ [2, 6] and |F | = 8
38
![Page 52: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/52.jpg)
Pool of final features
Hypothesis
Low bits values from images perhaps keep information about noise
low bits 3
1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
low bits 4 shifted 2
1 1 1 1 1 1 1 1 0 0 1 1 1 1 0 0
F = {lab,mscn, low bits 4 shifted 2}+ [low bits i ]
with i ∈ [2, 6] and |F | = 8
38
![Page 53: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/53.jpg)
Pool of final features
Hypothesis
Low bits values from images perhaps keep information about noise
low bits 3
1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
low bits 4 shifted 2
1 1 1 1 1 1 1 1 0 0 1 1 1 1 0 0
F = {lab,mscn, low bits 4 shifted 2}+ [low bits i ]
with i ∈ [2, 6] and |F | = 8
38
![Page 54: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/54.jpg)
Pool of final features
Hypothesis
Low bits values from images perhaps keep information about noise
low bits 3
1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
low bits 4 shifted 2
1 1 1 1 1 1 1 1 0 0 1 1 1 1 0 0
F = {lab,mscn, low bits 4 shifted 2}+ [low bits i ]
with i ∈ [2, 6] and |F | = 8
38
![Page 55: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/55.jpg)
Pool of final features
Hypothesis
Low bits values from images perhaps keep information about noise
low bits 3
1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
low bits 4 shifted 2
1 1 1 1 1 1 1 1 0 0 1 1 1 1 0 0
F = {lab,mscn, low bits 4 shifted 2}+ [low bits i ]
with i ∈ [2, 6] and |F | = 8
38
![Page 56: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/56.jpg)
Pool of final features
Hypothesis
Low bits values from images perhaps keep information about noise
low bits 3
1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0
low bits 4 shifted 2
1 1 1 1 1 1 1 1 0 0 1 1 1 1 0 0
F = {lab,mscn, low bits 4 shifted 2}+ [low bits i ]
with i ∈ [2, 6] and |F | = 838
![Page 57: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/57.jpg)
Dimension reduction using SVD
39
![Page 58: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/58.jpg)
Dimension reduction using SVD
39
![Page 59: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/59.jpg)
Dimension reduction using SVD
40
![Page 60: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/60.jpg)
Why the use of SV vector ?
Figure 3: Singular values vector obtained from images of Appart02 (A) scene
with L chanel
41
![Page 61: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/61.jpg)
Another reduction
• Positions set, P with |P| = 5
• Potential sub-vectors size,
N = [4, 8, 16, 26, 32, 40]
42
![Page 62: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/62.jpg)
Another reduction
• Positions set, P with |P| = 5
• Potential sub-vectors size,
N = [4, 8, 16, 26, 32, 40]
42
![Page 63: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/63.jpg)
Other parameters : normalization
K = [svd , svdn, svdne]
where
• svd : no normalization
• svdn : sub-vector is normalized itself
• svdne : sub-vector is normalized depending the min and max
sub-vectors interval values from the whole dataset
43
![Page 64: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/64.jpg)
Other parameters : normalization
K = [svd , svdn, svdne]
where
• svd : no normalization
• svdn : sub-vector is normalized itself
• svdne : sub-vector is normalized depending the min and max
sub-vectors interval values from the whole dataset
43
![Page 65: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/65.jpg)
Other parameters : training data
Zones are selected randomly with
Z = [4, 6, 8, 10, 12]
44
![Page 66: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/66.jpg)
Final features as model input
45
![Page 67: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/67.jpg)
Kind of models
We define 3 model architectures to fit as well as possible final features :
• M1 : Support Vector Machine
• M2 : Ensemble model (3 sub-models)
• M3 : Ensemble model v2 (5 sub-models)
Ensemble models configurations
These ensemble models are in fact voting classifier with principle of
fair voting and regulated on soft.
46
![Page 68: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/68.jpg)
Parameters : total combinations
Finally using all these parameters, we have a lot of combinations :
r = 3× |F | × |P| × |N| × |K | × |Z |= 3× 8× 5× 6× 3× 5
= 10800
47
![Page 69: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/69.jpg)
Build of specific dataset
48
![Page 70: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/70.jpg)
Build of specific dataset
49
![Page 71: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/71.jpg)
Build of specific dataset
50
![Page 72: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/72.jpg)
Build of specific dataset
51
![Page 73: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/73.jpg)
Build of specific dataset
52
![Page 74: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/74.jpg)
How to compare model ?
The Area Under The Curve Receiver Operating Characteristics score is
used to compare these models based on test dataset.
AUC - ROC score is a performance measurement
for classification problem. It tells how much model
is capable of distinguishing between classes.
53
![Page 75: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/75.jpg)
How to compare model ?
The Area Under The Curve Receiver Operating Characteristics score is
used to compare these models based on test dataset.
AUC - ROC score is a performance measurement
for classification problem. It tells how much model
is capable of distinguishing between classes.
53
![Page 76: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/76.jpg)
First results
Model feature size interval zones ROC Train ROC Val ROC Test
M3 lab (svd) 40 [80, 120[ 12 0.9418 0.9023 0.9219
M2 lab (svd) 32 [84, 116[ 4 0.9158 0.8724 0.9153
M2 lab (svd) 40 [80, 120[ 12 0.9629 0.9049 0.9145
M2 lab (svdne) 26 [87, 113[ 6 0.9337 0.8763 0.9089
M3 low bits 2 (svd) 40 [0, 40[ 12 0.9567 0.8417 0.9081
Table 2: The 5 best models found based on AUC ROC score
54
![Page 77: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/77.jpg)
Simulation from best model
Figure 4: Simulation of each zone obtained on scene Appart02 (A)
55
![Page 78: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/78.jpg)
Simulation from best model
Figure 5: Simulation of each zone obtained on scene SdbDroite (H)
56
![Page 79: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/79.jpg)
Other approaches
Features
• Statistics approach, use of statistics from sub-block
• Mean, Median, Percentile at 25%, Percentile at 75%, Variance, Area
under curvex
• Use of MSCN statistics
• Use of correlation matrix from SV
• Use of correlation between SV and labels
Model
• Use of Deep Learning approach as model
Remark
All of these approaches seems to give same results as before.. Hence,
overfitting or bad approximation.
57
![Page 80: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/80.jpg)
Other approaches
Features
• Statistics approach, use of statistics from sub-block
• Mean, Median, Percentile at 25%, Percentile at 75%, Variance, Area
under curvex
• Use of MSCN statistics
• Use of correlation matrix from SV
• Use of correlation between SV and labels
Model
• Use of Deep Learning approach as model
Remark
All of these approaches seems to give same results as before.. Hence,
overfitting or bad approximation.
57
![Page 81: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/81.jpg)
Other approaches
Features
• Statistics approach, use of statistics from sub-block
• Mean, Median, Percentile at 25%, Percentile at 75%, Variance, Area
under curvex
• Use of MSCN statistics
• Use of correlation matrix from SV
• Use of correlation between SV and labels
Model
• Use of Deep Learning approach as model
Remark
All of these approaches seems to give same results as before.. Hence,
overfitting or bad approximation.
57
![Page 82: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/82.jpg)
Other approaches
Features
• Statistics approach, use of statistics from sub-block
• Mean, Median, Percentile at 25%, Percentile at 75%, Variance, Area
under curvex
• Use of MSCN statistics
• Use of correlation matrix from SV
• Use of correlation between SV and labels
Model
• Use of Deep Learning approach as model
Remark
All of these approaches seems to give same results as before.. Hence,
overfitting or bad approximation.
57
![Page 83: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/83.jpg)
Other approaches
Features
• Statistics approach, use of statistics from sub-block
• Mean, Median, Percentile at 25%, Percentile at 75%, Variance, Area
under curvex
• Use of MSCN statistics
• Use of correlation matrix from SV
• Use of correlation between SV and labels
Model
• Use of Deep Learning approach as model
Remark
All of these approaches seems to give same results as before.. Hence,
overfitting or bad approximation.
57
![Page 84: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/84.jpg)
Other approaches
Features
• Statistics approach, use of statistics from sub-block
• Mean, Median, Percentile at 25%, Percentile at 75%, Variance, Area
under curvex
• Use of MSCN statistics
• Use of correlation matrix from SV
• Use of correlation between SV and labels
Model
• Use of Deep Learning approach as model
Remark
All of these approaches seems to give same results as before.. Hence,
overfitting or bad approximation.
57
![Page 85: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/85.jpg)
Conclusion
![Page 86: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/86.jpg)
Conclusion
Difficult to have a model which generalizes for each scene but why ?
• Scene seems to have each own components to describe the noise
• Difficult to find the best components from data
Solution :
• Find a way to choose components depending of the scene ?
• Use of work of Andre and Remi : SV entropy
58
![Page 87: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/87.jpg)
Conclusion
Difficult to have a model which generalizes for each scene but why ?
• Scene seems to have each own components to describe the noise
• Difficult to find the best components from data
Solution :
• Find a way to choose components depending of the scene ?
• Use of work of Andre and Remi : SV entropy
58
![Page 88: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/88.jpg)
Conclusion
Difficult to have a model which generalizes for each scene but why ?
• Scene seems to have each own components to describe the noise
• Difficult to find the best components from data
Solution :
• Find a way to choose components depending of the scene ?
• Use of work of Andre and Remi : SV entropy
58
![Page 89: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/89.jpg)
Future works
Noise detection
• Study of Singular Values behaviors from few noises
• Use of Generative Adversarial Network model
• Use of Transfer learning (Alexnet, Resnet, ...)
• Work with University of Lille 3
• Tackle the stereoscopic aspect
Denoising
• Use of Deep Learning approaches (autoencoder and others...) to
denoise images
• Create custom denoiser for synthesis images
59
![Page 90: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/90.jpg)
Future works
Noise detection
• Study of Singular Values behaviors from few noises
• Use of Generative Adversarial Network model
• Use of Transfer learning (Alexnet, Resnet, ...)
• Work with University of Lille 3
• Tackle the stereoscopic aspect
Denoising
• Use of Deep Learning approaches (autoencoder and others...) to
denoise images
• Create custom denoiser for synthesis images
59
![Page 91: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/91.jpg)
Development
Developments are centralized into the IPFML
python package
(https://github.com/jbuisine/IPFML).
60
![Page 92: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/92.jpg)
Questions?
60
![Page 93: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/93.jpg)
References i
Mittal, A., Moorthy, A. K., and Bovik, A. C. (2012).
No-reference image quality assessment in the spatial domain.
IEEE Trans. Image Processing, 21(12):4695–4708.
Moorthy, A. K. and Bovik, A. C. (2010).
A two-step framework for constructing blind image quality
indices.
IEEE Signal Process. Lett., 17(5):513–516.
N., V., D., P., Bh., M. C., Channappayya, S. S., and Medasani, S. S.
(2015).
Blind image quality evaluation using perception based
features.
In Twenty First National Conference on Communications, NCC
2015, Mumbai, India, February 27 - March 1, 2015, pages 1–6.
61
![Page 94: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/94.jpg)
References ii
Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. (2004).
Image quality assessment: from error visibility to structural
similarity.
IEEE Trans. Image Processing, 13(4):600–612.
Wang, Z., Simoncelli, E. P., and Bovik, A. C. (2003).
Multiscale structural similarity for image quality assessment.
In The Thrity-Seventh Asilomar Conference on Signals, Systems &
Computers, 2003, volume 2, pages 1398–1402. Ieee.
62
![Page 95: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/95.jpg)
Backup slides (PSNR)
PSNR = 10× log10
(d2
MSE
)(1)
where d is the signal dynamics (the maximum possible value for a pixel).
In the standard case of an image where the components of a pixel are
encoded on 8 bits, d = 255 and MSE (see Eq. 2) is the mean square
error between the 2 images.
MSE =1
mn
m−1∑i=0
n−1∑j=0
(Io(i , j)− Ir (i , j))2 (2)
where Io is the distorted image and Ir the reference image, both of size
m × n
![Page 96: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/96.jpg)
Backup slides (MSCN)
To calculate the MSCN matrix, we must first convert our RGB image to
a grayscale image. The MSCN will extract Natural Structure Scene
(NSS) information from this grayscale image. An operation is applied to
the luminance image I (i , j) to produce :
I (i , j) =I (i , j)− µ(i , j)
σ(i , j) + C(3)
where i ∈ 1, 2...M, j ∈ 1, 2...N are spatial indices, M, N are respectively
the height and width of the image, C is constant, which is set to 1 to
avoid instability
![Page 97: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/97.jpg)
Backup slides (MSCN)
and where,
µ(i , j) =K∑
k=−K
L∑l=−L
wk,l Ik,l(i , j) (4)
σ(i , j) =
√√√√ K∑k=−K
L∑l=−L
wk,l(Ik,l(i , j)− µ(i , j))2 (5)
In (4) and (5) w = {wk,l |k = −K , ...,K , l = −L, ..., L} is a circularly
symmetrical 2D Gaussian weighting function sampled at 3 standard
deviations (K = L = 3) and recalculated at unit volume. Then, the
transformed luminance values of (3) are called Mean Subtracted
Constrast Normalized (MSCN) coefficients.
![Page 98: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/98.jpg)
Kind of models
We define 3 models architecture to fit as well as possible final features :
• Support Vector Machine (with RBF kernel and cross validation
process)
• Ensemble model, composed of :
• SVM with same configuration as previous model
• Random Forest with 100 estimators
• Logistic Regression with liblinear kernel
• Ensemble model v2, composed of :
• SVM with same configuration as previous model
• Random Forest with 100 estimators
• Logistic Regression with liblinear kernel
• KNeighbors Classifier
• Gradient Boosting Classifier with 100 estimators and learn step set
to 1.0.
![Page 99: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/99.jpg)
Kind of models
We define 3 models architecture to fit as well as possible final features :
• Support Vector Machine (with RBF kernel and cross validation
process)
• Ensemble model, composed of :
• SVM with same configuration as previous model
• Random Forest with 100 estimators
• Logistic Regression with liblinear kernel
• Ensemble model v2, composed of :
• SVM with same configuration as previous model
• Random Forest with 100 estimators
• Logistic Regression with liblinear kernel
• KNeighbors Classifier
• Gradient Boosting Classifier with 100 estimators and learn step set
to 1.0.
![Page 100: ANR Prise 3D - Jérôme BUISINE€¦ · Thesis : Noise detection in stereoscopic synthesis images using machine learning J er^ome BUISINE LISIC - Imap. Table of contents 1. Noise](https://reader036.fdocuments.us/reader036/viewer/2022071501/6120bddc01a00e473b41f518/html5/thumbnails/100.jpg)
Kind of models
We define 3 models architecture to fit as well as possible final features :
• Support Vector Machine (with RBF kernel and cross validation
process)
• Ensemble model, composed of :
• SVM with same configuration as previous model
• Random Forest with 100 estimators
• Logistic Regression with liblinear kernel
• Ensemble model v2, composed of :
• SVM with same configuration as previous model
• Random Forest with 100 estimators
• Logistic Regression with liblinear kernel
• KNeighbors Classifier
• Gradient Boosting Classifier with 100 estimators and learn step set
to 1.0.