Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s...
-
Upload
dangkhuong -
Category
Documents
-
view
218 -
download
1
Transcript of Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s...
![Page 1: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/1.jpg)
© Fraunhofer HHI | 13.07.2017 | 1
Peter Eisert
Neue Verfahren der Bildverarbeitung – auch zur Erfassung von Schäden in Abwasserkanälen?
Peter Eisert
![Page 2: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/2.jpg)
© Fraunhofer HHI | 13.07.2017 | 2
Fraunhofer is Europe‘s largest organization for applied research.
69 institutes and research units in Germany
Research cooperation around the world
24 500 staff members
Budget: more than 2.1 billion €
Research highlights: MP3 audio codec, H.264, H.265 video codec, LTE mobile communications standard
Fraunhofer-Gesellschaft
Peter Eisert
![Page 3: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/3.jpg)
© Fraunhofer HHI | 13.07.2017 | 3
Fraunhofer Institute for Telecommunications,Heinrich Hertz Institute, HHI
1928 Founding of Heinrich Hertz Institute in BerlinSince 1960s Research on fiber optical transmissionSince 1980s Research on digital video codingSince 2003 Member of Fraunhofer-GesellschaftSince 2009 Branch site in Goslar, Germany
Peter Eisert
![Page 4: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/4.jpg)
© Fraunhofer HHI | 13.07.2017 | 4
Facts and Figures
Last updated: 2015
Peter Eisert
![Page 5: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/5.jpg)
© Fraunhofer HHI | 13.07.2017 | 5
Photonic Network and Systems
Photonic Components
Fiber Optical Sensor Systems
Wireless Communications and Networks
Vision and Imaging Technologies
Video Coding and Analytics
Departments
Photo: fotolia.com/SeanPavonePhoto | Edit: Fraunhofer HHI
Peter Eisert
![Page 6: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/6.jpg)
© Fraunhofer HHI | 13.07.2017 | 6
Image Processing Pipeline
geometry
motion / deformation
semantic data
inspection
Peter Eisert
scene acquisition rendering & display
Processing &
Analysis
![Page 7: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/7.jpg)
© Fraunhofer HHI | 13.07.2017 | 7
2D Scene Capturing
Projection of 3D scene into 2D image plane
Loss of 1 dimension
Still images or video
Higher and higher resolution
Peter Eisert
![Page 8: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/8.jpg)
© Fraunhofer HHI | 13.07.2017 | 8
Image Reprojection
Camera calibration determines parameters of projection
Image can be reprojected onto any other surface
But viewing position cannot be changed
Peter Eisert
perspective camera fisheye camera
![Page 9: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/9.jpg)
© Fraunhofer HHI | 13.07.2017 | 9
Image Reprojection
Peter Eisert
perspective camera fisheye camera
![Page 10: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/10.jpg)
© Fraunhofer HHI | 13.07.2017 | 10
Spectral Sensitivity
Regular camera and humans: RGB colors
Multi-spectral imaging can provide material properties
Information invisible to humans
Peter Eisert
RGB image of tumor tumor @ 500nm
NIR imaging
thermal imaging
![Page 11: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/11.jpg)
© Fraunhofer HHI | 13.07.2017 | 11
3D Scene Capturing
3D geometry can be estimated from multiple views (photogrammetry)
Ill-posed problem
Requires determination of corresponding points between views
Epipolar geometry constrains search space
Peter Eisert
xl xr
disparity d=xl-xr
z~
triangulation
![Page 12: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/12.jpg)
© Fraunhofer HHI | 13.07.2017 | 12
Depth Estimation from Stereo
Images captured with high resolution (10-20 MP)
Dense reconstruction from 2 or more views
Arbitrary objects
Reconstruction of entire head or persons (incl. hair, eyes, etc.)
Peter Eisert
![Page 13: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/13.jpg)
© Fraunhofer HHI | 13.07.2017 | 13
High Detail 3D Reconstruction
Peter Eisert
[Blumenthal, Computers & Graphics 2014]
![Page 14: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/14.jpg)
© Fraunhofer HHI | 13.07.2017 | 14
Textured 3D Model from 2 Views
Peter Eisert
![Page 15: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/15.jpg)
© Fraunhofer HHI | 13.07.2017 | 15
3D Models in Security Applications
Peter Eisert
augmented reality visualization security gates
![Page 16: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/16.jpg)
© Fraunhofer HHI | 13.07.2017 | 16
High Quality 3D Reconstruction
Peter Eisert
![Page 17: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/17.jpg)
© Fraunhofer HHI | 13.07.2017 | 17
Aerial Surveillance, Object Detection
Peter Eisert
![Page 18: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/18.jpg)
© Fraunhofer HHI | 13.07.2017 | 18
Analysis of Vegetation and Landscapes
Peter Eisert
![Page 19: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/19.jpg)
© Fraunhofer HHI | 13.07.2017 | 19
Inspection of Laser Welds
Peter Eisert
![Page 20: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/20.jpg)
© Fraunhofer HHI | 13.07.2017 | 20
3D Recording of Footprints
Peter Eisert
![Page 21: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/21.jpg)
© Fraunhofer HHI | 13.07.2017 | 21
Support of Large Range of Object Sizes
Peter Eisert
![Page 22: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/22.jpg)
© Fraunhofer HHI | 13.07.2017 | 22
Tracking and Motion Estimation
Object / camera motion can be determined from point correspondences
3D motion requires additional priors (e.g. static scene, deformation model)
Dense / sparse motion models
Peter Eisert
camera motion object motion
![Page 23: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/23.jpg)
© Fraunhofer HHI | 13.07.2017 | 23
3D Motion Tracking of Robot
3D tracking of motion path
Corresponding points with epipolar constraint
Assumption of near cylindrical pipe shape
Peter Eisert
![Page 24: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/24.jpg)
© Fraunhofer HHI | 13.07.2017 | 24
Enhanced Surface Textures
Projection onto 3D cylinder surface
Correction of lighting artifacts
Compared to simple unwrap
higher resolution
less stitching artifacts
Peter Eisert
Panoramounwrapping with 3D motion
![Page 25: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/25.jpg)
© Fraunhofer HHI | 13.07.2017 | 25
Deformable Surface Tracking
Peter Eisert
![Page 26: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/26.jpg)
© Fraunhofer HHI | 13.07.2017 | 26
Deformable Surface Tracking
Peter Eisert
![Page 27: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/27.jpg)
© Fraunhofer HHI | 13.07.2017 | 27
3D Surface Tracking for Navigated Surgery
Peter Eisert
![Page 28: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/28.jpg)
© Fraunhofer HHI | 13.07.2017 | 28
Image Analysis with Machine Learning
Deep neural networks highly successful for many image analysis tasks
image classification (e.g. detection of defects in pipes)
scene labeling / segmentation
image synthesis
…
Peter Eisert
[ImageNet]
![Page 29: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/29.jpg)
© Fraunhofer HHI | 13.07.2017 | 29
Image Classification
Peter Eisert
(labeled) Training data Classifier
“Fish“
Trai
nin
gTe
stin
g
![Page 30: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/30.jpg)
© Fraunhofer HHI | 13.07.2017 | 30
Convolutional Neural Network
Peter Eisert
![Page 31: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/31.jpg)
© Fraunhofer HHI | 13.07.2017 | 31
Explanation of Black Box
Black box classifier
Explanation of deep neural network decisions
Visualization via relevance feedback
Peter Eisert
![Page 32: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/32.jpg)
© Fraunhofer HHI | 13.07.2017 | 32
Detection of Morphing Attacks
Detection of morphing attacks in ABC Gates
Anomaly detection in passport images
Sematic image forensics
Machine learning
Peter Eisert
Original Morph
![Page 33: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/33.jpg)
© Fraunhofer HHI | 13.07.2017 | 33
Creation of Face Morphs
Morphing pipeline with exchangeable components
Feature detection
Warping
Blending
Automatic alignment of face images based of feature points/feature lines
Automatic adaptive blending that prevents visible cuts
Peter Eisert
Ori
gin
al im
age
sou
rce:
pic
s.st
ir.ac
.uk
Fiel
d w
arp
ing
Piec
ewis
e af
fin
e
Alp
ha
ble
nd
ing
(α=
.5)
Ou
r ad
apti
ve
![Page 34: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/34.jpg)
© Fraunhofer HHI | 13.07.2017 | 34
Detection of Face Morphs
Morphing attack detection using Deep Neural Networks
Creation of training and test datasets>1500 original images
Image pre-processing (filtering, noise) to increase variations in dataset
Comparison of different network architectures
Peter Eisert
![Page 35: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/35.jpg)
© Fraunhofer HHI | 13.07.2017 | 35
VR / AR for Interaction and Assistance
Peter Eisert
![Page 36: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/36.jpg)
© Fraunhofer HHI | 13.07.2017 | 36
Dynamic 3D Scene Capturing
Peter Eisert
![Page 37: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/37.jpg)
© Fraunhofer HHI | 13.07.2017 | 37
TimeLab
Peter Eisert
![Page 38: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/38.jpg)
© Fraunhofer HHI | 13.07.2017 | 38
Omnidirectional Video
High resolution 360° video
No stitching artifacts
2D and 3D
Real-time end to end streaming
Peter Eisert
![Page 39: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/39.jpg)
© Fraunhofer HHI | 13.07.2017 | 39
Sample Productions
Peter Eisert
![Page 40: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/40.jpg)
© Fraunhofer HHI | 13.07.2017 | 40
Summary
Image processing and analysis is a powerful tool for many applications
Significant progress in imaging hardware
High-resolution, ubiquitous and cheap cameras
VR / AR glasses
Increased computing power (mobile devices, graphics boards)
Machine learning enables novel AI technologies
New applications in water resource management ?
Peter Eisert
![Page 41: Neue Verfahren der Bildverarbeitung – auch zur Erfassung ... · PDF fileSince 1980s Research on digital video coding ... 3D motion requires additional priors ... Morphing attack](https://reader034.fdocuments.us/reader034/viewer/2022042707/5a726cbe7f8b9aac538d7ba0/html5/thumbnails/41.jpg)
© Fraunhofer HHI | 13.07.2017 | 41
Fraunhofer-Institut für Nachrichtentechnik,Heinrich-Hertz-Institut, HHI
WE PUT SCIENCEINTO ACTION.
Contact:
Peter [email protected]+49 30 31002 614
Einsteinufer 3710587 Berlin
Peter Eisert