RGB Camera
-
Upload
duongduong -
Category
Documents
-
view
230 -
download
0
Transcript of RGB Camera
![Page 1: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/1.jpg)
Sensor Data Acquisition
Domenico Daniele BloisiAndrea Pennisi
Robot ProgrammingElective in Artificial IntelligenceArtificial Intelligence and Robotics
![Page 2: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/2.jpg)
Contact
Sensor Data Acquisition Robot Programming
Domenico Daniele Bloisi, PhDAssistant Professor
Location: DIAG A209
[email protected]://www.dis.uniroma1.it/~bloisi
![Page 3: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/3.jpg)
Introduction
Sensor Data Acquisition Robot Programming
![Page 4: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/4.jpg)
Microsoft Kinect
Sensor Data Acquisition Robot Programming
Kinect was launched in North America on November 4th 2010
Kinect is a motion sensing input device
![Page 5: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/5.jpg)
Microsoft Kinect
Sensor Data Acquisition Robot Programming
Depth Sensors
RGB Camera
Motorized Tilt
Multi-array Mic
![Page 6: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/6.jpg)
Depth Sensors
Sensor Data Acquisition Robot Programming
IR Projector
CMOS Sensor
Consists of IR projector and a CMOS sensor
IR beam bounces off subject and is captured by the CMOS sensor
Sensor uses time to measure distance of objects
![Page 7: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/7.jpg)
Depth Sensors
Sensor Data Acquisition Robot Programming
http://www.wired.com/gadgetlab/2010/11/tonights-release-xbox-kinect-how-does-it-work/
![Page 8: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/8.jpg)
RGB Camera
Sensor Data Acquisition Robot Programming
RGB Camera
640 x 480 pixels at 30 frames per second (fps)
11-bit depth
![Page 9: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/9.jpg)
RGB + Depth
Sensor Data Acquisition Robot Programming
http://graphics.stanford.edu/~mdfisher/Kinect.html
![Page 10: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/10.jpg)
Camera Physical Limitations
Sensor Data Acquisition Robot Programming
![Page 11: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/11.jpg)
Microphones
Sensor Data Acquisition Robot Programming
4 Microphones on the device
Supports single speaker voice recognition
16-bit audio sampled at 16kHz
Microphone Array
![Page 12: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/12.jpg)
Visualizing RGBD Data
Sensor Data Acquisition Robot Programming
ROS
OPENCV
![Page 13: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/13.jpg)
OpenCV
Sensor Data Acquisition Robot Programming
OpenCV (Open Source Computer Vision) is a library of programming functions for realtime computer vision.
BSD Licensed - free for commercial use
C++, C, Python and Java (Android) interfaces
Supports Windows, Linux, Android, iOS and Mac OS
More than 2500 optimized algorithms
http://opencv.org/
![Page 14: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/14.jpg)
kinectGrabber
Sensor Data Acquisition Robot Programming
![Page 15: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/15.jpg)
Sensor Data Acquisition Robot Programming
Libraries//ROS#include <ros/ros.h>#include <image_transport/image_transport.h>#include <cv_bridge/cv_bridge.h>#include <sensor_msgs/image_encodings.h>//OpenCV#include <opencv2/opencv.hpp>#include <opencv2/imgproc/imgproc.hpp>#include <opencv2/highgui/highgui.hpp>
![Page 16: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/16.jpg)
Sensor Data Acquisition Robot Programming
main
int main(int argc, char **argv) { ros::init(argc, argv, "kinectgrabber"); ros::NodeHandle n; ros::Subscriber sub =
n.subscribe("/camera/rgb/image_color", 1, rgbCallback); ros::Subscriber depth =
n.subscribe("/camera/depth/image", 1, depthCallback); ros::spin(); return 0;}
![Page 17: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/17.jpg)
rgbCallBack
Sensor Data Acquisition Robot Programming
void rgbCallback(const sensor_msgs::ImageConstPtr& msg){
cv_bridge::CvImageConstPtr cv_ptr;try{ cv_ptr = cv_bridge::toCvCopy(msg, msg->encoding);}catch (cv_bridge::Exception& ex){
ROS_ERROR("cv_bridge exception: %s", ex.what()); exit(-1); }
cv::imshow("RGB", cv_ptr->image); cv::waitKey(30);}
![Page 18: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/18.jpg)
depthCallBack
Sensor Data Acquisition Robot Programming
void depthCallback(const sensor_msgs::ImageConstPtr& msg){ cv_bridge::CvImageConstPtr cv_ptr; try { cv_ptr = cv_bridge::toCvCopy(msg, msg->encoding); } catch (cv_bridge::Exception& ex) { ROS_ERROR("cv_bridge exception: %s", ex.what()); exit(-1); }
cv::imshow("DEPTH", cv_ptr->image); cv::waitKey(30);}
![Page 19: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/19.jpg)
Result
Sensor Data Acquisition Robot Programming
![Page 20: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/20.jpg)
Source code
Sensor Data Acquisition Robot Programming
http://www.dis.uniroma1.it/~bloisi/didattica/RobotProgramming/kinectGrabber.zip
![Page 21: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/21.jpg)
PCL – Point Cloud Library
Sensor Data Acquisition Robot Programming
The Point Cloud Library (PCL) is a standalone, large scale, open project for 2D/3D image and point cloud processing
Collection of Libraries focused on Point Cloud processing
More than 450 developers/contributors
Over 60 Tutorials and many examples
BSD Licensed - free for commercial use
http://pointclouds.org/
![Page 22: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/22.jpg)
Visualizing 3D Data
Sensor Data Acquisition Robot Programming
ROS
OPENCV PCL
![Page 23: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/23.jpg)
Point Cloud
Sensor Data Acquisition Robot Programming
![Page 24: RGB Camera](https://reader033.fdocuments.us/reader033/viewer/2022051201/58a2d8e11a28abaa338b78da/html5/thumbnails/24.jpg)
Point Cloud
Sensor Data Acquisition Robot Programming
Perception with RGBD sensor- 3D, November 19th 3D data processing