Automated 3D Model Building Using a Robot and an RGB-Depth Camera

Jacob Kneibel, Kexiang Qian, Nick Saxton, David Zoltowski, Nick ZuzowECE480 Team 2, Michigan State University

Executive SummaryNew applications of RGB-Depth cameras are in high demand due to their widening availability to industry and to consumers. 3D modeling is one such application, and while done before in various research projects, the process of using RGB-Depth cameras to model a space has yet to become an automated process [1]. The goal of this project was to combine a robotic moving platform and a commercially available RGB-Depth camera to autonomously build a 3D model of an unknown environment.

Objectives● Fully navigate room

autonomously● Avoid collisions● Complete low-level (table height)

3D model ● Integrate color and/or texture

into model

Results

Budget

Sponsor: Dr. Daniel MorrisFacilitator: Dr. Hayder Radha

Number Item Cost ($)

1 iRobot Create2 Programmable Robot

211.99

2 Microsoft Kinect Camera for Xbox v2

149.99

3 Kinect to Windows Adapter 49.99

4 Power Supplies 67.06

5 Structure 50.00

Total - 529.03

Acknowledgements and ReferencesWe would like to acknowledge our sponsor, Dr. Daniel Morris, for his guidance and advice; our facilitator, Dr. Hayder Radha, for his suggestions and comments; Matthias Nießner and Michael Zollhoefer for their Voxel Hashing code and Matthias Nießner’s correspondence; and our team member, Kexiang Qian, for loaning his laptop for the project.

[1] "BIG." Projects. Bristol Interaction and Graphics. Web. 20 Feb. 2015. <http://big.cs.bris.ac.uk/projects/mobile-kinect/>.[2] Nießner, Matthias, et al. "Real-time 3d reconstruction at scale using voxel hashing." ACM Transactions on Graphics (TOG) 32.6 (2013): 169.

Components

Hardware● iRobot Create 2 Programmable

Robot● Microsoft Kinect for Windows v2● Laptop● Assembled structure on robot● Power Supplies

Software● Depth sensing application● 3D model building software: Real-

time 3D Reconstruction at Scale using Voxel Hashing [2]

● Python script controlling robot

Design

Conclusion

No

Method Flow Chart

Device

Depth Processing: Left: The direct depth data from Kinect 2 camera, the X and Y values are the pixels. Right: depth data transformed into an occupancy grid in front of robot, the Y axis corresponds to the X axis of Left.

Above are three models created from the Voxel Hashing software, that was used as our robot navigated the room using the depth data processing to decide the path.

We have made great strides towards accomplishing the initial goal: to create a fully autonomous system capable of creating a 3D model of an unknown environment.

Successes● Combined different equipment into one functional

machine● Able to detect most objects in the robots path● Capable of constructing a 3D model

Future Work● Increase in collision avoidance functionality● Room completion detection