Mitsubishi Robotic Arm
Transcript of Mitsubishi Robotic Arm
![Page 1: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/1.jpg)
Vision-based Robot Drawing-- A case study of Human-Machine Interaction by Melfa RV2-SD
1
Haoran XieObuchi lab
&. Igarashi lab
![Page 2: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/2.jpg)
Human-Machine Interaction
excavator.
Traditional for experts, skill training, programming
control unit.
Intelligent for common user, no special skill, from physical states or emotions [1]. Stanton, et al. Teleoperation of a humanoid robot using full-body motion capture, example movements, and machine learning, Robotics and Automation, 2012[2]. Ponce, et al. Dancing Humanoid Rob ots: Systematic use of OSID to Compute Dynamically Consistent Movements Following a Motion Capture Pattern, IEEE Robotics, 2015
[1] [2]
![Page 3: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/3.jpg)
HMI Types
Computer VisionSpeech Recognition
Lego sound sensor Electro encephalography
Signal Processing Sensor FusionComputer Science
Devices & Sensors
Categories
game controller
PHANTOM Omni
Pressure sensors
Gesture Recognition
[3] Cannan, et al. Human-Machine Interaction (HMI): A Survey, University of Essex, 2010
![Page 4: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/4.jpg)
Motivation
Human Movement Motion Projection
![Page 5: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/5.jpg)
![Page 6: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/6.jpg)
Environments• Webcam
• Camera iBuffalo BSW20KM11• Identify object from 2D scenes
• Robot arm• RV2-SD• Joint coordinates (inverse kinematics)• Pick up object and put to desired location• Melfa Basic V programming
• Toolbox• (matlab) image acquisition toolbox• (matlab) image processing toolbox • (matlab) robotic toolbox – inverse kinematics• (matlab) Mitsubishi Melfa robot control toolbox – robot communication• (Mitsubishi) RT toolbox – offline evaluation
![Page 7: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/7.jpg)
Algorithm
• 1. Capture video input • Motion capture of human movement in 2D
• 2. Object identification• Recognize the designated object by webcam
• 3. Trajectory generation• Generate the trajectory of movement
• 4. Code generation in Melfa Basic V• Make the program in Basic V programming
• 5. Robot control unit transform / communication• Communication via USB/TCP connection
![Page 8: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/8.jpg)
Guiding System: Motion CaptureWeb camera
![Page 9: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/9.jpg)
Color Recognition
• Image crop• Avoid trivial info
• Sharpen image
• Convert to grayscale
• Color subtraction• green
![Page 10: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/10.jpg)
Shape Recognition
• Catch image boundary
• Get bounding box
• Calculate shape compactness• Compactness c = perimeter * perimeter / area• If shape is circle c = (2*pi*r)^2/(pi*r^2) = 4*pi• If shape is square c = (4*r)^2/r^2 = 16• If shape is rectangle c > 16
![Page 11: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/11.jpg)
Trajectory generation
• Desktop• Size range
• Web camera• Location
• Robot arm• Location of robot base
• Object position• From pixel to location (millimeter)
• Mapping
![Page 12: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/12.jpg)
Object recognition
![Page 13: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/13.jpg)
Projection System:Robot ControlMelfa RV2-SD robot arm
![Page 14: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/14.jpg)
Robot Configuration
![Page 15: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/15.jpg)
Melfa Basic V programming
![Page 16: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/16.jpg)
Programming example
![Page 17: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/17.jpg)
Programming Interface
![Page 18: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/18.jpg)
ControllersTeach Box
Emergency Stop
Emergency Stop
![Page 19: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/19.jpg)
Program Error
![Page 20: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/20.jpg)
Inverse KinematicsINPUTThe length of each linkThe position of some point on the robot
OUTPUTAngles of each joint needed to obtain that position
Using robot toolbox
![Page 21: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/21.jpg)
Examples
INPUT
Case 1
Case 2
Trajectory Inverse Kinematics
21
![Page 22: Mitsubishi Robotic Arm](https://reader031.fdocuments.us/reader031/viewer/2022030321/5870d2b51a28ab613f8b9c2f/html5/thumbnails/22.jpg)
Conclusion
• A guiding system using image processing
• A projection system using inverse kinematics