Microsoft Robotics Soccer Challenge: Movement Optimization...

30
Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1 1 Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Mestrado Integrado em Engenharia Electrotécnica e de Computadores João Oliveira

Transcript of Microsoft Robotics Soccer Challenge: Movement Optimization...

Page 1: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 11

Microsoft Robotics SoccerChallenge: Movement Optimization

of a Quadruped RobotMestrado Integrado em Engenharia Electrotécnica e de Computadores

João Oliveira

Page 2: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 22

Presentation Outline

� Introduction

� Objectives� Robotic Soccer

� Optimization and Metaheuristics

� Project and Implementation

� Results and Discussion

� Conclusion and Future Work

Page 3: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 33

Introduction

� What is a robot?

� One definition may be that it is a physical agent that generates intelligent connection between perception and action

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 4: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 44

Legged Locomotion

� Two main means of locomotion: legs or wheels.

� Why is legged locomotion so common among animals?

� Why isn’t it widely used in human built machines?

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 5: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 55

Legged Locomotion

� Legged locomotion requires full coordination

� Complex task to control

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 6: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 66

Objectives

� Improve the movement of a four-legged robot, using a set of optimization algorithms

� Understand which optimization algorithms can be used in the optimization of the movement of a four-legged robot

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 7: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 77

RoboCup

� International robotics project that started in 1997

� Provides a standard problem where a wide range of technologies can be integrated and examined

� Four major types of competitions: RoboCup Soccer, RoboCup Rescue, RoboCup Junior and RoboCup@Home

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 8: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 88

RoboCup

“By mid-21st century, a team of fully autonomous humanoid robot soccer players shall win the soccer game, comply with the official rule of the FIFA, against the winner of the most recent World Cup.”

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 9: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 99

Microsoft Robotics Soccer Challenge

� Started in 2007� Aims to promote the Microsoft Robotics Studio with the

potential of the RoboCup� 2 vs 2 games of simulated robuDOG robots� Simplified environment� Complexity of the robot turned this league into a big

challenge

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 10: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1010

Microsoft Robotics Soccer Challenge

robuDOG

� Developed with theRoboCup in mind

� Front legs designed to kick the ball with betteraccuracy

� Powerful 3D SimulationEnvironment

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 11: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1111

robuDOG

� Single movement:� Front right� Front left� Rear right� Rear left

� Leg movement:� Shoulder: forward and

backward� Shoulder: left and right� Knee: forward and

backward� Ankle (front legs only)

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 12: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1212

Optimization and Metaheuristics

Optimization

� From an initial model, its parameters are progressively refined in the light of experience.

� The goal is to determine the maximum or minimum value of some function of one or more variables.

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 13: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1313

Optimization and Metaheuristics

Metaheuristics

� Methods used to search the solution space with a strategy to find optimum solutions

� Incorporates concepts based on biological evolution, intelligent problem solving, mathematical and physical sciences, nervous systems and statistical mechanics

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 14: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1414

Hill-Climbing

Hill-Climbing Algorithm:

1. Generate an initial solution.2. Loop until a stop criterion is met or there no untested neighbour solutions:

(a) Test a neighbour solution to the current one that hasn’t been tested yet

(b) If the new solution is better than thecurrent solution make it the current solution

3. Return the current solution

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 15: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1515

Hill-Climbing

Weaknesses

� Local maximum� Plateau� Ridges

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 16: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1616

Tabu Search

� Allows non-improving moves� Keeps a tabu list so that the algorithm doesn’t

go back to past solutions� With this it tries to solve the weaknesses of the

Hill-Climbing Algorithm

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 17: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1717

Simulated Annealing

� Similar to the atoms rearranging themselves in cooling metal

� It can accept worse solutions with a probability dependent of the “temperature”

� It can escape local optima to find the global optimum

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 18: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1818

Project

� Slow process, and high number of iterations needed to produce results� This eliminated the Tabu Search and other

algorithms that would require more computation time

� Focus on rotation movement� Focus on reducing iteration time and noise

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 19: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 1919

Project

� Metaheuristics tested:� Hill-Climbing� A simplified version of

Tabu Search� Simulated Annealing

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 20: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 2020

Project

Simplified Tabu Search

� Tries to increase speed of optimization� Changes that improved

the solution before have a higher probability of being tested

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Hill-Climbing History list

Simplified Tabu

Search

Page 21: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 2121

Project

� Evaluation function for the rotation movement� Maximize rotation of the robot ( )� Minimize translation of the robot (p)

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

pSf ∆−∆= 100)( θ

θ

Page 22: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 2222

Results

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 23: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 2323

Results

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 24: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 2424

Results

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 25: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 2525

Results

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 26: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 2626

Results

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 27: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 2727

Results

� High iteration time, making it impossible or unpractical to test exhaustively for more complex movements

� Big influence of noise and variation between results, making it difficult to evaluate each set of parameters

� Improvements were nonetheless convincing

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 28: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 2828

Conclusions

� Metaheuristics are effective in this problem

� Initial solution is important, with a good one even the Hill-Climbing Algorithm performs well

� Noise and simulation time are problems that affect the effectiveness of this method

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 29: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 2929

Conclusions

Possible future changes

� Find a way to make the simulator faster, optimizing it for this approach, and using a faster computer

� Testing several times each solution to make the method more immune to noise

� After these problems are addressed, other algorithms can be used, like Tabu Search or Genetic Algorithms

Introduction | Objectives | Robotic Soccer | Opt. and Met. | Project | Results | Conclusions

Page 30: Microsoft Robotics Soccer Challenge: Movement Optimization ...niadr/TRIAD/2008/MSRoboticsChallenge.pdf · Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped

Microsoft Robotics Soccer Challenge: Movement Optimization of a Microsoft Robotics Soccer Challenge: Movement Optimization of a Quadruped Robot Quadruped Robot 3030

Microsoft Robotics SoccerChallenge: Movement Optimization

of a Quadruped RobotQuestions?

João [email protected]