Nov 1 - 2 2005: Review MeetingACCLIMATE Overview of Unmanned Aerial Vehicle Researches in UC,...

Nov 1-2 2005: Review Meeting ACCLIMATE

Overview of Unmanned Aerial Vehicle Researches in UC, Berkeley

David H. Shim, Shankar Sastry

Project Manager, Berkeley Aerobot TeamUniversity of California, Berkeley

ACCLIMATE Review

November 2, 2005

http://robotics.eecs.berkeley.edu/bear





Acknowledgements

•Principal Investigator: Professor Shankar Sastry, Dept of EECS, UC Berkeley

•Project Manager:David Hyunchul Shim, Ph.D.

•Postdoctoral Researchers:

Jonathan Sprinkle, Ph. D. (embedded software)

Mike Eklund, Ph.D. (underwater vehicle, Boeing OCP)

•Graduate Researchers:

Hoam Chung (model predictive control)

Todd Templeton (vision-based geographic reconstruction)

Jongho Lee (time-triggered RTOS)

•Technical Support

Travis Pynn


Mission Statement

•Develop a comprehensive set of theories and technologies for the joint operation of heterogeneous autonomous agents and human participants.

•Implement scalable, effective, and reliable platforms for autonomous systems research

•Validate the proposed ideas on the developed


Research Fields

• Flight Control system identification, feedback law design• Guidance static/dynamic path planning formation, collision avoidance, pursuit-evasion• Sensing Enhancement vision-based landing, terrain map building, • Navigation: INS/GPS integration• System integration• Software development: OCP, Giotto• Communications


Exemplary Scenarios

• Perch and Move (DARPA Seedling project, 2004)Develop a set of methodologies to locate a radio signal source in a complex and changing environment using readily deployable unmanned aerial vehicles equipped with radio sensors

Fields of research- Sensing and Navigation

Collision avoidance Map building - Flight control automatic take-off

landing site selection - Vehicle Development

electric helicopter with payload sensors Operation in complex urban environment


Exemplary Scenarios

• Convoy Protection (Fort Hunter Liggett, 2005)Develop a set of UAVs that fly with the convoy, providing realtime surveillance video feed with various angles. The UAVs typically take off from a small pod and land on one of the convoy vehicle with high accuracy

Fields of research- Sensing and Navigation

Target detection/following Vision-based navigation

Collision avoidance Map building

- Flight control precision automatic take-off and

landing


Berkeley UAV Platforms


BEAR Fleet: rotorcrafts

Ursa Magna 1,2 (June 1999~present)Advanced navigation&control algorithm development platformMulti-agent scenarios, formation flight,Vision-based landing

Ursa Major 1 (Nov. 2002 ~)Low-cost, high-payload platformAggressive Maneuver, Vision-based landingMulti-agent scenarios, Model-predictive control

Ursa Electra (July 2003~present)Fully autonomous electric helicopter First-hand test vehicle for advanced concepts

Ursa Maxima (July 2000~present)High-payload platform for Multi-agent scenarios, formation flight, obstacle avoidance


BEAR Fleet: Ursa Magna1,2

Laser Rangefinder

UltrasonicHeight meter

Length: 3.5m Body Width:0.7mHeight: 1.08mDry Weight: 44 kg Payload: 20kgEngine Output: 12 hpRotor Diameter: 3.070mFlight time: 30 minAvionics operation time: 180 min

IntegratedNav/Comm Module

Wavelan Antenna

GPS Antennanow equipped with laser scanner in place of camera for OA experiment


BEAR Fleet: Ursa Maxima 1,2

Length: 3.63m Body Width:0.72mHeight: 1.08mDry Weight: 58 kg Payload: 30kgEngine Output: 21 hpRotor Diameter: 3.115mFlight time: 30 minAvionics operation time: 180 min

Laser Scanner on pan/tilt mount

Flight Control System

Wavelan Antenna

GPS Antenna

Laser scanner control computer

Equipped with onboard alternator to power avionics and sensors


BEAR Fleet: Ursa Major1

Avionics Enclosure

Length: 1.5m Body Width:0.7mHeight: 0.7mDry Weight: 7 kg Payload: 10kgRotor Diameter: 2.0mFlight time: 30 minSystem operation time: 60 min

Based on Bergen Industrial Twin Helicopter

Twin gas engine

onboard computer


Length: 2.2 m Width: 0.26mHeight: 0.41m Weight: 8.5 kgRotor Diameter: 1.8m

Flight time: 20 min (40V 8000mAh)System operation time: 90 min

Payload Device

Avionics

Ultrasonic sensorsFlight Battery

GPSAntenna

BEAR Fleet: Ursa Electra 1


Length: 1.2 m Width: 0.35mHeight: 0.7m Weight: 8.2 kgRotor Diameter: 1.5mFlight time: 20 minSystem operation time: 90 min

GPSAntenna

• IMU on vibration isolation mount

AvionicsNav-computerGPS receiverWireless comm

BEAR Future UAV Testbeds


BEAR Future UAV Testbeds

Small-size Coaxial Helicopter for indoor navigation

Payload: up to 1KG


Berkeley UAV System Architecture

UAV Onboard System

INS

Wireless LAN (EC/S)

OnboardPayload

computer

ReferenceTrajectory

AdjustedTrajectory

VehicleManagement

Computer

GPS

Ultrasonic

ContactSWs

GroundMonioring

Station

DGPS correction

ServoChannels

Air

fra

me

Other AgentsUAV

orUGV

EngineRPM

GPS BaseStation

Wireless Network Channel (IEEE 802.11b)

ExternalMPC Engine

Sensors

Scalable, dynamically configurable system for multiple heterogeneous agent scenario is achieved by

• Standard sensor/actuator configuration • Common code architecture• Standardized communication protocol and interfacing method

• Scalable ad-hoc wireless

network


Modified light weight scannerMaximum Detection Range: 50m

Laser Range Finder

Wireless Communication

Short Range: Wireless LANLong Range(up to 30mi): Wireless Modems

Pentium III 700MHzQNX RTOS

Flight Controller: ADL P3

IMU+GPS100Hz Update

Inertial Navigation Sensor

CMIGITS-II

2cm StDAccuracy20Hz Update

GPS: Novatel OEM4

Laptop/PDAWindows XP/Windows CE

Ground Control Station

Motorized scanner tilt mount

Berkeley BEAR Fleet: Ursa Maxima 1


Avionics Software Architecture

Wirele

ss E

thern

et

Radio Receiver/Servo Control Unit

ShMemPRTKBDQIGPS

INS

DQICONTMain loop

ShMemXe

RS-232NovAtel

GPS

ShMemusensor

VCOMM LSCANOnboard MPC

ComputerRS232 RS232

3D LaserScanner

I/O

Ac c

ess

4 Hz

100 Hz

OrinocoEC/S

RS

-23

2

DG

PS

corr

ection

ServoInterface

ShMemRXIN

46 HzProxy Proxy

ShMemLaser

GroundStation

GPSBase station

Wirele

ss E

thern

et

ExternalMPC Engine

Ground systems

• Multi-process architecture with IPC• Processes dedicated for major navigation sensors and actuators running at

nonuniform rates• Interprocess communication: POSIX-compliant shared memory with QNX-

proprietary proxy for thread synchronization• Telemetry via wireless modem or IEEE802.11b ad-hoc mode


Pioneer Ground Robots

• Hardware

– Micro controller: motion control

– Onboard computer: communication, video

processing, camera control

• Sensors – Sonars: obstacle avoidance, map building

– GPS & compass: positioning

– Video camera: map building, navigation, tracking

• Communication– Serial– Wave-LAN: communication between robots and

base station– Radio modem: GPS communication

GPS Antenna

Ultrasonic array

Lucent WaveLAN

PTZ Color Camera

GPS

4WD PT


BEAR Facility: Hydraulic Landing Deck

- Hydraulically actuated Stewart Platform

- Controlled by NI 6-axis motion control package

- Capable of reproducing ship motions under various types of waves

Designed by Tullio Cellano III


Mobile Ground Station

• Agent monitoring/commanding• Full duplex voice communication• DGPS broadcasting• Auto-tracking video camera w/ recording• Wireless video reception• Weather station• Hoist


Hand-held UAV Interfaces

Mobile Access and Command Extension (MACE) - “fly-by-fingertip” operation using touch screen- minimal obscurity monocle display- Real-time vehicle location & status monitoring- Waypoint programming


UAV Component Technologies


Berkeley UAV System Capability

• Waypoint navigation- preloaded/interactive waypoint navigation - high speed tracking mode- automatic take-off/landing

• MPC-based dynamic path planning - model-based trajectory generation- collision avoidance (TCAS)- aerial pursuit-evasion game

• Vision-based navigation and landing• Formation flight


Flight Mode-based Way-point navigation

• A helicopter flight can be composed with a number of flight primitives

such as take-off, hover, turn, vertical flight, forward flight, etc.• A script language (VCL) provides an abstraction layer from high-level

waypoint request to low-level vehicle control

Take- off Hover

Pirouette

Land Ascend/Descend

Forward Flight

Low-speed

X- Y flight

Bank- to-turn

GO {AUTO,MANUAL} : Change Flight mode to either automatic or manual

TakeoffTo <coord>{abs,rel} : perform autonomous take-off

Hover <coord>{abs,rel} {heading=<heading>{deg,rad}} <duration>{sec,min} : hover with given heading angle for given time

FlyTo <coord>{abs,rel} {vel <velocity>{mps,kmps,fps,knots,mph}} {passby,stopover} {autoheading}

: Cruise to certain way point stopping over or passing by

MoveTo <coord>{abs,rel} {vel <velocity>{mps,kmps,fps,knots,mph}} {autoheading}

: Visit a sequence of way points with fixed heading

Land : Perform automatic landing

** Helicopter flight is often more convoluted than a sequence of flight

primitives MPC-based waypoint navigator


VCL-based waypoint navigation

Perch & Move Flight Sequence (Aug 2004)


MPC-based Path Generation

Vision-based landing test platform

INS

Wireless LAN (EC/S)

NavStatus

ReferenceTrajectory

VehicleManagement

Computer

GPS

Ultrasonic

ContactSWs

GroundMonioring

Station

DGPS correction

ServoChannels

Air

fra

me

EngineRPM

GPS BaseStation

Wirelewss Communication Backbone

VisionSystem

WirelessLAN

Camera

CalibrationINS/GPS


Vision-based landing


Model-predictive approach

• A promising control methodology for a system with input saturation, system nonlinearity, and state constraints in a dynamic environment

• Stabilization and tracking problem is solved as an optimization problem of a cost function

• Tracking error and control energy are penalized over a finite time interval from present to future

• Control input saturation can be easily handled by simply forcing the saturation during optimization

• State constraints can be included by introducing appropriate cost functions


• A sequence of control {u(t)}, t=tk,…tk+h that minimizes the weighting

function over a finite horizon is computed every sampling time

• Only u(t) at t=tk is used for control output. The rest may be recycled as

the initial value to speed up the optimization at next sample hit

• Generally very expensive in terms of computing power (>100 MFLOPs)

but within reach of most latest CPUs (Pentium 4 class)

• Very efficient optimization algorithm is desired

Gradient-search method (Bitmead et al) is adopted here.



• Partially nonlinear system model is discretized (Euler)

• Cost function is defined as the sum of quadratic tracking error and

control input over finite horizon: t=tk … tk+N

where

Cost function for tracking error



MPC: input and state constraints

• Input saturation is enforced by bounding the input with maximum

control value during the optimization process

• State saturation is enforced by including the following state penalty –


MPC: application to urban navigation

• Obstacle avoidance is addressed by including the proximity penalty in

the cost function J

• At any given time, the nearest point to

adjacent buildings is found and used.• Although it sounds simple, it assumes

the full information of the surrounding

environment • A partial map would suffice for MPC

Distance to the wallR0


MPC: gradient-based approach

• The cost function is rearranged with Lagrange multiplier:

• We want to choose that minimizes J

By choosing such that

We have


Obstacle Avoidance: Motivation

• UAVs have been successfully deployed for numerous missions, mostly restricted to mid to high-altitude

• Near-future UAVs are expected to interact with the surroundings: active sensing and realtime trajectory replanning is necessary

• Many UAV programs require obstacle avoidance as a standard capability of UAV flight control system

(DARPA UCAR, OAV programs)


Obstacle Avoidance System

• Dynamic path planning: real-time path generation using model predictive control

• Sensing: onboard 3D laser scanner or preprogrammed obstacle maps

• Experiment system: Berkeley UAV architecture implemented on Yamaha industrial helicopter platform with 3D laser scanner

Key Elements


MPC-based Path Generation

Vision-based landing test platform

INS

Wireless LAN (EC/S)

DedicatedMPC

Computer(or Process)

NavStatus

ReferenceTrajectory

VehicleManagement

Computer

GPS

Ultrasonic

ContactSWs

GroundMonioring

Station

DGPS correction

ServoChannels

Air

fra

me Engine

RPM

GPS BaseStation

Wirelewss Communication Backbone

LaserScanner

VisionSystem

WirelessLAN

Camera

CalibrationINS/GPS

NavStatus

WayptRequest


20

40

60

80

-20

0

20

40

25303540

Collision pt:(50,0,33)ft

20

40

60

80

-20

0

20

40

25303540

Collision Avoidance using NMPTC

•Five helicopters are given a destination point.•The shortest (optimal) trajectory will lead to a collision.

•Each vehicle can detect other vehicles position only within the sensing/communication region.

•Can each vehicle fly safely and optimally?

Unsafe Desired Trajectory Resolved by NMPTC with Collision Avoidance


Collision Avoidance using MPC

•Two UAVs are intentionally set on a head-on crash course

• Model-predictive control-based trajectory planner computes safe trajectories with sufficient clearance in real time

• Each vehicle’s current coordinate is used for MPC at each computation

May 2003


Collision Avoidance using NMPTC


NMPTC for Dynamic Environment

1. Initial position 2. Popup wall

3. Ceiling lowers 4. column


NMPC in 3-D complex environment

NMPC

• The nearest point of the surrounding buildings found• The obstacle weighting function is applied.• Receding horizon approach is advantageous

than potential function approach when the

helicopter stuck in deadlock by including the finite

future behavior in the cost function


NMPC in a Complex 3-D Environment

Potential function

NMPC

Stuck in a local minimum

Break away from a local minimum point


NMPC in 3-D complex environment

Nov 1-2 2005: Review Meeting ACCLIMATE 0

10

20

30

40

50

-10

0

10

20

30

40

0123

Obstacle Sensing: map-based

• Map-based approach

- Obstacle map is measured and stored in the computer

- Upon request, the nearest obstacle coordinate is passed to

the MPC unit

- Sensing is always perfect,

thus eliminating any risks due to

sensing failure or any

unpredicted control behavior


Obstacle Sensing: SICK laser scanner

- An industrial two-dimensional laser scanner is used (www.sick.de)

-

It could be slightly heavy for most R/C helicopter-based UAV platforms

Custom weight reduction is available from a local machineshop service


Object Detection with Laser Scanner

Simulated Urban Objects

UAV in the air

Electric powerlines

Trees

Powerpole

- Laser scanner in general can detect natural and

artificial objects very well with great accuracy

(powerlines, buildings, trees… but not glass)

- active sensing: no lighting is required


Nearest-point Method

min/O BX

min

2( ) arg min

iO obs

iO ref O ref

S

X

X X X X

( ( )) ( ( )) ( ( ))trk obstq k q k q k x x x

1( ( )) ( ) ( ) ( ) ( )

2

Ttrkref refq k k k Q k k x y x y x

Cost Function:

where

min

min

2

2min

2 1

( ( )) (( ( ) ( ))

( ( ) ( ))

( ( ) ( )) )

obst Sobs

S

S

q k K x k x k

y k y k

z k z k

x

At each k over finite horizon, the nearest

point on the obstacle from the

hypothetic vehicle location is found


Local partial map building for MPC

Raw scan datafrom LaserScanner

Filter outinaccurate

measurements

Convert tolocal cartesian

coordinates

Find N1 pointsclosest to host

vehicle

Register in adatabase with

N2 points

Nearestobstacle pt to

MPC

Vehicle positionand attitude from

Host vehicle

Data is held in FIFO


Experiment Setup

• Testbed Ursa Magna 2 will be used for map-based OA Ursa Maxima 1 will be used for laser scanner based OA

• System Setup Vehicle onboard navigation system Ground Monitoring and Command Station Ground-based MPC engine running on MATLAB in realtime Ground-based 3D visualization station of laser scanner in realtime Wireless Ethernet communication among systems

•Crew Experiment Coordinator: in charge of experiment sequencing Safety pilot: in charge of routine flight and emergency take-over Ground Operator: assists Experiment Coordinator to monitor Ground station Camera Crew: takes video record of all experiment activities


Scanner Control Computer Tilt Mount- Encoder

- Servo- Micro controller

Flight Computer

Ground Station

- GPS+INS- PIII 700MHz PC104 module

- Real-time 3D Visualization

- PIII 700MHz PC104 module

MPC Engine

- Real-time optimization

Position Command

Encoder reading

Measured range data

Minim

um range data

Vehicle state

Reference trajectory

Navigation dataMinimum range

data

Light weight2D Laser Scanner

Measured range data

- 361meas/scan

Obstacle Sensing using Laser Scanner


Urban Flight Experiment

North

10’ X 10’ Easy-up Canopy

• 6 canopies to simulate urban environment• Secured by stakes at four corners• Resistant to wind gust of rotor downwash• Sufficient distances each other for helicopters to fly through

Original path

Adjusted path by MPC

Vehicle Launching Pt.

Ground Station

Obstacles

Richmond Field Station, UC Berkeley, Richmond, California


DARPA/Berkeley Press Release (Dec 2004)


Urban Flight Experiment: birdeye view


Urban Flight Experiment: on ground


Aerial Pursuit-evasion Scenario

• Pursuer wants to position itself in a good position to “shoot down” the evader, e.g., follow the target’s tail and align its heading with the relative position vector, XE-XP

• Evader wants to shake off the pursuer, e.g., get out of the hotspot

• Pursuer and evader avoid colliding into each other within a closed 3-D space

• State variables such as roll and pitch angles are constrained


Helicopters in Pursuit-Evasion

• Pursuer and Evader in a closed 3-D space with the additional cost

Jonathan Sprinkle and Mike Eklund, CDC 2004


Helicopters in Pursuit-Evasion


SEC Capstone Demonstration on PEG

• Capstone Demonstrations were proposed to highlight and test the technologies developed in the DARPA SEC program

• One would be a fixed wing UAV flight test– 8 participant technology developers (TDs)

• Honeywell, Northrop Grumman, U Minnesota, MIT, Stanford, U Colorado, CalTech and UCB

– System Integrator was Boeing– OCP would be software framework– A T-33 trainer as UAV surrogate– An F-15 as wingman/opponent

• 12-14 month schedule May 03 – June 04• UCB had two project/experiments By Jonathan Sprinkle and Mike Eklund


Flight Test: Range Area R-2515

Ingress

Endpoint

36 nmi

26 nmi

By Jonathan Sprinkle and Mike Eklund


Flight Test: Range Area R-2515

By Jonathan Sprinkle and Mike Eklund


Additional Slides


Vision-based landing: motivation

• Challenges• Hostile operating environments

– high winds, pitching flight deck, ground effect

• UAV undergoing changing nonlinear dynamics

Why the vision sensor?• Passive sensor (for stealth)• Gives relative UAV motion to

flight deck

Goal: Autonomous UAV landing on a ship’s flight deck

U.S. Navy photo


Ship deck landing problem (ONR 2002)


Formation Flight: Mesh Controller

• Mesh Controller tasks:– Obtain the leader and 2 neighboring helicopters’ current positions

– Compute mesh stable trajectories based on the acquired position information and send commands to the navigation computer

FlightComputer

MeshController

RS-232Wireless

Token Ring

Neighbor 1

Neighbor 2

Leader

On UAV


Experiment: Mesh Controller (2002)

Nov 1 - 2 2005: Review MeetingACCLIMATE Overview of Unmanned Aerial Vehicle Researches in UC,...

Documents

Transcript of Nov 1 - 2 2005: Review MeetingACCLIMATE Overview of Unmanned Aerial Vehicle Researches in UC,...