Artificial Intelligence InCommodity Hardware Aerial Drones

Dept. of CIS - Senior Design 2012-2013

Nina Charness

chanina@seas.upenn.edu

Univ. of Pennsylvania

Philadelphia, PA

Charles L. Kong

ckong@seas.upenn.edu

Univ. of Pennsylvania

Philadelphia, PA

Matthew Rosenberg

rmatt@seas.upenn.edu

Univ. of Pennsylvania

Philadelphia, PA

Sam Shelley

sshel@seas.upenn.edu

Univ. of Pennsylvania

Philadelphia, PA

Camillo Jose Taylor

cjtaylor@cis.upenn.edu

Univ. of Pennsylvania

Philadelphia, PA

ABSTRACTRecent computer science research has been particularly inter-ested in autonomous or semi-autonomous unmanned aerialvehicles (UAVs) because of their application in situations toodangerous for human involvement. Modern artificial intel-ligence advancements have allowed for the implementationof highly complex spatial awareness algorithms, which en-able autonomous flight on a variety of di↵erent flying de-vices. However, these approaches usually involve aerial ve-hicles with expensive sensors or a large number of surround-ing cameras that are capable of extracting an entire 3D con-struction from multiple 2D images. Unfortunately, there aremany situations when financially, or locationally, expensivesensors are not a viable option.

This paper defines a set of improvements that can be madeto the AR Drone 2.0 quadrocopter, a simple flying vehiclewith limited hardware capabilities. These improvements en-able it to simulate some of the tasks which more expensiveimplementations have already achieved. Software alterna-tives for visual guidance and obstacle avoidance are achievedby modeling techniques used by insects, such as the bumble-bee, including object recognition, optical flow, and odometry.These improvements could be widely applicable and providegreater access to UAVs for situations where recovery is un-likely, or for other financially constrained applications.

1. INTRODUCTIONQuadrocopters (also known as quadroters, or quadcopters)

are flying vehicles propelled by four equally sized rotors. Thequadrocopter’s multiple rotors provide control redundancyand enable more precise maneuverability, allowing it to per-form aerial maneuvers impossible for a helicopter to achieve.Although they introduce additional control over flight, theadditional rotors dramatically increase flight complexity andconsequently require a flight computer and onboard sensorsin order to maintain in-air stabilization. The introduction ofsmaller and lighter electronics has made quadrocopters pop-ular choices for UAVs used for military reconnaissance be-cause of their application in situations too dangerous for hu-man involvement. Recently, component costs have dropped

even further, such that hobby companies have created moreaccessible quadrocopters such as the AR Drone 2.0 by Par-rot, which retails for $300 [1] (Figure 1).

The AR Drone 2.0 includes its own sensors and stabi-lizing algorithms and provides hobbyists the ability to flytheir own quadrocopter using their computer, Android, oriOS device. The drone has two video sources: one camerathat faces forward and one that faces downward. It also usessonar to help with altitude stabilization. While the size andweight of smaller vehicles like the AR Drone 2.0 serve as anadvantage in maneuvering through confined, cluttered envi-ronments, they do not include the sophisticated sensors andGPS chips used by larger UAVs. The subsequent sectionsoutline a series of improvements that can be made to thedrone so that, despite its comparatively rudimentary sen-sors, it can mirror some of the more complicated functional-ities of more costly quadrocopters and autonomous systems.

What follows is a framework for more advanced investiga-tion into artificial intelligence for quadrocopters and othervehicle systems with cheaper, more rudimentary sensors.These implementations and similar attempts will hopefullylead to more accessible quadrocopters, which if implementedusing cheaper components, could be used more broadly. Po-tential practical uses include search and discovery missionsfor local forces, which are currently unable to a↵ord theextremely expensive drones the military is using, and even-tually even delivery services.

In order to accomplish these goals, the most basic prob-lems associated with semi-autonomous quadrocopter flightneed to be solved, namely: odometry (determining one’s lo-cation and orientation), path re-tracing, obstacle avoidance,and starting point recognition. To achieve these goals, tech-nical research concerning the techniques used by bumblebeesand other insects for movement must be synthesized.

There is substantial related work in the field of computervision and object detection. The challenge is that many ofthese algorithms assume sophisticated hardware which flieswith precision and reports accurate sensor data. Althoughalgorithms have been developed and refined in the mathe-matical world, these algorithms require modifications in or-der to work with a real-world device due to computational

speed and inevitable inaccuracies in data returned by thedevice’s sensors.

This research presents proof-of-concept solutions to manyof the aforementioned problems, laying the groundwork forfuture exploration.

Figure 1: The AR Drone 2.0 [1]

2. RELATED WORKDue to the high costs of complicated visual sensors, many

researchers have used insects and animals as a guide for de-veloping more e�cient robotic behavioral patterns. There isevidence that insects, such as the bumblebee, use shortcutsto perform functions such as measuring distance travelledand obstacle avoidance and some of these shortcuts havebeen implemented in autonomous vehicles with varying de-grees of success.

2.1 OdometryIn order to survive, insects make many foraging excur-

sions, in which they locate food or supplies and then returnhome. Many scientists believe that they use an internal“odometer” to estimate the distance they have travelled toallow them to make roundtrips between home and a foodsource [15]. Odometry is a navigational technique in whicha moving body estimates its change in position over timebased on information gathered from its sensors. Bees useoptical flow techniques to infer distance by measuring im-age movement through their vision [13]. They use “visualflow,” the perceived movement across visual fields, for tasksranging from maintaining a straight path to regulating flightspeed. In robotics, odometry uses cameras, accelerometersand other synthetic sensors to simulate the biological onesfound in bees and other animals. In many past implemen-tations of robotic odometry, position tracking utilized GPSreadings. The AR Drone, however, does not possess a GPSchip so position tracking must be accomplished by integrat-ing the velocities provided by the SDK and multiplying themby a rotation matrix. The rotation matrix accounts for thedrone’s heading so that all position tracking is done with re-spect to the same frame of reference throughout the flight.

Recently with the increase in research into autonomousvehicles and robots, visual odometry has gained popular-ity. Visual odometry is the use of a camera to estimate thechange in distance. This technique is known as visual SLAM(Simultaneous Localization and Mapping) and is normallyaccomplished by calculating the change in position of pix-els between two consecutive images from the camera. Thedi�culty with this process is estimating the scale by whichthe distance between pixels should be multiplied in order toobtain distance in the real world [7].

2.2 Object DetectionIn real-life environments, e↵ective methods of object de-

tection are crucial for survival and navigation. These con-cepts are equally important for flying autonomous robots.Current technology can develop three-dimensional maps, heatmaps and x-ray images to detect obstacles by using multiplesophisticated sensors. However, processing the multitudeof data produced by a large number of sensors is computa-tionally expensive and time consuming. Other related workexists, which uses simpler algorithms targeted at monocularvision [18].

In order to move towards a target object, the target mustfirst be detected. Many methods for object detection involvesegmentation, which partitions the image into di↵erent re-gions based on some heuristic [12]. The goal of segmentationis to separate an image such that each pixel within a regionis similar in color, intensity or texture. The regions can thenbe separated into either object or background. Choosing anappropriate algorithm is key in performing successful seg-mentation.

One simple approach is color thresholding, which classi-fies pixels as part of the “object” if their value is within athreshold value [14]. If the color of the pixel is outside ofthe threshold, it is classified as “background.” The result isa binary image of object and not-object, where a pixel isset to 1 (or “on”) if it is part of the object and 0 otherwise.Although its relative simplicity makes it easy to implement,this technique performs poorly for multicolored images andobjects.

A more sophisticated approach focuses on finding objectedges. Edge detection is based on the observation that thereis often a sharp adjustment in intensity at region bound-aries [2]. The Canny edge detector is one such edge detec-tor, which is popular because it is easy to integrate with alarge number of object recognition algorithms used in com-puter vision and other image processing applications [5].One problem with edge detection is the presence of noise.Noisy images can cause the position of detected edges to beshifted from their true locations. Many edge detection tech-niques, including the Canny algorithm, are implemented inOpenCV, the Open Source Computer Vision Library [4].

Another method uses Haar features, image sections whichdescribe an object in a digital image, such as edges and lines,in order to detect objects. This method, first described byViola and Jones [16], creates many simple classifiers basedon Haar features from a set of training data. These classi-fiers are then applied sequentially to repeatedly disqualifyregions of an image from being an example of the targetobject. A large number of Haar-like features (derived froma large set of training data) are necessary to be su�cientlyaccurate in detecting objects.

The aforementioned techniques used for detecting specificobjects work best when qualities specific to the target objectare known in advance. In order to detect objects withoutknowing their qualities in advance, di↵erent algorithms arerequired. These algorithms focus on information gatheredfrom the pixels in an image or the change in pixels fromimage to image. These algorithms use this information tomake inferences about object positions.

A simple algorithm for ground vehicles with monocularvision segments an image into ground or obstacle by identi-fying every pixel in the image as either ground or obstacleand directing the robot to take the path of least hindrance

[2]. However, this algorithm assumes that the ground isalways of constant intensity and obstacles have sharp, dis-tinguishable features which allow the robot to easily discernbetween obstacles and non-obstacles. Such clear and con-sistent world definitions would likely be more di�cult todevelop for aerial vehicles.

A more e�cient option is a histogram-based method, whichcomputes a histogram from all of the pixels in the image [12].The peaks and valleys created can be used to locate objectsin the image based on color or intensity. This method canalso be recursively applied to objects in order to subdividethem. This option is more e�cient because it requires onlyone pass. Unfortunately, it may be di�cult to identify sig-nificant peaks and valleys in relatively homogenous images(or especially heterogeneous ones).

Another method uses the concept of appearance varia-tion cue in order to identify variations between textures in asingle image [8]. This can be used to detect impending col-lisions using the assumption that as an object moves closerthere is generally less variation in the image recorded froma viewpoint. However, this does not work in all cases, suchas when an object in an image has a detailed texture. Thistechnique may need to be combined with another objectavoidance algorithm in order to have more reliable decisionmaking processes.

2.3 Moving Towards Targets and AvoidingCollisions

In order to move towards a target object or avoid an ob-stacle, a system must not only detect objects, but resolveits position relative to the object in order to determine howto move towards or away from it. Visual looming is a phe-nomenon that has been used to determine if and when anobstacle collision could occur. Looming describes a situationwhen the projected size of an object increases on a viewer’sretina as the relative distance between the viewer and theobject decreases. Previous works have identified this phe-nomenon in insects, including bumblebees, and flying locusts[10]. Because visual looming signifies an impending threatof collision, it can be used for autonomous obstacle avoid-ance in robots [11]. Researchers have developed theoreticalmodels which try to measure this e↵ect quantitatively froma sequence of 2D images by studying the relative rate ofchange of image irradiance (the light it reflects when view-ing it from di↵erent positions). However, this property maybe di�cult to measure on vehicles with limited image pro-cessing capabilities.

A more generic method proposed to detect impending col-lisions is calculating the optic flow and inferring whether anobject is close based on the calculated values [10] [17]. Opti-cal flow is the apparent motion of an object, surface or edgecaused by the relative motion between the observer and itsenvironment. Insects not only use optic flow to maintain an“odometer” as discussed above, but also in obstacle avoid-ance. Optic flow is a function of the insect’s forward velocity,angular velocity, distance from an object, and angle betweenits direction of travel and the object. Objects that are closer,have higher optical flow values, so by turning away from re-gions of high optic flow, an insect can avoid imminent colli-sions. Optic flow research focuses on examining the opticalflow in images and estimating depth, velocity and motion ofan object with the flow information. In the research involv-ing a robot with two vertical cameras, the two frontal images

provided orientation information that allowed the robot toact based on its confirmation of the nature of an oncomingobstacle [17].

There are two approaches to calculating the optical flowbetween two images. Sparse techniques calculate the opti-cal flow of specific, pre-calculated image features, such ascorners, or edges. While these techniques are faster, densercalculations of optical flow look at every pixel in the im-ages, providing more accuracy [6]. In a paper by GunnarFarneback [9], dense and sparse optical flow techniques arecompared for their density. A sparse, quicker algorithmdeveloped by Lucas and Kanade, has 35 % density, whereFarneback’s has 100 % density.

Farneback’s dense optical flow algorithm estimates thedisplacement of pixels between two images [9]. Optical flowis therefore the magnitude of the displacement of one pixelfrom one image to the next. First, the algorithm approx-imates each neighborhood of both images (or frames) byquadratic polynomials. By using the polynomial expansiontransform, this calculation can be done e�ciently [9]. Thepolynomial expansion of both images gives expansion coe�-cients for the first and second image which are then used toestimate the displacement field of the image [9]. Final pixeldisplacement is estimated by integrating over a neighbor-hood of each pixel and smoothed out using Gaussian tech-niques. This is based on the assumption that pixels in prox-imity move in similar directions.

This algorithm assumes that the displacement field is slowlyvarying, which can lead to complications in a real-time en-vironment. In order to overcome this, segmentation tech-niques like those described earlier can be used in conjunc-tion. The algorithm produces a displacement field from twosuccessive video frames, which represents the optical flowat each pixel between two images. This displacement fieldcan subsequently be used to determine if an obstacle is ap-proaching which needs to be avoided.

3. SYSTEM MODEL

Drone Wifi Connection

Algorithms toprocess input

data form droneand convert to

directional output

Remote ProcessingDevice (Laptop)

Velocity, Video, Time data

Commands, Direction

Figure 2: The system model

A basic model which outlines the sequence of communica-tion to the drone is depicted in Figure 2. The drone receivesinput data 15 times per second from its onboard sensors andpasses the information through TCP (Transmission ControlProtocol) and UDP (User Datagram Protocol) WIFI con-nections to a remote device (for example, a laptop). The re-mote device analyzes the information provided by the droneand determines new instructions for the drone. Finally, itsends out a command back to the drone, and the cycle re-peats.

Applying previously described related work combined withvarious smoothing techniques during the analyzation stagecan produce a model which enables the drone to autonomously

navigate in its environment.

3.1 Environment and Testing ModelDrone testing occured exclusively indoors, and was con-

fined to a 5 meter by 5 meter area. For simplicity, only two-dimensional navigation was tested. The drone was given anarbitrary set of commands to go out (e.g. move forward 4seconds, turn right for 3 seconds, move forward for 3 sec-onds) that simulated a human instructing the drone usinga joystick or other manual control device. Once the dronefinished executing these commands the added autonomousflight algorithms were invoked.

The drone then navigated back to its starting location us-ing the recorded odometry information. Along the way ifan object was detected using optical flow, the drone exer-cised a predefined avoidance routine. The size of the objectsbeing avoided were bounded by the size of the drone’s cam-era (e.g. any object can be used as long as it fits withinthe drone’s vision) and were held stationary during flight.A predetermined object was placed at the drone’s startinglocation. When the target object was detected (in the thiscase a brightly-colored sticky note to improve color thresh-olding e↵ectiveness), the drone switched its control methodand used its camera to move close to the sticky note beforelanding.

As this task of autonomously returning to a target ob-ject while avoiding any obstacles on its way involves manycomponents, it is practical to divide the problem into threesmaller subgoals.

3.2 OdometryIn order for the quadrocopter to return to its starting po-

sition, it needs positional awareness - that is, after travelingin a certain direction for a certain period of time, it mustknow its current location (x, y coordinates) relative to itsstarting point.

This can be accomplished by taking velocity measure-ments either through accelerometer readings or by measur-ing image translation through a downward facing camera.Velocity measures can be integrated over time to find direc-tion and distance. For example, if a 2D world is assumed forsimplicity sake, then given v

x

and v

y

(x, y velocities), as wellas time, t, one can derive current position with the formulasx = x0(previous position) + v

x

⇤ t and y = y0 + v

y

⇤ t.With these measurements, a velocity and travel time can

be calculated which would return the quadrocopter to itsstarting point. In order to tackle this problem, the dronemust be able to model a very basic version of the shortestpath algorithm. For example, given that the drone is 3 me-ters north, and 4 meters east of the origin, it should be ableto rotate its head to point directly toward the origin, thentravel for 5 meters back to its starting position.

3.3 Locating an Object at the OriginAlthough the drone should be able to guide itself most

of the way back to its starting point using odometry, it islikely that these readings will not be su�ciently accurate toretrace its steps.

Therefore, in addition to odometry readings, the dronewill rely on a brightly-colored sticky note at its start point,used to signify the target location. Consequently, once thedrone is within range of the origin, it can use its frontward

facing camera to detect the sticky note and compute the di-rection it needs to turn or move towards in order to reachthe exact starting point. To accomplish this goal, the droneuses color thresholding for object detection and a simplisticform of visual looming for relative positioning.

Once the drone is capable of determining its physical dis-tance from the object on the screen, it can make algorithmicdecisions regarding the direction and speed it should main-tain in order to move away from or towards its target.

3.3.1 Color ThresholdingAs previously described, color thresholding algorithms de-

tect objects by segmenting an image into two color spaces,pixels that fall into a predefined color range, and those thatdo not. The algorithm’s output is thus an image that con-tains only two types of colored pixels, white and black.

Thresholding the image makes it easy to find the object’simage contours which in turns allows the description of theobject as a solid space, with a height, width, and a x, y po-sition that can be translated into an object.

One algorithm for contour detection (and the one that isimplemented in OpenCV) is described in a paper by Yokoi,Toriwaki, and Fukumura [19]. Their algorithm, titled “bor-der following” chooses positive points in a binary image, andexamines the point’s neighbors for other positive points. Ittakes these positives points and recursively examines neigh-bors until it has visited all positive points on the image.Within any discovered cycles of positive points in the image,all pixels adjacent to a pixel with a value of 0 (a backgroundpixel) can be considered part of the border line of the object(part of its contour) [19].

3.3.2 Moving Towards the Target ObjectInstead of trying to figure out the image’s position in the

real world, a computationally simpler technique is to estab-lish an “initial” x, y, height, and width of the target objectand then calculate the di↵erence between that position andthe object’s current position on the video stream. The dronecan then be instructed to move in the direction which wouldmove the target object closer to its “initial” position.

Determining whether the drone is too close or too far fromthe target can be accomplished by subtracting the currentwidth from the initial width. Determining its left, rightorientation, however is more di�cult. The two dimension-ality of the image means that there is no depth component.Left/right and up/down displacement must therefore be cal-culated by taking into account the change in the size of theobject due to its distance change in the third dimension.This can be accomplished by subtracting the current x ory position from the initial x or y position, and then addingback half the change in size since we know if the sticky notehas changed its size it is because it has moved closer or fur-ther away. These calculations are described in Figure 3

This technique enables a simplistic form of visual loom-ing, whereby the drone can move and repeatedly recalibrateits position relative to the target, based on the aforemen-tioned calculations. When the drone is within the range ofthe target, it can cease movement, and land.

3.4 Odometry and Obstacle AvoidanceWhile the drone is flying autonomously back to its ori-

gin point, it should also be able to account for obstacles

original location

current location

currentWidth

initialWidth

�x

Figure 3: Because the target object exists in threedimensions, the amount it should turn left or rightis a function of both the change in relative size aswell as the objects x,y o↵set, where offset = �x +(currentWidth� initialWidth)/2

introduced since it passed on its way to its initial destina-tion. This model treats obstacle avoidance and odometry asseparate problems, using odometry to move towards the ori-gin, and switching to an obstacle avoidance routine when anobstacle is detected. Now that the drone can theoreticallytrack its position at all times and find the origin, when an ob-ject is detected using optical flow movement, navigation canbe temporarily overridden to avoid the object it detects. Inthis scenario the drone follows a predefined avoidance pathto move around the obstacle, continuing to record positionchanges, and then resuming its path to the origin once theroutine is completed.

Obstacle detection is naturally a prequisite for being ableto avoid obstacles. As previously mentioned, specific objectdetection techniques will not work in this scenario, as thereare no predefined characteristics known about potential ob-stacles. For this reason, an implementation of optical flowis an e↵ective solution.

3.4.1 Optical FlowThe OpenCV library provides a variety of functions for

calculating optical flow between two images [4]. While thesparser algorithms look at the entire image and focus onthe optical flow of features (see Figure 4), the denser algo-rithms look at the optical flow for each pixel (see Figure 5).Though it is more computationally expensive to use a denseoptical flow algorithm, it provides a more accurate evalua-tion for an impending collision [6]. Given that the largerthe optical flow, the closer the pixel is, the magnitude ofoptical flow can be used to detect a potential object colli-sion [17]. A simplistic model treats obstacle detection asa boolean question. When examining di↵erent areas of thepixel map (or image) the algorithm calculates the sum ofthe optical flow in that region and determines if it exceeds apre-calculated threshold. If it exceeds the threshhold, the al-gorithm returns that an object is detected, and engages theappropriate commands to avoid the object before resumingmovement towards the origin.

4. SYSTEM IMPLEMENTATIONThe three problems discussed above assumed a perfect

world in order to give a general overview of the algorithmsthat will be used to solve the outlined tasks. In this section,

Figure 4: Optical flow using Lucas-Kanade sparseoptical flow algorithm for corners

Figure 5: Optical flow using Gunnar Farneback’sdense optical flow algorithm

a more detailed description and implementation is presented,using the hardware and SDK of the Parrot AR Drone 2.0.

4.1 AR Drone Move CommandThough it may sound trivial, commanding the drone to

move in a particular direction is not straightforward and isa strict prerequisite for implementing any of the presentedcomponents. As the SDK Development Guide states, thereis only one function call that is used to direct the drone tomove: ardrone_at_set_progress_cmd() [4]. This functiontakes as input four floating numbers, all between -1 and 1: �(left/right bending angle), ✓ (front/back bending angle), gaz(vertical speed), and yaw (rotational or angular speed). It isimportant to note that it does not take in time, or durationas parameters. For example, a sample call to the functionmay look like ardrone_at_set_progress_cmd(.5,0,0,0)

which tells the drone to bend rightward. However, becauseall the values are between -1 and 1, they represent the per-centage of the maximum corresponding values as set in thedrone parameters. So a value of .5 would mean for the droneto tilt and move to the right 50% of the maximum anglebending rightward. In order to instruct the drone to move aset distance, the function required calibration through a se-ries of trial tests to determine how many meters to the rightthe drone would go after the above sample command. Ad-ditionally, because these commands are instantaneous, theprogram must sleep for a few milliseconds after issuing a

command so that the drone will have su�cient time to exe-cute the instruction and realize its e↵ects.

This movement technique was used throughout each ofthe task implementations. Because of the lack of granularityfor movement commands, inaccuracies related to movementneeded to be taken into account for each of these steps.

4.2 OdometryIn Section 3.2, an algorithm that involved the basic inte-

gration of velocity over time was presented to achieve loca-tion tracking at any given point in time. However, there aremany more factors that need to be taken into account whenactually implementing this algorithm for the AR Drone 2.0.

The first thing that must be considered is the frame of ref-erence that is being used to measure the di↵erent velocities.The AR Drone 2.0 takes all of its measurements with respectto its own frame of reference, so it is critical that one makesthe necessary adjustments to transform these measurementsto the frame of reference of the world. For example, if thedrone goes 5 mm/s in its y direction and makes a 90� turnto the left and continues to go 5 mm/s in its y direction,in the world it is actually going 5 mm/s in the y direction,then 5 mm/s in the �x direction after the turn. To adjustfor this, a rotation matrix is needed used to transform thevelocities with respect to the drone to velocities with respectto the world. In 2D, this can be expressed as:

R =

cos✓ �sin✓

sin✓ cos✓

�

Once this rotational transformation has been applied tothe velocity vectors, they can integrated with respect to timeto compute relative position.

However, there are many technical issues related to thehardware that require additional smoothing algorithms inorder to make the measurements as precise as possible. Forexample, one issue is that the drone will produce an inac-curate velocity reading from the time to time as it tries tostabilize itself when it flies. To remedy this, rather than useall of the velocity readings to calculate position, a runningmedian is taken every ten readings to reduce noise.

Another issue involves the drone’s magnetometer, whichit uses to determine magnetic north, and thus aid it whencalculating angular orientation. Due to magnetic currentsthat occur indoors, occasionally the drone produces inaccu-rate angular readings. To remedy this issue, a threshold of30� per second was used. That is, if the drone produced tworeadings that were more than 30� apart within a second,the latter reading was ignored. This threshold was devel-oped during pilot testing of the drone, as it was discoveredthe drone could not physically turn this fast.

The overall process of determining its current locationis depicted in Figure 6. In the actual implementation, twothreads are used - a read thread where the input data (ve-locity, time) are being passed in to be stored and analyzed,and a write thread where output data (direction and mag-nitude) are being passed to the drone so it knows where tomove next.

4.3 Locating an Object at the OriginOnce odometry has been implemented with reasonable

accuracy, the second improvement requires the implemen-tation of video processing algorithms. As previously men-tioned OpenCV provides utilities and implementations of

Input: Specifiedpath (ex. go

forward 2 meters)

Receive velocityand time datafrom drone

Adjust for frameof referenceand integrate

velocity over time

Output: Currentposition of dronerelative to start

Read thread

Write thread

Figure 6: AR Drone Odometry process

many of the aforementioned algorithms. Its API was there-fore used to complete many tasks required for object detec-tion and eventually obstacle avoidance [4].

4.3.1 Video Processing with the ARDrone SDKOnce distance and time instructions can be converted to

AR Drone movement commands, video processing can beused to augment path accuracy.

Although in the past few years, OpenCV has been largelyrewritten in C++, it maintains a complete interface intothe library through C methods. As the AR Drone SDK iswritten entirely in C, it makes the most sense to use the Cinterface when implementing the following techniques [4].

The ARDrone Video SDK provides access to the videobu↵er through pre-decoding and post-decoding stages. Thesestages have access to the raw bu↵er of image frame data, andcan modify the data for subsequent stages in the pipeline toaccess.

The ARDrone outputs video streams in RGB (RGB888)or RGB565 formats. RGB565 is a 16-bit format whereas“standard”RGB is a 24-bit format. RGB565’s 16-bit permitsonly 65,536 unique colors, whereas the much larger 24-bitspace permits 16,777,216 colors. Using standard RGB is re-quired for integration with most libraries, including OpenCVand is consequently used in this implementation.

Once the data is accessible through a bu↵er, OpenCV pro-vides a framework which can display an image bu↵er in aGUI window. The OpenCV GUI library expects data inBGR format (the opposite of RGB), which requires that thedata be converted prior to display [4].

4.3.2 Color ThresholdingThe accuracy of color thresholding is highly dependent on

the range of colors provided to indicate the object to be de-tected. The color must be both well defined, and unique inthe general environment which is being recorded. For thisreason a bright pink sticky is used to maximize accuracy(Figure 7).

Although the RGB image format provides a good abstrac-

tion for color storage for humans (because this is how we per-ceive color), its ability to represent di↵erent shades of thesame color is rather poor (there is no clear axis along whichdi↵erent shades and hues can be described). Consequently,another color space, HSV (Hue, Saturation, Value) was de-veloped which enables manipulation of color data based onsaturation and value, which relate to the “lightness” and“darkness” of the color hue [14].

OpenCV provides an API for converting from RGB toHSV and another for returning a modified 1 channel im-age that contains only white or black pixels dependent onwhether or not the original pixel fell within the predefinedcolor range (Figure 7).

Figure 7: Color thresholded AR Drone Videostream with thresholding applied to a pink stickynote

Once a binary image representing the object the edges ofthe object can be detected with the help of OpenCV APIs,specifically cvFindContours. This method seeks out con-tours (edges) in the binary image by using the aforemen-tioned edge finding algorithm [19]. The returned contourscan then be drawn on the image either by using provided linedrawing methods, or an additional contour drawing method,cvDrawContours [3] (Figure 8). As color thresholding is bydefinition very sensitive to the environmental changes, theresulting borders can be very noisy, and generate false pos-itives. This can be rectified by setting a minimum area ofthe object. Although this reduces the distance at which anobject can be detected from, it dramatically reduces detec-tion of stray areas of color within the defined threshold.

All of the aforementioned procedures were implementedin a separate video thread so as not to interfere with therunning time of the other procedures.

4.3.3 Moving Towards the Target ObjectColor thresholding can be combined with odometry to en-

able movement towards an target object once it has beendetected. In theory, this requires implementation of Section3.3.4, however practical implementation required additionalconsiderations.

Firstly, a clear definition of “detection” is needed. In prac-tice, when the object is at the edge of its range for accu-rate object detection there are situations in which the dronewould briefly detect the object, but then fail to detect itin subsequent frames. Moreover, occasional visual artifactswould introduce situations in which the aforementioned al-gorithms would generate false positives on objects that werenot the target destination. To rectify this issue, an addi-

Figure 8: AR Drone Video stream with green con-tours drawn around detected color thresholded area

tional threshold was added that required that the targetappear in multiple sequential frames, prior to switching tothe targeting portion of the system.

Secondly, problems related to consistency of the videostream, caused similar detection problems and introducedsituations in which blank frames would cause the object toincorrectly “disappear.” This made it impossible to distin-guish between situations in which the object had actuallymoved out of drone’s view (a situation that would warrantan emergency landing) and situations in which a frame hadbeen temporarily dropped. Similar to the previous solution,thresholds were added to ensure that the sticky note wasgone for multiple frames prior to engaging in an emergencylanding.

Once these problems had been rectified the drone couldbe instructed to move and then repeatedly rexamine its po-sition ever 10 milliseconds to determine if it had reached itstarget. Due to the inaccuracies related to the drone’s move-ment commands, a range was established to ensure that thedrone would not repeatedly hover in place, unable to moveaccurately enough to reach its target location.

4.4 Odometry and Obstacle AvoidanceOnce odometry and target detection were implemented,

obstacle avoidance could be combined to produce an au-tonomous drone.

A general flow chart describing the task of odometry, ob-stacle avoidance and object detection for origin targeting ispresented in Figure 9. It depicts how the drone uses itsread thread to retrieve data to compute the next directionof travel and detect both the target and potential obstacles,while its write thread tells the drone to go to the destinationand back based on the computed next direction of travel. Ifat any point an obstacle is detected, the write thread in-structs the drone to navigate around the object before con-tinuing. Finally, once the drone can detect the sticky note atthe origin, odometry stops, and the drone continously movestowards the detected object, before landing.

4.4.1 Optical FlowAs video is simply a bu↵er of images, the ARDrone SDK

provides access to the image bu↵er of the video being recordedby the drone. Gunnar Farneback’s previously dense opticalflow algorithm discussed was used to calculate the opticalflow between the previous image and the current image for

Input: Destination[Ex. 4 meters,30 degrees NWfrom origin]

Commanddrone to fly incomputed next

direction of travel

Compute currentposition and nextdirection of travel

Obstacle detected?avoid obstacle,then reengage,

otherwise continue

Reachorigin/detect

sticky?

Switch to stickyhoning algorithm

Is drone closeenough toorigin?

Commanddrone turn/flytowards originand then land

Receive realtimeimage dataand compute

optical flow andattempt to detectsticky determinedistance from it

Write threadRead thread

No

Yes

No

Figure 9: Odometry with Target Detection and Ob-stacle Avoidance

each pixel [9]. This was achieved by calling the methodin OpenCV, cvCalcOpticalFlowFarneback which took twoimages, a previous image, and the current image and cal-culated the optical flow between the two in a 3 channel,32-bit floating point depth image. The function could alsobe given various values to implement di↵erent degrees andtechniques of smoothing. The previous image frame wastherefore stored in a bu↵er during each processing step, andthen accessed when the next frame arrived to process opti-cal flow. The resulting optical flow data was summed in aregion on the image and if it exceeded a threshold, a bit wasflipped indicating the presence of an obstacle that the droneneeded to avoid.

Unfortunately there were many di�culties associated withthe use of a dense optical flow algorithm. First, looking atthe pixel in each image and calculating its displacement iscomputationally very expensive. In order to e↵ectively avoidobstacles, object detection needs to operate fast enough for

the drone to respond in real time. This potential delay be-tween detection and the commands to avoid the object, isone of the key technical issues within obstacle detection andavoidance. For example, one can imagine the drone ap-proaching an obstacle which is 1 meter away - it gives thenecessary input to the program and after all its calculations,it determines that it is indeed 1 meter away from an obsta-cle. However, by the time the calculations are complete, thedrone has moved another .5 meters forward, and so the ob-stacle is actually half a meter away when the drone receivesthe next instructions, which are now invalid.

A few techniques were implemented to improve the speedof the optical flow algorithm. The resolution of the imagewas reduced so that there were fewer pixels for which tocompute displacement. Another assumption that increasedthe processing speed was to further crop the video stream,assuming that there existed a region of collision in the centerarea of the drone’s viewport. Once this portion of the imagewas determined (see green box in Figure 5), the rest of theimage was cropped, so that the reduced area could be usedwith Farneback’s optical flow algorithm. A final optimiza-tion was made by reducing the number of frames for whihflow was calculated, instead calculating optical flow only ev-ery other video frame. By calculating optical flow less fre-quently, the drone no longer entered situations in which itfell further and further behind the live image stream. Thesetechniques, however, only partially solved the problem ofdelay.

The most e↵ective solution involved introducing an inter-

0 1 2 3 4

0

1

2

3

Time (seconds)

xposition(m

eters)

Actual

Delay

Figure 10: Simple portrayal of hovering e↵ect. As-sume a 1 second delay for time it takes to send datafrom drone to program and for program to calcu-late current position. Assume obstacle is 4 metersaway, and obstacle can be detected within 2 meters.Red represents actual path of drone, Blue representsperceived path of drone by program after delay. Att = 2, drone is at x = 2 meters and therefore seesobstacle at x = 4 meters; however program only re-alizes this at t = 3, but it tells the drone to hover sodrone still remains at x = 3 while program catchesup. This problem was repeatedly faced while calcu-lating optical flow.

mediary step between input and output, which will be called“hovering” or “syncing.” In this step, the drone hovered inplace without movement in any direction. This step enabledthe program to sync up to the current position of the drone.For instance, in the above example, once the drone first de-tected the existence of an obstacle in front of it, it wouldsend the corresponding information to the program. Thedrone would still have moved half a meter by the time theprogram calculates that there was an obstacle 1 meter infront of it, however, at this time the program would tell thedrone to hover in place so it could sync up and collect themost recent data and derive that it is actually only half ameter away (Figure 10).

The second di�culty is that using a dense optical flowalgorithm and examining all pixels, instead of just featuressuch as the corners in an image, causes the calculated op-tical flow values to be extremely sensitive to the environ-ment. Consequently, the optical flow for the same scenario(drone, obstacle, path) varied greatly, depending on whetherthe tests were conducted in a dark or well-lit room (due tolight particle movement). Thus, the optical flow threshold atwhich the algorithms decided an obstacle was present werepre-calibrated for each testing environment.

This e↵ect could be somewhat reduced by cropping theimage to reduce the number of pixels. Using this technique,the optical flow values remained more consistent, unless thelight source was directly in front of and facing the drone

Finally, in order to reduce variance futher, and achievemore consistency, Farneback’s optical flow algorithm wascombined with a segmentation technique, greyscaling theimage. In the implementation, the images passed to cvCal-

cOpticalFlowFarneback are first converted to grayscale toreduce random pixel variance. More sophisticated segmen-tation techniques, such as Canny edge detection discussedabove, could potentially have produced even more consistentresults, but carry an additional computational cost.

The aforementioned modifications mitigated some of thee↵ects of the two mentioned problems, enabling optical flowto be successfully integrated into the system.

5. SYSTEM PERFORMANCEAs the system for autonomous flight was implemented as

standalone modules, they were also evaluated individually(in addition to being integrated together)

5.1 OdometryThere were two essential tasks that the drone was pro-

grammed to perform. The first task was for the drone toknow its current location at all times. To test this, thedrone was given commands to fly to an arbitrary point, andthe di↵erence between the actual position and predicted po-sition by the drone was measured. Success was evaluated interms of percent error of total distance travelled, as shownin Table 1. For example, looking at the first trial, the dronebelieved that it was at 3.84 meters in front and .63 me-ters to the left of its starting location, when it was actually3.86 meters in front and .7 meters to the left of the start-ing location. The total distance travelled of 3.923 is simplythe pythagorean distance of its actual location to the origin(0,0). The di↵erence of .082 is the pythagorean distance be-tween the predicted location and actual location. Finally,the error of .021 is the di↵erence divided by the total dis-tance travelled (.082 / 3.923).

A total of ten trials were performed to measure how accu-rate the drone was in predicting its current location. In theend, the drone had on average a 5.1% error in measurement,or a 94.9% accuracy in predicting its location at any time.

Table 1: Accuracy of Knowing Current Location

Pred. X Pred. Y Actual X Actual Y Distance Di↵ Err.

3.84 .63 3.86 .7 3.923 .082 .0212.52 -1.96 2.46 -1.85 3.078 .125 .0412.67 2.34 2.64 2.12 3.386 .222 .0663.5 .25 3.4 .15 3.403 .141 .0422.4 1.6 2.25 1.56 2.738 .155 .0574.4 1.7 4.62 1.9 4.995 .297 .063.7 3.5 3.55 3.65 5.092 .212 .0423.8 -2.6 3.7 -2.4 4.41 .224 .0512.9 2.2 2.8 2.4 3.69 .224 .0612.75 -1.5 2.88 -1.7 3.344 .239 .071

The second task was for the drone to be able to returnto its starting location at any time. The drone was firstgiven commands to fly to an arbitrary point. Then, it wascommanded to return to the origin - to accomplish this, itfirst calculated the correct orientation it had to turn, andafter orienting itself correctly, tried to fly straight back to itsstarting location. Because the drone naturally drifts left orright when it turns or moves, it was programmed to adjustits course every second to account for these unpredictablemovements.

To evaluate the success of this task, again percent errorof total distance travelled was used, as shown in Table 2.For example, looking at the first trial, the drone travelled adistance of 7.96 meters (3.98 meters each way). It landed.25 meters away from its starting location, and as a result,had an error of .31 (.25 / 7.96), or 3.1%. A total of ten trialswere performed to measure how accurate the drone was inreturning back to its starting location. In the end, the dronewas on average 4.1% o↵ from the starting location.

Note that these trials were performed in situations inwhich the magnetometer was able to correctly calculate mag-netic north. When it was unable to do so, it was impossileto calculate angle, and consequently impossile to determinecorrect facing (despite knowing the correct x, y displaymentfrom the origin.)

Table 2: Accuracy of Returning to Origin

Total Distance Distance AwayFrom Origin

Error

7.96 .25 .0317.22 .35 .0485.9 .05 .0085.96 .25 .0428.4 .4 .0486.92 .3 .0436.6 .35 .0535.4 .16 .0295.0 .26 .0526.8 .37 .054

5.2 Locating an Object at the Origin

Once the aforementioned smoothing techniques were im-plemented, successfully moving to the sticky note at the ori-gin depended on how close the drone was to the target whenthe odometry step terminated. The accuracy of odometryaided this process greatly, but tests showed that the stickycould be detected from a substantial distance, providing apermissible margin of error if odometry were to malfunc-tion and produce a lesser degree of accuracy. Table 3 showsthe distance at which the sticky note was first detected as afunction of the size of the sticky note.

Once the sticky note was successfully detected, moving to-wards the target succeded 100% of the time over the courseof over 10 trials, provided that the sticky did not move outof the viewport and no additional objects falling into the de-fined color threshold entered the viewport during the move-ment process.

Although the drone was able to navigate to the targetevery time, the path it took varied depending on when thedrone first engaged with the sticky. This would a↵ect whetheror not the drone would accidentally overshoot the targetduring movement commands and have to readjust in theopposite direction. These readjustments slowed down theprocess overall, but did not a↵ect whether or not the dronewas ultimately successful in reacting the target and landing.

Table 3: Maximum Distance for Target Detection

Distance at Which Target isDetected

Size of Target

186 cm 7.4 cm2

215 cm 10.1 cm2

313 cm 14.5 cm2

5.3 Obstacle Detection and AvoidanceThe success of obstacle detection and avoidance greatly

depended on the speed and the correct calibration (giventhe environment) of the optical flow algorithm. As dis-cussed in Section 4.4.1, detecting an obstacle, required thatthe threshold of optical flow (to indicate an impending ob-stacle) greatly depended on the environment in which thedrone was flying. If there was a lot of light in the room thecalculated optical flow value threshold needed to be muchhigher than if calculated in a darker room. Depending on theenvironment the drone was being flown in, the optical flowalgorithm had to be recalibrated by changing the thresholdvalue.

While the ability to detect an object was determined bythe calibration of the optical flow algorithm, the ability toavoid an object once detected depended on the response timeof the drone. The response time was limited by the computa-tional speed of the optical flow algorithm and the computeron which it was running. If the delay between obstacle de-tection and the avoid command were too large, the dronewould not respond in time to avoid a collision.

In tests, the ARDrone was given 3.5 seconds to fly straightwith an obstacle in its path. It was instructed to land whenit had detected an object and determined that they wouldcollide. Table 4, contains data related to the sequential im-provements made to improve the computation time of op-tical flow discussed in Section 4.4.1. The first column indi-cates the speed-up technique and the last column indicates

whether the drone landed before colliding with the obstaclewhen using this technique with all previous.

Table 4: Response Time Based on Algorithm Im-provements

Improvement Delay(seconds)

Respond BeforeCollision

Reduce image resolution >3.5 NoReduce number of frames tocalculate optical flow from

1.0-3.5 Yes with far ob-stacle, no other-wise

Crop image to determinedsize (See Figure 5)

<1.0 yes

The delay column indicates the delay between optical flowcrossing the threshold (when an object is detected) and theresponse of the drone. In order to avoid obstacles, the droneneeds to respond to an object detection with enough timeto maneuver around it.

If the delay was larger than 3.5 seconds, the drone didnot respond before colliding. This was observed when thedrone would print that it had detected an object after the3.5 seconds it had been flying.

When the delay was between 1.0 and 3.5 seconds, thedrone would respond in time if the object was su�cientlyfar away. It was observed that the delay should be less thanabout 1.0 second to respond to obstacles at any distance.

When the improved optical flow algorithm is calibratedcorrectly for the environment in which the drone is flying,and the listed improvements were implemented, always de-tected an obstacle and executed the avoidance routine priorto colliding with the obstacle.

6. ETHICSThere are a number of dilemmas that accompany the act

of making drones with artificial intelligence more readilyavailable.

First, the making of inexpensive drones could potentiallyaid malicious individuals seeking to conduct illegal surveil-lance which harms both civilians or governments. By reduc-ing the barriers to entry for covert surveillance, this inex-pensive technology would allow anyone previously incapableof acquiring this type of technology to violate privacy andpotentially have implications for individual security.

Secondly, although these artificial intelligence algorithmsare primarily for navigation purposes, the physical drone isstill a dangerous device which can potentially cause harmto civilians in certain situations. One such situation wouldbe a drone navigating home which failed to correctly en-gage in obstacle avoidance and unintentionally collided witha human. The sharp rotors could potentially harm the per-son, thus raising questions of liability regarding the oper-ator, the manufacturer, and the engineer. This issue andmany other related to potential malfunction would need tobe thoroughly examined before using these drones in com-mercial or more residential settings.

Finally, the cheap components of the AR Drone enable thedrone to be produced in bulk and used with less of a focuson accidental or intentional damage done to any one drone.Although this redundancy is a powerful concept for systemdesign, it also introduces environmental concerns related to

the large scale use of materials to produce these drones andthe waste associated with their potentially short operationlife. Companies producing the AR Drone and other similarcheap drones in the future should take into account the en-vironmental impact of the materials used in the productionof the drone, avoid potentially hazardous materials (thatcould be left behind in case of failed recovery), and attemptto reduce overall materials usage to prevent overall waste.

7. FUTURE WORKMoving forward, the two primary tasks for future work

include modifying the algorithms implemented so that theyare more adaptive to the external environment and movingall of the computational processing to the drone’s onboardCPU.

7.1 Environment AdaptationCurrently, because of the limited hardware capabilties, the

drone can only accurately track its current velocities if it isflying over a high contrast surface. This is because the ve-locities the drone’s SDK reports are calculated based on amix of accelerometer data as well as image data retrievedfrom the downward facing camera. It is this image datawhich skews the velocities based on the surface contrast.To improve odometry on lower contrast surfaces, the ARDrone’s internal velocity calculation algorithms would needto be improved to rely less on the downward facing camera’srecognition of movement over a surface.

Color thresholding is e↵ective as an initial implementationfor target detection, however, in real-world environmentsit is almost impossible to choose a color range su�cientlyunique to isolate. Therefore, other detection methods, likethe aforementioned Haar classification should be explored toincrease detection accuracy.

Finally, when looking at the optical flow algorithm to de-tect obstacles, a threshold must be set for the sum of opticalflow sensed to represent an impending collision. However,the threshold that is set is highly variant on the amountof light in the environment. To make this algorithm moreadaptive, one option is to formulate a function to determinehow the threshold should vary based on light levels withinthe environment. Another option is to use an optical flowalgorithm like Farneback’s in conjunction with a segmenta-tion technique like those discussed in Section 2.2.1. Thiswould focus on the optical flow of features, which shouldvary less in di↵erent environments. This would also improvethe speed of the optical flow algorithm, allowing for fasterresponse times.

7.2 Onboard ComputationIn order to move all of the calculations onboard to the

drone’s CPU, all of the algorithms would have to be simpli-fied to decrease their computational intensity. During test-ing, the computations were processed on a 2.50 GHz 64 bitIntel Core i5 CPU with 8 GB of ram via the WiFi connec-tion. Even with these very fast components, the algorithmswere barely processed quickly enough for the drone to reactin real time. On the drone there is only a 1 GHz 32 bit ARMCortex A8 processor with 1 Gbit of RAM. Although moretime will be available for processing if the calculations aremoved onboard since the WiFi relay will be unnecessary, be-cause of the significantly decreased technical specificationsof the hardware onboard the drone, the algorithms will still

need to be much more e�cient in order to be processed inreal time.

8. CONCLUSIONDespite many of the listed caveats, autonomous flight on

the AR Drone is clearly possible, and additional work spentfine tuning and developing algorithms that are more adap-tive to the surrounding environment will open up a widevariety of beneficial use cases for this and other similar inex-pensive drone models. Inexpensive drone technology has thepotential to revolutionize surveillance and reconnaissance inthe short term as well as object delivery and search and rese-cue in the long term. Moreover, many of the sensor systemswhich are considered expensive now will eventually decreasein price, enabling even more sophisticated flight for thoseseeking cost e↵ective solutions.

Odometry, target detection, and obstacle avoidance arethe fundamental components of any autonomous flight sys-tem, and the described implementation provide a proof ofconcept, demonstrating that such techniques are employableeven with severely limited sensors. The success of these ba-sic implementations speaks to the potential market for thefuture of these aerial drone products, and the value thatthey can ultimate provide for a wide variety of importanttasks.

9. REFERENCES[1] AR Drone 2.0 (http://ardrone2.parrot.org).[2] A. Aggarwal, A. Kukreja, and P. Chopra. Vision

based collision avoidance by plotting a virtual obstacleon depth map. In Information and Automation(ICIA), 2010 IEEE International Conference on,pages 532 –536, June 2010.

[3] Dr. Gary Rost Bradski and Adrian Kaehler. Learningopencv, 1st edition. O’Reilly Media, Inc., first edition,2008.

[4] G. Bradski. The OpenCV Library. Dr. Dobb’s Journalof Software Tools, 2000.

[5] J Canny. A computational approach to edge detection.IEEE Trans. Pattern Anal. Mach. Intell.,8(6):679–698, June 1986.

[6] R. Chellappa, Gang Qian, and S. Srinivasan. Structurefrom motion: sparse versus dense correspondencemethods. In Image Processing, 1999. ICIP 99.Proceedings. 1999 International Conference on,volume 2, pages 492–499 vol.2, 1999.

[7] Sunglok Choi, Ji Hoon Joung, Wonpil Yu, and Jae-IlCho. What does ground tell us? monocular visualodometry under planar motion constraints. pages26–29, October 2011.

[8] G.C.H.E. de Croon, E. de Weerdt, C. De Wagter,B.D.W. Remes, and R. Ruijsink. The appearancevariation cue for obstacle avoidance. Robotics, IEEETransactions on, 28(2):529 –534, April 2012.

[9] Gunnar FarnebAd’ck. Two-frame motion estimationbased on polynomial expansion. In Josef Bigun andTomas Gustavsson, editors, Image Analysis, volume2749 of Lecture Notes in Computer Science, pages363–370. Springer Berlin Heidelberg, 2003.

[10] W.E. Green, P.Y. Oh, and G. Barrows. Flying insectinspired vision for autonomous aerial robot maneuvers

in near-earth environments. In Robotics andAutomation, 2004. Proceedings. ICRA ’04. 2004 IEEEInternational Conference on, volume 3, pages 2347 –2352 Vol.3, april-1 may 2004.

[11] K. Joarder and D. Raviv. A novel method to calculatelooming cue for threat of collision. In ComputerVision, 1995. Proceedings., International Symposiumon, pages 341 –346, nov 1995.

[12] G. N. Srinivasan and G. Shobha. Segmentationtechniques for target recognition. W. Trans. onComp., 7(10):1555–1563, October 2008.

[13] M.V. Srinivasan, J.S. Chahl, K. Weber, S. Venkatesh,M.G. Nagle, and S.W. Zhang. Robot navigationinspired by principles of insect vision. Robotics andAutonomous Systems, 26:203 – 216, 1999.

[14] George Stockman and Linda G. Shapiro. ComputerVision. Prentice Hall PTR, Upper Saddle River, NJ,USA, 1st edition, 2001.

[15] Juergen Tautz, Shaowu Zhang, Johannes Spaethe,Axel Brockmann, Aung Si, and Mandyam Srinivasan.Honeybee odometry: Performance in varying naturalterrain. PLoS Biol, 2(7), 07 2004.

[16] Paul A. Viola and Michael J. Jones. Rapid objectdetection using a boosted cascade of simple features.In CVPR (1), pages 511–518, 2001.

[17] K. Weber, S. Venkatesh, and M.V. Srinivasan. Insectinspired behaviours for the autonomous control ofmobile robots. pages 226–248. Oxford UniversityPress, 1997.

[18] Jonas Witt and Uwe Weltin. Robust real-time stereoedge matching by confidence-based refinement. InChun-Yi Su, Subhash Rakheja, and Honghai Liu,editors, Intelligent Robotics and Applications, volume7508 of Lecture Notes in Computer Science, pages512–522. Springer Berlin Heidelberg, 2012.

[19] Shigeki Yokoi, Jun ichiro Toriwaki, and TeruoFukumura. An analysis of topological properties ofdigitized binary pictures using local features.Computer Graphics and Image Processing, 4(1):63 –73, 1975.