Using mmWave sensors to enhance drone safety and productivity

Dennis BarrettMarketing manager

Dan WangSystem engineer

Adeel AhmadSystem engineer

Vaibhav MahimkarApplications engineer

Texas Instruments

Using mmWave sensors to enhance 2 May 2017drone safety and productivity

Overview

Aerial drones have taken flight, from package-delivery trials, to entertainment,

sporting events and their generous availability at any electronics store. By 2022,

the global unmanned aerial vehicle (UAV) market is forecasted to reach more than

USD $21 billion[1] and drive a global market economy for business services valued

at over $127 billion[2]. Industries that will be rapidly disrupted by drones include

infrastructure, agriculture, transportation, security, entertainment and media, insurance,

telecommunications and mining.

Designing and customizing drones for specific

industries optimizes productivity, enhances safety

and lessens the environmental impact of current

methods. Drone designers do face a myriad of

design challenges to realize real-world deployment,

however, including:

• Operation in all conditions. Grounding drone

flights because of weather, lighting conditions,

visibility obstructions caused by smoke or fog,

or literal obstruction caused by trees and other

objects can significantly impact the actual

productivity of drone systems in deployment.

Designing drones that can work accurately

across environmental conditions is necessary.

• Lightweight design. In its simplest terms,

decreasing the weight of the drone platform

increases the time of flight on a given battery.

The lighter the drone system, the easier it is to

deploy. Reducing weight in a drone platform

also enables more payload budget, which can

then be dedicated to onboard application-

specific tools. All of these considerations greatly

enhance the productivity of the drone platform.

• High speed. The faster drones fly and perform,

the higher the productivity—but designers must

balance speed against safety considerations.

Drones can achieve maximum horizontal

velocities over 70 kph; two drones flying directly

toward one another represent a closure rate

of 40 m per second and create a significant

challenge to sense-and-avoid functions.

• Intelligent operation. The most dangerous

times of drone operation are when it is close to

the ground during takeoff or landing, because

of the reduced error margin with the ground.

Operator error here can result in damage or loss

of the drone, drastically impacting productivity

and introducing many safety concerns. A key

factor is the speed as the drone approaches

the ground, so the ability to accurately detect

distances at the centimeter level is critical.

Another must-have feature is for the drone to be

able to detect the type of surface it intends to

safely land on, and whether it is wet or dry.

• Object detection and avoidance. Drone

platforms operate in environments with physical

obstacles. The ability to detect obstacles and

then take avoidance measures reduces the

potential of damage or loss of the drone and

damage to surroundings. Combined with high

speeds these measures must be taken quickly

with calculations in real-time.

Using mmWave sensors to enhance 3 May 2017drone safety and productivity

The application of millimeter wave (mmWave) silicon sensors

TI’s millimeter wave (mmWave) sensor, integrates

RF processing, calibration, high-speed ADC,

microcontroller (MCU), digital signal processing

(DSP) and memory on a single complementary

metal-oxide semiconductor (CMOS) monolithic

chip, which accurately reports range, velocity and

angle between the sensors and objects around the

drone. The resulting level of integration allows for

a scalable family of devices that address different

processing outputs, shown in Figure 1, which

can be used in a variety of system architectures to

output real-time intelligence to the drone-control

system about the surrounding environment and

potential obstacles.

This real-time intelligence enables designers to

make drone systems that can operate with high

productivity in real-world deployments and meet

the design challenges listed above. Deploying

multiple sensors and sensing modalities adds safety

redundancy, and mmWave sensing has unique

attributes to address these challenges.

Operation in all conditions

Radar technology is not new; British physicist Sir

Robert Watson–Watt created the first practical

system in 1935[3]. Similar to microwave radar

used in modern aviation, mmWave devices in the

30 GHz–300 GHz spectrum operate in all weather

conditions. Looking at the frequency spectrum in

Figure 2, mmWave sensors operate in the spectrum

between photonics and microwaves.

Operating in this spectrum makes mmWave sensors

interesting because they:

• Can penetrate materials and see through

plastic, drywall and clothing

TI Confidential – NDA Restrictions

Figure 1

1

RF Front-End

ADC ADC Data

IWR1443

IWR1642

Pre- Processing

(Interference Mitigation)

1st Dim FFT

(Range)

2nd Dim FFT

(Velocity)

3rd Dim FFT (Angle Arrival)

Detection

Point Cloud [Range, Velocity,

Angle]

Clustering Tracking Object Classification Objects

Figure 1. mmWave signal processing

Figure 2. Frequency spectrum

Courtesy of Electronic Design, http://www.electronicdesign.com

Using mmWave sensors to enhance 4 May 2017drone safety and productivity

• Can see through sleet, rain, snow, fog and other

hazardous conditions

• Feature highly directional compact-beam

steering with 1-degree angular accuracy

• Offer small wavelengths with submillimeter

range accuracy

• Employ standard optical techniques for focus

and steering

• Offer large absolute bandwidths with the ability

to distinguish between two objects

Lightweight design

Using CMOS silicon technology to integrate most

required functions on a monolithic die just the TI

millimeter wave device, power management and

boot prom integrated circuits plus a PCB board

antenna are all that are required to implement the

full sensor. Figure 3 shows the size of an existing

module from D3 Engineering using the TI mmWave

sensor in a 3 transmit and 4 receiver antenna

configuration. Table 1 compares the mmWave

module dimensions and weight with a state-of-

the-art LIDAR rangefinder showing an almost 3×

reduction in size at less than half the weight.

The compactness of the solution allows easy

placement of the sensor behind plastic enclosures,

enabling a rugged design that adds minimal weight.

Contrast this with an optical or an infrared (IR)-

based solution that requires lens housings, tooling

and calibration during manufacturing test and a

lower assembly cost is suggested.

High speed

Drones can fly very fast, with maximum

specifications of 72 kph. Thus, any sensing

technology must be able to measure velocities at

these speeds. The Vmax, or maximum measurable

velocity (maximum relative velocity if both the sensor

and the object are moving) is related to the total

chirp period of the mmWave transmit signal, as

shown in Equation 1:

V = /(4T )max cl (1)

where Tc is the chirp duration/period total, l is the

distance traveled in one cycle and Vmax is in meter/sec.

From the data sheet of the IWR1443 device,

l = 3.9 mm for a start frequency of 76.5 GHz.

For a Vmax of 72 kph, the value of Tc = 48.75 µs.

Based on inverse proportionality, as long as Tc is

less than 48.75 µs, then a Vmax greater than 72 kph

is detectable. Calculating the Tc across a range of

Vmax values generates the curve shown in Figure 4.

Figure 3. Typical mmWave module design

38.10

34.04

2

17

.02

17

.02

.03

1.75

38

.10

5X7.452.03

mmWave ModuleLIDAR rangefinder

ModuleReduction

%

Module size 38 × 38 × 7.5 mm 20 × 48 × 40 mm 72%

Module weight 7.5 g 16 g Note 1 53%

Note 1–includes optics and housing

Table 1. Comparison of size and weight of D3 mmWave sensor module with LIDAR module

Figure 4. Maximum measurable velocity vs. chirp duration/period total

Using mmWave sensors to enhance 5 May 2017drone safety and productivity

Intelligent operation

To maximize drone productivity in real-world

applications, it is important to assist operators with

intelligence from the sensors on the drone. As we

discussed earlier, the most perilous time for a drone

is when it is in close proximity to the ground, such

as in landing scenarios. mmWave technology can

provide navigational information during landing

and assess the suitability of the landing surface.

The sensors are not susceptible to wind buffeting

or dust generated from the drone propellers when

close to the ground, as is the case with other

sensor technologies.

Using the IWR1443 mmWave evaluation module

(EVM) a 2-cm accuracy can be achieved with an

altitude range from the ground to 40 m, at speed

from hover to exceeding 25 cm per second.

Figure 5 shows a screen capture of a drone landing

demonstration, capturing the drone at a 19.73 cm

altitude and a speed of 22.55 cm per second from

the ground.

The accuracy of mmWave sensors also has the

potential to determine the type of landing surface

based on surface movements generated from

the drone propellers during hover with the drone

stationary right before final touchdown. Experiments

conducted at TI using the IWR1443 mmWave EVM

(under static conditions without the drone flying)

show that the ground was distinguishable from

water, based on the differences in reflectivity and

by measuring microvibrations from the material

surface, as shown in Figure 6. In case it does

detect water, the drone can abort its landing and

thus avoid damage or total loss of its cargo, or of

the drone itself.

Object detection and avoidance

At the core of an intelligent sense-and-avoid

operation is a drone’s ability to detect likely

obstructions that it will encounter in its line of flight,

which could result in total loss or damage to the

platform, negatively impacting productivity. Beyond

the ability of mmWave sensors to detect objects in

conditions without regard to lighting, smoke, dust or

fog, they are uniquely positioned to detect objects

that are difficult using other sensor technologies.

One such example is the detection of wires such as

power lines, telephone lines, aerial antennas or wire

fencing strung in the drone’s path. TI ran a series of

experiments in its anechoic chamber to evaluate the

detection of different types of wire. Figure 7 shows

the lab setup.Figure 5. Screen capture of a drone landing demonstration

Figure 6. Using mmWave to detect ground or water surfaces

Figure 7. Anechoic chamber test setup for wire-line testing

Using mmWave sensors to enhance 6 May 2017drone safety and productivity

The types of wire tested included an electrical

extension cord, a Category 5 Ethernet cable, a

line of nonmetallic (rubber) cabling, two 30-gauge

copper wires twisted together and a single

30-gauge copper wire shown in Figure 8.

The IWR1443 mmWave EVM antenna collected

measurements in the vertical and horizontal position

(with suffix flip) at a distance of 1 m. Table 2 shows

the test results. Compared to a vertical orientation,

the horizontal orientation achieved higher signal-

to-noise (SNR) figures. The higher SNR was due to

lower clutter in the test chamber in that orientation,

especially for the thin wire, which was a single non-

twisted strand.

We also observed a 6-dB gain by applying RX digital

beamforming (BF) using the four-reciever antenna

on the IWR1443 EVM.

Parameter ValueStart frequency 77 GHz

Frequency slope 33 MHz/µs

Sampling rate 10 MHz

ADC samples_3.3G 1000

CFAR SNR

Test no BF BF

ethernet_1m 21 24

ethernet_1m_flip 21 28

extend_1m 12 14

extend_1m_flip 21 33

rubber_1m 21 31

rubber_1m_flip 8 18

thin_1m –4 –6

thin_1m_flip 10 18

twist_1m 17 24

twist_1m_flip 14 23

Table 2. SNR analysis with the IWR1443 EVM

Besides the IWR1443 mmWave EVM, we used

an internal characterization board with a horn

antenna for the wire test. The horn antenna has

a much narrower antenna beam to reduce clutter

significantly. During this test, we placed the wire

at 4.5 m and measured the Constant False Alarm

Rate (CFAR) SNR for each type of wire. Equation 2

calculates the maximum detectable range of the

wire as:

SNR ~ 1/R3 (2)

Figure 9 highlights the expected maximum

detection range for each wire type if the CFAR

detection threshold is 15 dB.

The analysis showed that TI’s mmWave sensors can

detect all tested wire types, with the very thin and

most challenging 30-gauge single-strand wire from

an extrapolated distance of 9 m, up to 21 m for the

largest gauge tested, in our case an extension cord.

The IWR1443 mmWave EVM—with multiple input/

multiple output (MIMO) enabled—took outdoor data

of actual overhead power lines. As Figure 10 on

the following page shows, the EVM can robustly

detect power lines from 25 m up to 38 m for the

smaller-diameter power line barely visible in the

photo (fourth wire from the red arrow). This actual

data highlights the aggressive nature of the lab wire-

line sample types selected and the conservative

extrapolation results.

Figure 8. 30-gauge copper wire

0 5 10 15 20 25

range(m)

0

5

10

15

20

25

30

35

SN

R (

dB)

EthernetExtensionrubberthinWiretwist

X: 9.45Y: 15.33

X: 12.15Y: 15.06

X: 13.05Y: 15.13

X: 15.3Y: 15.06

X: 20.7Y: 15.12

Figure 9. Detection range extrapolation by wire type

Using mmWave sensors to enhance 7 May 2017drone safety and productivity

As a side observation, the tree foliage seen in the

test pictures also showed up in the sensors’ field of

view. TI’s mmWave sensors can detect trees with

leaves regardless of moving foliage or shadowy

conditions, and without excessive computational

requirements that consume power.

mmWave sensors can robustly detect wire lines,

from very fine single-strand wire to standard

overhead power lines at distances approaching

40 m.

We selected this use case for analysis because of

its sensing difficulty and calculation complexity with

other sensing technologies. It also illustrates a real-

world deployment of industrial drones, which must

detect and then navigate around objects to produce

the expected productivity and safety levels.

Summary

TI’s integration of mmWave sensors on a monolithic

piece of silicon is enabling designers to create

drone platforms that disrupt industries and increase

economic productivity. These sensors offer superior

performance to operate in all conditions, at high

rates of speed, with landing and takeoff intelligence

and the ability to detect objects such as power

lines. Realizing this performance in sensors that

are small, lightweight, rugged and easily encased

in drone plastics has brought TI’s mmWave sensor

devices to the forefront of sensing solutions for

drone manufacturers.

Texas Instruments has introduced a complete

development environment for engineers working on

industrial mmWave sensor products that includes:

• Hardware EVMs for IWR1443 and IWR1642

mmWave devices.

• An mmWave software development kit (SDK)

that includes a real-time operating system

(RTOS), drivers, signal-processing libraries, the

mmWave application programming interface

(API) and security (available separately).

• mmWave Studio offline tools for algorithm

development and analysis that include data

capture, a visualizer and a system estimator.

To learn more about the mmWave sensor portfolio,

tools and software, see www.ti.com/mmwave.

Figure 10. Outdoor power-line detection with the IWR1443 EVM

References

1. Markets and Markets. “Drones Market by Type

(Fixed Wing, VTOL, STUAS, MALE, HALE,

UCAS ), Payload (Up to 25 Kg, Up to 150 Kg,

Up to 600 Kg, Above 600 Kg), Application

(Precision Agriculture, Retail, Media &

Entertainment), Component, and Geography

– Global Forecast to 2022.” October 2016.

2. PwC. “Clarity from above. PwC global report

on the commercial applications of drone

technology.” May 2016.

3. Invention of the Radar – Obstetric

ultrasound.

4. RobotShop.com. LIDAR-Lite 3 Laser

Rangefinder.

SPYY001© 2017 Texas Instruments Incorporated

Important Notice: The products and services of Texas Instruments Incorporated and its subsidiaries described herein are sold subject to TI’s standard terms and conditions of sale. Customers are advised to obtain the most current and complete information about TI products and services before placing orders. TI assumes no liability for applications assistance, customer’s applications or product designs, software performance, or infringement of patents. The publication of information regarding any other company’s products or services does not constitute TI’s approval, warranty or endorsement thereof.

The platform bar is a trademark of Texas Instruments. All other trademarks are the property of their respective owners.

IMPORTANT NOTICE FOR TI DESIGN INFORMATION AND RESOURCES

Texas Instruments Incorporated (‘TI”) technical, application or other design advice, services or information, including, but not limited to,reference designs and materials relating to evaluation modules, (collectively, “TI Resources”) are intended to assist designers who aredeveloping applications that incorporate TI products; by downloading, accessing or using any particular TI Resource in any way, you(individually or, if you are acting on behalf of a company, your company) agree to use it solely for this purpose and subject to the terms ofthis Notice.TI’s provision of TI Resources does not expand or otherwise alter TI’s applicable published warranties or warranty disclaimers for TIproducts, and no additional obligations or liabilities arise from TI providing such TI Resources. TI reserves the right to make corrections,enhancements, improvements and other changes to its TI Resources.You understand and agree that you remain responsible for using your independent analysis, evaluation and judgment in designing yourapplications and that you have full and exclusive responsibility to assure the safety of your applications and compliance of your applications(and of all TI products used in or for your applications) with all applicable regulations, laws and other applicable requirements. Yourepresent that, with respect to your applications, you have all the necessary expertise to create and implement safeguards that (1)anticipate dangerous consequences of failures, (2) monitor failures and their consequences, and (3) lessen the likelihood of failures thatmight cause harm and take appropriate actions. You agree that prior to using or distributing any applications that include TI products, youwill thoroughly test such applications and the functionality of such TI products as used in such applications. TI has not conducted anytesting other than that specifically described in the published documentation for a particular TI Resource.You are authorized to use, copy and modify any individual TI Resource only in connection with the development of applications that includethe TI product(s) identified in such TI Resource. NO OTHER LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE TOANY OTHER TI INTELLECTUAL PROPERTY RIGHT, AND NO LICENSE TO ANY TECHNOLOGY OR INTELLECTUAL PROPERTYRIGHT OF TI OR ANY THIRD PARTY IS GRANTED HEREIN, including but not limited to any patent right, copyright, mask work right, orother intellectual property right relating to any combination, machine, or process in which TI products or services are used. Informationregarding or referencing third-party products or services does not constitute a license to use such products or services, or a warranty orendorsement thereof. Use of TI Resources may require a license from a third party under the patents or other intellectual property of thethird party, or a license from TI under the patents or other intellectual property of TI.TI RESOURCES ARE PROVIDED “AS IS” AND WITH ALL FAULTS. TI DISCLAIMS ALL OTHER WARRANTIES ORREPRESENTATIONS, EXPRESS OR IMPLIED, REGARDING TI RESOURCES OR USE THEREOF, INCLUDING BUT NOT LIMITED TOACCURACY OR COMPLETENESS, TITLE, ANY EPIDEMIC FAILURE WARRANTY AND ANY IMPLIED WARRANTIES OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT OF ANY THIRD PARTY INTELLECTUALPROPERTY RIGHTS.TI SHALL NOT BE LIABLE FOR AND SHALL NOT DEFEND OR INDEMNIFY YOU AGAINST ANY CLAIM, INCLUDING BUT NOTLIMITED TO ANY INFRINGEMENT CLAIM THAT RELATES TO OR IS BASED ON ANY COMBINATION OF PRODUCTS EVEN IFDESCRIBED IN TI RESOURCES OR OTHERWISE. IN NO EVENT SHALL TI BE LIABLE FOR ANY ACTUAL, DIRECT, SPECIAL,COLLATERAL, INDIRECT, PUNITIVE, INCIDENTAL, CONSEQUENTIAL OR EXEMPLARY DAMAGES IN CONNECTION WITH ORARISING OUT OF TI RESOURCES OR USE THEREOF, AND REGARDLESS OF WHETHER TI HAS BEEN ADVISED OF THEPOSSIBILITY OF SUCH DAMAGES.You agree to fully indemnify TI and its representatives against any damages, costs, losses, and/or liabilities arising out of your non-compliance with the terms and provisions of this Notice.This Notice applies to TI Resources. Additional terms apply to the use and purchase of certain types of materials, TI products and services.These include; without limitation, TI’s standard terms for semiconductor products http://www.ti.com/sc/docs/stdterms.htm), evaluationmodules, and samples (http://www.ti.com/sc/docs/sampterms.htm).

Mailing Address: Texas Instruments, Post Office Box 655303, Dallas, Texas 75265Copyright © 2017, Texas Instruments Incorporated