AshishTondwalkarAugust3,2017

Daedalus:Summer2017InternshipProject

FinalReport

WhoamI?

Senior - TJHSST Class of 2018

Hobbyist, Chess player, Game Enthusiast

Snr. Research Plan: Locate an object's position using Sound

Interning @ USC-ISI under Dr. Andrew Schmidt to research Microphone Array Beamforming for Sound Source Localization - Project DAEDALUS

Daedalus

Overview: Research methods to improve accuracy of localization Possible applications:

•  Military: •  Locating long range sharp shooters •  Inform squads of firefight locations

Possible areas to research accuracy:

•  Microphone Distance •  Array Shape •  Microphone Count

Setup

4

•  Pynq - Z1 Board •  Jupyter to run python •  OLED Display to output status & data •  Microphones in triangle configuration •  Servo placed in the middle to provide direction •  Speaker to calibrate microphone recordings

PYNQ

OLED MIC0

MIC1

MIC2

Speaker

Servo

ProjectPlan

5

1.  Time Sound thresholds 2.  Record Small intervals of sound from microphones 3.  Convert data to manipulatable format 4.  Filter Noise by Butterworth 5.  Calibrate Recordings using speaker sound 6.  Cross Correlation for Time Phase Delay 7.  Direction: Plug Time Differences into Matrix Equation &

retrieve unit vector components 8.  Distance: Plug Time Differences into Algebraic Equation &

retrieve estimated circle radius 9.  Output Vector Data through display and servos

DirecEon

6

Given:

•  Position Vectors of Microphones •  Sound time delay for each Microphone

Unknown:

•  Direction of Sound

DirecEon

7

Distance

8

Curviness of arc estimated by registering time delay of microphone input

ImplementaEon

9

ImplementaEon

10

Issues:

•  Eq Ax = b requires A to be square and Invertible •  Cannot Implement more than 1+D Microphones •  Solving using SciPy augmented matrix yields incorrect answer; does not

like floats

On average solution requires less than 1 ms to calculate

SimulaEngSoluEon

11

ReconfiguringBoard

12

•  Change System Block Design for more microphones, removing unnecessary blocks, reassigning port inputs

•  Made mistakes in top level source file -> Vivado taking 30 minutes every time to tell me I made a mistake…

•  Messed up address assignment which forced me to redownload original source files and redo necessary changes

Issues: •  Mic Input is very sharp; requires load sounds

- IOB issues? - Mics are bad? - Pull Down in constraints file? - Misinterpretation of Design?

ReformaMng&BuOerworth

•  PDM->PCM•  ~30secondsper1secondclip

•  Bu>erworthcentersaudio•  ~10secondsper1secondclip

•  ~2Minutestoreformat3recordingsof1second

13

AlternaEves

•  Using raw data from audio buffer - 192000 Hz sampling rate!

•  Manipulate FFT Array to filter audio •  iFFT and continue to GCC

14

RAW DATA

FFT

iFFT

ParallelRecording

•  Did not use hardware to record in sync •  Python Threading to record in parallel; not in sync Calibration Solution: •  Record speaker in center of mic array initially, then make sound •  Realign recording using GCC of speaker recording segment •  GCC modified recordings

15

Speaker initially played to detect offset of mic recordings

Segment of Recording to detect target’s sound

Threading

16

SpeakerPlacement

•  Microphones located at 3 vertices of a square •  Speaker Placed in center of square •  Speaker Pointed upward for even sound propagation

17

HardwareRestricEon•  Microphones not very sensitive, cannot pickup conversational

speech

•  Microphones not omnidirectional

18

NOT ALIGNED! Speaker playing twice

Calibration

Testing Calibration without all microphones pointing directly at speaker

HardwareRestricEon

•  Audio Segments of speaker played twice are better aligned when microphones are pointed toward the speaker

•  MicrophonesnotomnidirecEonal&notverysensiEvethroughcausesinaccuraciesinposiEoncalculaEons

19

Calibration

CalibraEon

20

GCC Time Difference

Shifting Recording Arrays

ServoOutput

•  PWM signal •  Duty Cycle Determines Position •  Smaller Period -> larger range for duty cycle -> better

precision •  Period of 2 ms

•  Angle: [0, π/2] ~ Duty: (21%, 55%)

21

OLEDOutput

•  Status of the Program •  Displays which Phase program is in •  Target Position Output

22

Conclusion

23

Project Conclusion: •  Proof of concept completed •  Demonstration of Project available •  Able to indicate direction with fair accuracy •  Final Program ~ 400 Lines of Python •  Total Notebook Code ~ 1800 Lines of Python

Areas for further improvement: •  Implementing simultaneous recording through hardware •  Higher quality omnidirectional microphones •  Find a method to improve distance estimation •  Research accuracy improvement

Conclusion

24

Some of what I have gained from the internship: •  Learned Python and Jupyter •  Major experience in using an FPGA •  Far better understanding of Verilog and Vivado •  Beginner level research experience •  Interest in pursuing Research over Industry •  Had Fun

SeniorResearch

•  Larger budget will allow for better microphones •  More time will allow me to implement hardware enabled

recording

Build Up from Internship Project •  Sound bar mimicry

•  Time-delay Speaker Beam forming to produce stereo sound at target location

25

Sources

26

W. Jiang, Z. Cai, M. Luo, and Z. Yu. A Simple Microphone Array For Source Direction and Distance Estimation. J. Valin, F. Michaud, J. Rouat, and D. Letourneau. Robust Sound Source Localization Using a Microphone Array on a Mobile Robot. N. Kwok, J. Buchholz, G. Fang, and J. Gal. Sound Source Localization: Microphone Array Design and Evolutionary Estimation. M. Omologo, and P. Svaizer. Use of the Crosspower-Spectrum Phase in Acoustic Event Location. SpecialThanksto:Dr.SchmidtOtherRCGMembers