Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer...
Transcript of Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer...
![Page 1: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/1.jpg)
AshishTondwalkarAugust3,2017
Daedalus:Summer2017InternshipProject
FinalReport
![Page 2: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/2.jpg)
WhoamI?
Senior - TJHSST Class of 2018
Hobbyist, Chess player, Game Enthusiast
Snr. Research Plan: Locate an object's position using Sound
Interning @ USC-ISI under Dr. Andrew Schmidt to research Microphone Array Beamforming for Sound Source Localization - Project DAEDALUS
![Page 3: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/3.jpg)
Daedalus
Overview: Research methods to improve accuracy of localization Possible applications:
• Military: • Locating long range sharp shooters • Inform squads of firefight locations
Possible areas to research accuracy:
• Microphone Distance • Array Shape • Microphone Count
![Page 4: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/4.jpg)
Setup
4
• Pynq - Z1 Board • Jupyter to run python • OLED Display to output status & data • Microphones in triangle configuration • Servo placed in the middle to provide direction • Speaker to calibrate microphone recordings
PYNQ
OLED MIC0
MIC1
MIC2
Speaker
Servo
![Page 5: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/5.jpg)
ProjectPlan
5
1. Time Sound thresholds 2. Record Small intervals of sound from microphones 3. Convert data to manipulatable format 4. Filter Noise by Butterworth 5. Calibrate Recordings using speaker sound 6. Cross Correlation for Time Phase Delay 7. Direction: Plug Time Differences into Matrix Equation &
retrieve unit vector components 8. Distance: Plug Time Differences into Algebraic Equation &
retrieve estimated circle radius 9. Output Vector Data through display and servos
![Page 6: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/6.jpg)
DirecEon
6
Given:
• Position Vectors of Microphones • Sound time delay for each Microphone
Unknown:
• Direction of Sound
![Page 7: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/7.jpg)
DirecEon
7
![Page 8: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/8.jpg)
Distance
8
Curviness of arc estimated by registering time delay of microphone input
![Page 9: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/9.jpg)
ImplementaEon
9
![Page 10: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/10.jpg)
ImplementaEon
10
Issues:
• Eq Ax = b requires A to be square and Invertible • Cannot Implement more than 1+D Microphones • Solving using SciPy augmented matrix yields incorrect answer; does not
like floats
On average solution requires less than 1 ms to calculate
![Page 11: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/11.jpg)
SimulaEngSoluEon
11
![Page 12: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/12.jpg)
ReconfiguringBoard
12
• Change System Block Design for more microphones, removing unnecessary blocks, reassigning port inputs
• Made mistakes in top level source file -> Vivado taking 30 minutes every time to tell me I made a mistake…
• Messed up address assignment which forced me to redownload original source files and redo necessary changes
Issues: • Mic Input is very sharp; requires load sounds
- IOB issues? - Mics are bad? - Pull Down in constraints file? - Misinterpretation of Design?
![Page 13: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/13.jpg)
ReformaMng&BuOerworth
• PDM->PCM• ~30secondsper1secondclip
• Bu>erworthcentersaudio• ~10secondsper1secondclip
• ~2Minutestoreformat3recordingsof1second
13
![Page 14: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/14.jpg)
AlternaEves
• Using raw data from audio buffer - 192000 Hz sampling rate!
• Manipulate FFT Array to filter audio • iFFT and continue to GCC
14
RAW DATA
FFT
iFFT
![Page 15: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/15.jpg)
ParallelRecording
• Did not use hardware to record in sync • Python Threading to record in parallel; not in sync Calibration Solution: • Record speaker in center of mic array initially, then make sound • Realign recording using GCC of speaker recording segment • GCC modified recordings
15
Speaker initially played to detect offset of mic recordings
Segment of Recording to detect target’s sound
![Page 16: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/16.jpg)
Threading
16
![Page 17: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/17.jpg)
SpeakerPlacement
• Microphones located at 3 vertices of a square • Speaker Placed in center of square • Speaker Pointed upward for even sound propagation
17
![Page 18: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/18.jpg)
HardwareRestricEon• Microphones not very sensitive, cannot pickup conversational
speech
• Microphones not omnidirectional
18
NOT ALIGNED! Speaker playing twice
Calibration
Testing Calibration without all microphones pointing directly at speaker
![Page 19: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/19.jpg)
HardwareRestricEon
• Audio Segments of speaker played twice are better aligned when microphones are pointed toward the speaker
• MicrophonesnotomnidirecEonal¬verysensiEvethroughcausesinaccuraciesinposiEoncalculaEons
19
Calibration
![Page 20: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/20.jpg)
CalibraEon
20
GCC Time Difference
Shifting Recording Arrays
![Page 21: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/21.jpg)
ServoOutput
• PWM signal • Duty Cycle Determines Position • Smaller Period -> larger range for duty cycle -> better
precision • Period of 2 ms
• Angle: [0, π/2] ~ Duty: (21%, 55%)
21
![Page 22: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/22.jpg)
OLEDOutput
• Status of the Program • Displays which Phase program is in • Target Position Output
22
![Page 23: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/23.jpg)
Conclusion
23
Project Conclusion: • Proof of concept completed • Demonstration of Project available • Able to indicate direction with fair accuracy • Final Program ~ 400 Lines of Python • Total Notebook Code ~ 1800 Lines of Python
Areas for further improvement: • Implementing simultaneous recording through hardware • Higher quality omnidirectional microphones • Find a method to improve distance estimation • Research accuracy improvement
![Page 24: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/24.jpg)
Conclusion
24
Some of what I have gained from the internship: • Learned Python and Jupyter • Major experience in using an FPGA • Far better understanding of Verilog and Vivado • Beginner level research experience • Interest in pursuing Research over Industry • Had Fun
![Page 25: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/25.jpg)
SeniorResearch
• Larger budget will allow for better microphones • More time will allow me to implement hardware enabled
recording
Build Up from Internship Project • Sound bar mimicry
• Time-delay Speaker Beam forming to produce stereo sound at target location
25
![Page 26: Daedalus: Summer 2017 Internship Project Final Report Daedalus Final Presentation.pdf · Summer 2017 Internship Project Final Report Who am I? Senior - TJHSST Class of 2018 Hobbyist,](https://reader035.fdocuments.us/reader035/viewer/2022062403/6044e57276dade0f767eb39e/html5/thumbnails/26.jpg)
Sources
26
W. Jiang, Z. Cai, M. Luo, and Z. Yu. A Simple Microphone Array For Source Direction and Distance Estimation. J. Valin, F. Michaud, J. Rouat, and D. Letourneau. Robust Sound Source Localization Using a Microphone Array on a Mobile Robot. N. Kwok, J. Buchholz, G. Fang, and J. Gal. Sound Source Localization: Microphone Array Design and Evolutionary Estimation. M. Omologo, and P. Svaizer. Use of the Crosspower-Spectrum Phase in Acoustic Event Location. SpecialThanksto:Dr.SchmidtOtherRCGMembers