Team 2: Mind Readers
description
Transcript of Team 2: Mind Readers
![Page 1: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/1.jpg)
Team 2: Mind Readers
Krishna Jharjaria TJ Strzelecki Rick Schuman Matt Waldersen
![Page 2: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/2.jpg)
Project Overview The Mind Reader is a mobile brain-computer interface. Computer applications will be presented to the user
through commercially available video glasses. An EOG and commercially available EEG will be
mounted inside of a common enclosure and will enable the user to navigate and select various applications.
A dsPIC microcontroller will be used to acquire the EOG and EEG signals, the EEG signals will be analyzed by an FPGA and a BeagleBoard XM will control the virtual reality environment as well as execute all of the computer applications
![Page 3: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/3.jpg)
Project-Specific Success Criteria
1) An ability to encode/decode data packets from a NeuroSky EEG.
2) An ability for a user to select applications based on signals from a NeuroSky EEG.
3) An ability for a user to navigate between different applications on a display using EOG signals.
4) An ability for the system to interactively train the user to effectively operate the device.
5) An ability to display a live video stream from an external camera module, and integrate applications into the video system.
![Page 4: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/4.jpg)
Block Diagram
![Page 5: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/5.jpg)
Component Selection RationaleMicrocontroller
Considerations Signal Processing abilities Digital Communication pins Optimized for C compiler Processing speed Resources and reference material
DSPIC33EP512MU810 Extensive DSP Library with built in FFT function 4-UART; 4-SPI; 2-I2C Optimized for C compiler ~53K of RAM Large online community
![Page 6: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/6.jpg)
EOG Op-Amp Selection Rationale (TLC2272/4 & TLV2211 )
Low Noise Dual Polarity Model Available in PSPICE Multiple Op-Amp Models Available in DIP and SMD
configurations With exception of the TLV2211
High Accuracy
![Page 7: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/7.jpg)
EOG Op-Amp Selection Criteria
![Page 8: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/8.jpg)
ADC Selection Rationale (ADS1210) High Resolution
19.5 Effective Resolution (Bits RMS) Differential input and 2.5V reference
output eliminates need for virtual ground
Integrated Programmable Gain Amplifier
Integrated Digital Filter Available in both DIP and SMD
packages
![Page 9: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/9.jpg)
Considerations Size (Logic Blocks) Number of I/O Pins Built in Functionality Power Consumption
DLP-HS-FPGA3 USB - FPGA Module 25,344 Logic Blocks (11,264 slices) 63 Available user I/O channels 32 Dedicated Multipliers, libraries for floating point arithmetic FPGA module contains: regulators, clock generator, SDRAM,
USB programmer, SPI Flash, compatible with Xilinx ISE WebPack
FPGA operates on 3.3V, sinks/sources approximately 24mA per I/O pin
Component Selection RationaleFPGA Module
![Page 10: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/10.jpg)
BeagleBoard-xM 1.0 GHz ARM Cortex-A8 Processor 512 MB RAM 8 GB microSD HDD ▪ libraries and main program
4 USB 2.0 Slots ▪ Mouse/Keyboard dev, Webcam
Expansion Header (Data Transfer)▪ o SPI▪ o UART▪ o I2C▪ o GPIO
Component Selection RationaleBeagleBoard xM
![Page 11: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/11.jpg)
Packaging Design
Modular design Maintains separation of digital circuitry
and sensitive analog EOG circuitry Enables the EOG circuitry to be placed
as close possible to signal electrodes Video glasses will be used in order to
ensure that the video display is directly in front of the field of view
Light Weight Single Board Computer
![Page 12: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/12.jpg)
Packaging DesignBeagleBoard-xM Size: 3.35” x 3.45” Weight: < 1 lb.
Logitech C310 HD Cam 1280x720p frame capture < 8 oz.
Vuzix Wrap 920 Twin 640x480p Display < 3 oz.
![Page 13: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/13.jpg)
Main Circuit BoardEOG inputs
Power SupplyMicrocontroller
Interface
FPGA and Beagle Board
![Page 14: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/14.jpg)
Power Supply
11.1V rechargeable Lithium ion Battery
5V convertorFor FPGA and Beagle Board
3.3 V convertorFor Microcontroller
2 Switch Mode regulators
![Page 15: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/15.jpg)
EOG inputs
Connected to EOGvia SPI
Connected to EOG via SPI
Logic Level convertorFrom 5V to 3.3V
Registered Jack (RJ11)
![Page 16: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/16.jpg)
Micro, in-system-programmer, EEG Dongle, Oscillator
Oscillator
In System Programmer
PCB mountReset
Through holeReset Connections
Switch debouncer
Pin 13 MCRL
Pin 15 PGECPin 16 PGED
EEG input via UART
![Page 17: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/17.jpg)
UNIVERSAL ASYNCHRONOUS RECEIVER TRANSMITTER (UART)
Baud rate = 115200 bps Specified by the NeuroSky MindWave
![Page 18: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/18.jpg)
EEG signals FFT EEG frequency
range 4-7hz
Array size 512
Output of the FFT resides in RAM
Inbuilt FFT API: FFTComplexIP() BitReverseComplex() SquareMagnitudeCplx()4-7hz
![Page 19: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/19.jpg)
FPGA Module, Logic Level Converter, Beagle Board
Beagle Board
FPGA Module
Logic Level ConvertorFrom 3.3V to 5V
Select button
Switch debouncer
4 bit EOG outputUp, down, left, right
2x5 bit EOG input
4 I/O for EOG3 SPI pins for EEG
![Page 20: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/20.jpg)
PCB LayoutDesign Considerations
Main Board Multiple Voltages needed Power Traces at least 40 mil, signal traces no smaller than
10 mil Isolate oscillator Physical location of connectors Decoupling Capacitors
EOG Board Sensitive analog circuitry Distance between electrodes and filtering/amplification Physical size of EOG PCB
![Page 21: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/21.jpg)
PCB Layout
![Page 22: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/22.jpg)
PCB Layout – Main BoardFGPA Module
![Page 23: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/23.jpg)
PCB Layout – Main BoardFGPA Module
![Page 24: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/24.jpg)
PCB Layout – Main BoarddsPIC Micro
![Page 25: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/25.jpg)
PCB Layout – Main BoarddsPIC Micro
![Page 26: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/26.jpg)
PCB Layout – Main BoardPower Supply
![Page 27: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/27.jpg)
PCB Layout – Main BoardPower Supply
![Page 28: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/28.jpg)
PCB Layout – Main BoardEOG/SBC Connection & LLT
![Page 29: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/29.jpg)
PCB Layout – Main BoardEOG/SBC Connection & LLT
![Page 30: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/30.jpg)
EOG Theory of Operation
Electrooculography measures the signal given off of corneo-retinal potentials in the eyes
An EOG circuit consists of an instrumentation amplifier, amplifiers, low pass filtering and high pass filtering
A high pass filter is used to eliminate a naturally occurring DC drift
A low pass filter is used to remove all other external noise
![Page 31: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/31.jpg)
Instrumentation AmplifierInstrumentation
Amplifier High Input
Impedance Includes both a
high pass filter to eliminate DC drift and a second order low pass filter
![Page 32: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/32.jpg)
Instrumentation Amplifier
![Page 33: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/33.jpg)
Sallen-Key 2nd Order LPFSallen-Key 2nd-Order Low Pass Filter Eliminates noise from external sources
![Page 34: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/34.jpg)
Sallen-Key 2nd Order LPF
![Page 35: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/35.jpg)
First Order LPF - AmplifierFirst Order Low Pass Filter – Amplifier Circuit Eliminates noise as well as amplifies the signal
![Page 36: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/36.jpg)
First Order LPF - Amplifier
![Page 37: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/37.jpg)
Linear Regulator
![Page 38: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/38.jpg)
Voltage Inverter
![Page 39: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/39.jpg)
Analog-to-Digital Converter
![Page 40: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/40.jpg)
Reference Ground & Input Jack
![Page 41: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/41.jpg)
![Page 42: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/42.jpg)
EOG Signal Conditioning
Voltage (V)
Time (s)
![Page 43: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/43.jpg)
EOG Signal Conditioning
Voltage (V)
Time (s)
![Page 44: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/44.jpg)
Normalizing EOG Signal
∆Voltage
Sample Number
![Page 45: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/45.jpg)
Software Design/Development StatusArtificial Neural Network
What is an Artificial Neural Network? Simplified mathematical model of the human brain Utilizes a network of “neurons” to model relationships
between inputs and outputs How does a ANN work?
ANN implement non-linear functions in each neuron This function sums weighted input values and passes them
through a non-linear function, usually a Sigmoid function This output then propagates to further network layers for
the process to be repeated Why use an FPGA implementation
Highly parallel algorithm, with transistor like output Need for quick, complex calculations
![Page 46: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/46.jpg)
Software Design/Development StatusArtificial Neural Network
![Page 47: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/47.jpg)
Software Design/Development StatusArtificial Neural Network
Basics of A Neuron Each neuron contains a number of synapses, one for each
previous layer neuron connected A synapses will take inputs and multiply it by a predetermined
specific weight The weighted values will then be accumulated and funneled into
a nonlinear activation function (Sigmoid Function)
Hardware Design Considerations Need multiple multipliers, one for each synapses (38 multipliers)▪ Need to condense the size of a neuron▪ Using control logic and multiple clock cycles we can lower the
required multipliers to a single multiplier per neuron (12 multipliers) Implementation of non-linear Sigmoid Function▪ Look-up Table
![Page 48: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/48.jpg)
Artificial Neural NetworkPreliminary Block Diagram
![Page 49: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/49.jpg)
Software Design/Development StatusArtificial Neural Network
Where we proceed from here Training and testing of preliminary ANN using
Matlab ANN toolbox▪ This will allow the weighted values to be hardcoded into
the FPGA Design and testing of single neuron in VHDL Development of a proven working topology
![Page 50: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/50.jpg)
Software Design/Development StatusHeads-Up Display
Tentative HUD Features
•Webcam Live Feed
•EEG Signal Data display
•Home Screen• 3 x 3 application layout• Concentration to select app
•Programs• Minimal key input• Closed apps return to Home
![Page 51: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/51.jpg)
Software DesignSingleboard Program Flow Diagram
![Page 52: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/52.jpg)
Project Completion Timeline
October 15-19 PCB Proof of Parts
October 22-26 Simple Software Suite Sample EOG circuit ADC microcontroller
communication October 29 – Nov 2
Micro to Single-board communication
EOG ANN finished in MATLAB
Nov 5 – Nov 9 Live Feed Visual Software
Suite EEG ANN prototyped in
MATLAB PCB assembled
Nov 12 – 16 EOG ANN finished on FPGA
Nov 19 – 23 Software Test Application User Training Application
Nov 26 – Nov 30 Troubleshoot
Dec 3 – Dec 7 Project Complete
![Page 53: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/53.jpg)
Questions?
![Page 54: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/54.jpg)
Reference (DSPIC33EP)
![Page 55: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/55.jpg)
Reference (PIC Programmer)
![Page 56: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/56.jpg)
Reference (RJ-12 Connection)
![Page 57: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/57.jpg)
Reference (Oscillator/Reset)
![Page 58: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/58.jpg)
Reference (DC Voltage Inv)
![Page 59: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/59.jpg)
Reference (ADS1210 ATD)
![Page 60: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/60.jpg)
Reference (TLC2272)
![Page 61: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/61.jpg)
Reference (TLC2272)
![Page 62: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/62.jpg)
Reference (TLV2211)
![Page 63: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/63.jpg)
Reference(DLP FPGA Module)
![Page 64: Team 2: Mind Readers](https://reader033.fdocuments.us/reader033/viewer/2022052219/5681692d550346895de07070/html5/thumbnails/64.jpg)
Reference (MAX764-DC Inverter)