Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD...
-
Upload
sheila-paul -
Category
Documents
-
view
221 -
download
0
Transcript of Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD...
![Page 1: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/1.jpg)
The University of Texas at Brownsville and Texas Southmost College
Locate Potential Support Vectors for FasterSequential Minimal Optimization
Hansheng Lei, PhDAssistant Professor
Computer and Information Sciences Department
![Page 2: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/2.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
Outline
• Background and Overview• Fisher Discriminant Analysis (FDA)• SVM vs. FDA• Combining FDA and SVM• Experimental Results• Computing Infrastructure at UT Brownsville• Application Projects
![Page 3: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/3.jpg)
The University of Texas at Brownsville and Texas Southmost College
Classification
How to classify this data?
w x + b=0
w x + b<0
w x + b>0
![Page 4: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/4.jpg)
The University of Texas at Brownsville and Texas Southmost College
Linear Classifiers
a
f
x y
f(x,w,b) = sign(w x + b)
How to classify this data?
![Page 5: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/5.jpg)
The University of Texas at Brownsville and Texas Southmost College
Linear Classifiers
How to classify this data?
a
f
x y
f(x,w,b) = sign(w x + b)
![Page 6: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/6.jpg)
The University of Texas at Brownsville and Texas Southmost College
Linear Classifiers
which is best?
a
f
x y
f(x,w,b) = sign(w x + b)
![Page 7: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/7.jpg)
Linear SVM
![Page 8: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/8.jpg)
The University of Texas at Brownsville and Texas Southmost College
Solving the Optimization Problem
Find w and b such thatΦ(w) =½ wTw is minimized; and for all {(xi ,yi)}: yi (wTxi + b) ≥ 1
Subject to
![Page 9: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/9.jpg)
The University of Texas at Brownsville and Texas Southmost College
Sequential Minimal Optimization (SMO) John C. Platt, 1998
The algorithm proceeds as follows:1. Find a Lagrange multiplier α1 that violates KKT conditions for the optimization problem. 2. Pick a second multiplier α2 and optimize the pair (α1,α2).3. Repeat steps 1 and 2 until convergence.
Heuristics are used to choose the pair of multipliers so as to accelerate the rate of convergence.
![Page 10: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/10.jpg)
The University of Texas at Brownsville and Texas Southmost College
SVM vs. Fisher Discriminant Analysis
1. Similar Format:
![Page 11: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/11.jpg)
The University of Texas at Brownsville and Texas Southmost College
SVM vs. Fisher Discriminant Analysis
2. Similar Projection:
![Page 12: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/12.jpg)
The University of Texas at Brownsville and Texas Southmost College
SVM vs. Fisher Discriminant Analysis
2. Similar Projection:
![Page 13: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/13.jpg)
The University of Texas at Brownsville and Texas Southmost College
Distribution of Support Vectors (SV)
![Page 14: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/14.jpg)
The University of Texas at Brownsville and Texas Southmost College
F-SMO = FDA+SMO
![Page 15: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/15.jpg)
The University of Texas at Brownsville and Texas Southmost College
Experimental Results
![Page 16: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/16.jpg)
The University of Texas at Brownsville and Texas Southmost College
Experimental Results
![Page 17: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/17.jpg)
The University of Texas at Brownsville and Texas Southmost College
Experimental Results
![Page 18: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/18.jpg)
The University of Texas at Brownsville and Texas Southmost College
Experimental Results
412 827 1587 3107 6260 1180005
101520253035404550
F-SMO, libsvm and SMO on Gaussain Kernel
SMO/GaussianF-SMO/Gaussianlibsvm/Gaussian
Number of Points
Tim
e (s
econ
d)
412 827 1587 3107 6260 118000
500
1000
1500
2000
2500
F-SMO, libsvm and SMO on Linear Kernel
SMO/LinearF-SMO/Linearlibsvm/Linear
Number of Points
Tim
e (s
econ
d)
![Page 19: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/19.jpg)
The University of Texas at Brownsville and Texas Southmost College
Experimental Results
412 827 1587 3107 6260 118000
2
4
6
8
10
12
14
F-SMO vs libsvm on Gaussian Kernel
F-SMO/Gaussianlibsvm/Gaussian
Number of Points
Tim
e (s
enco
nd)
412 827 1587 3107 6260 1180005
1015202530354045
F-SMO vs libsvm on Linear Kernel
F-SMO/Linearlibsvm/Linear
Number of Points
Tim
e (s
econ
d)
![Page 20: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/20.jpg)
The University of Texas at Brownsville and Texas Southmost College
Computing Infrastructure
• Graphics Processing Unit (GPU)• Cluster• Field-programmable gate array
(FPGA)• GPU Visualization• Advanced CM Flex Lab
![Page 21: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/21.jpg)
The University of Texas at Brownsville and Texas Southmost College
FUTURO cluster
• IBM® iDataPlex• 320 Cores @ 2.4Ghz• 216 TB Storage• QDR Infiniband @ 40Gbps• 40 Intel®XeonE5540 nodes• 192GB RAM per node max• 24 TB RAID per node max• NSF MRI funded
![Page 22: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/22.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
Futuro Architecture Design
![Page 23: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/23.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
FUTURO
![Page 24: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/24.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
FUTURO Gallery
![Page 25: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/25.jpg)
The University of Texas at Brownsville and Texas Southmost College
GPU Server
• AMAX® ServMax PSC-2n• 940 GPU Cores @ 1.3Ghz• 12 CPU Cores @ 2.8 Ghz• 4 teraflops max• 80 GB memory max• 4 Nvidia®Tesla nodes• 2 Intel® Xeon EP 5600 • NSF MRI funded
![Page 26: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/26.jpg)
The University of Texas at Brownsville and Texas Southmost College
FPGA Computing
• 1.2M logic cells• 80K system gates• 1.1M flip flops• 1.7K 18x18Multipliers• 532K Slices• 16 Xilinx®Spartan FPGAs• Impluse C supported• NSF LSAMP funded
![Page 27: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/27.jpg)
The University of Texas at Brownsville and Texas Southmost College
GPU Visualization• Dual Nvidia®QuadroPlex • 960 Nvidia® CUDA cores• 3.73 Teraflops • 33.3 Mega Pixels• 7680x4320 resolution• 16 GB Frame Buffer• 3D Stereo • US ED CCRAA funded
![Page 28: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/28.jpg)
The University of Texas at Brownsville and Texas Southmost College
Computational Science Flex Lab
• 32 SUN Ultra nodes • Intel® Q9650 @ 3.0 Ghz• 128 CPU Cores• 1024 CUDA Cores• 320GB RAM • 8.8TB Storage• US ED CCRAA funded
![Page 29: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/29.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
Enabled Projects
1. Tracking LIGO Detector Noise for Gravitational Wave Detection (NSF)
2. Genetic Data Analysis in Complex Human Diseases (University of Texas Health Science Center)
3. Dynamical Systems and Stellar Populations(NASA) 4. Collaborative Filtering using Multispectral Information(*) 5. Visualization of High-dimensional Data (NSF pending) 6. Practical Algorithms for the Subgraph Isomorphism Problem
![Page 30: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/30.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
• Tracking LIGO Detector Noise for Gravitational Wave Detection (PI: Lei, Tang, Mukherjee, Mohanty, co-PI: Iglesias)
Computing infrastructure and distributed KDD research.
Subproject 1– Parallel and Distributed ClusteringSubproject 2 – Parallel and Distributed ClassificationSubproject 3: Parallel and Distributed Rule Discovery
Distributed KDD
Futuro
Infrastructure
Noise Reduction
Clus
terin
g
Clas
sific
atio
n
Representation
Indexing
Rule Discovery
Dist
ribut
ed
Clus
terin
g
Dist
ribut
edCl
assif
icat
ion
InteractiveExploration
Parallel Rule Discovery
Grid Network
![Page 31: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/31.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
• Genetic Data Analysis in Complex Human Diseases (PI: Figueroa)
Genetic data analysis.
![Page 32: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/32.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
• Visualization of High-dimensional Data (PI: Quweider , co-PI: Mukherjee, Mohanty)
Visualization Framework.
...
Sensor 1 Sensor NSensor 2
Preprocessing Feature Selection /
Extraction
Data Clustering
Data Streams Data Streams
Data Reduction
Da
tabas
es/S
torage/
Retriev
al
...
Visualization
Data Mining
Hu
man
C
om
pute
r Interactio
n
![Page 33: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/33.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
Application Projects
• Automated optical inspection (AOI)
• Special Sound Detection,
![Page 34: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/34.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
Automated Optical Inspection
![Page 35: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/35.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
AOI components
• Computer vision software• Machine vision hardware for data acquisition,
e.g.. CCD camera and optical lens, or X-ray, • Auto control system• Illumination system
Optimal AOI, Viking Test Ltd
![Page 36: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/36.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
Special Sound Detection
Help !!
Up to 100 ft distance
Shout sound
Alarm signal
Communication
![Page 37: Locate Potential Support Vectors for Faster Sequential Minimal Optimization Hansheng Lei, PhD Assistant Professor Computer and Information Sciences Department.](https://reader038.fdocuments.us/reader038/viewer/2022110401/56649e155503460f94afe903/html5/thumbnails/37.jpg)
The University of Texas at Brownsville and Texas Southmost College
The University of Texas at Brownsville and Texas Southmost College
The EndWelcome to Visit UTB