Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on...
-
Upload
stella-lawson -
Category
Documents
-
view
215 -
download
1
Transcript of Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on...
Peter RichtarikSchool of Mathematics
Optimization with Big Data * in a billion dimensional space on a foggy day
Extreme* Mountain Climbing=
BIG DATA
• digital images & videos• transaction records• government records• health records• defence• internet activity (social media, wikipedia, ...)• scientific measurements (physics, climate models, ...)
BIG Volume BIG Velocity BIG Variety
Sources
BIG Volume BIG Velocity BIG Variety
Western General Hospital(Creutzfeldt-Jakob Disease)
Arup (Truss Topology Design)
Ministry of Defence dstl lab(Algorithms for Data Simplicity)Royal Observatory
(Optimal Planet Growth)
GOD’S Algorithm = Teleportation
If you are not a God...
x0x1
x2 x3
Optimization as Lock Breaking
Setup: Combination maximizing F opens the lock
x = (x1, x2, x3, x4) F(x) = F(x1, x2, x3, x4)
A number representing the
“quality” of a combination
Optimization Problem: Find combination maximizing F
Optimization Algorithm
How to Open a Lock with Billion Interconnected Dials?
F : Rn R# variables/dials = n = 109
x1
x2
Assumption:F = F1 + F2 + ... + Fn
-----------------------Fj depends on the neighbours of xj only
x3
x4
Example:F1 depends on x1, x2, x3 and x4
F2 depends on x1 and x2, ...
xn
Optimization Methods
Computing Architectures• Multicore CPUs• GP GPU accelerators• Clusters / Clouds
• Effectivity• Efficiency• Scalability• Parallelism• Distribution• Asynchronicity• Randomization
Optimization Methods for Big Data
• Randomized Coordinate Descent– P. R. and M. Takac: Parallel coordinate descent
methods for big data optimization, ArXiv:1212.0873 [can solve a problem with 1 billion variables in 2 hours using 24
processors]• Stochastic (Sub) Gradient Descent
– P. R. and M. Takac: Randomized lock-free methods for minimizing partially separable convex functions
[can be applied to optimize an unknown function]• Both of the above
M. Takac, A. Bijral, P. R. and N. Srebro: Mini-batch primal and dual methods for SVMs, ArXiv:1302.xxxx
Theory vs Reality
start
settle for this
holy grail
Parallel Coordinate Descent
TOOLSProbability
Machine LearningMatrix Theory
HPC