Ugo Montanari On the optimal approximation of descrete functions with low- dimentional tables.
-
date post
19-Dec-2015 -
Category
Documents
-
view
222 -
download
1
Transcript of Ugo Montanari On the optimal approximation of descrete functions with low- dimentional tables.
Ugo Montanari
On the optimal approximation of descrete functions with low-
dimentional tables.
Overview
Introduction Approximation with a sum of low-dimensional
functions Optomal approximation with a given interaction
graph Optimal approximation with a fixed amount of
memory
Introduction
Problem of storing large high-dimentional arrays is often critical (dynamic programming optimization techniques, belief propagation etc.)
Montanari proposes methos of optimal approximation (in the least square sence) of the given function with a sum of lower-dimentional functions.
Advantages
The decoding process is very simple (a fixed number of summations)
The compression ratio often is high, mean error can be small, if the interaction between separated variables is limited
The approximation function has a form of a sum of terms and is therefore suitable for the dynamic programming optimization
Approximation with a sum of low-dimentional functions
F – function of n discrete variables (with same domains)
Lettice
Approximation with a sum of low-dimentional functions
Average projection of the function:
Proper function:
A function g(Xi) such that its average projections on all
the subsets of Xi are identically zero will be called a
proper function of Xi.
Theorem 2.1
The set Si of all the proper functions of X
i is a vector space and
is called the proper space of Xi.
Theorem:
The proper space Si of all the elements X
i of lattice L are mutually
orthogonal.
Proof:
Characteristic function B
B:L -> 0,1 Monotonocity constraint:
The meaning of the characteristic function B is to specify the form of an approximate sum of terms for function F
Example of characteristic function
We want to approximate function F(x1,x
2,x
3) with
a sum of the form F = f1(x
1,x
2)+f
2(x
2,x
3)
Function B:
Characteristic space
SB:
Problem A
Algorithm A that solves Problem A
Step 1. Compute the average projections of F on all elements Xi
of lattice L.
Step 2. Let
Step 3. Execute next step for all r = 1,...,n
Step 4. For all elements Xi of L having cardinality r, let
where the summation is extended to all Xj of L smaller then X
i
Compute function:
Theorem 2.2
Theorem 2.2 proves validity of Algorithm A
Proof of theorem 2.2
(a) For every we have:
(b) and (c). We assume inductively the thesis is true for function k
j(X
j) and spaces S
j with cardinality X
j smaller then r, and prove
for r.
Proof of theorem 2.2 cont'd
Prove that :
If , then
From written as
Proof of theorem 2.2 cont'd
is proved to be solution to Problem A
Optimal approximation with a given interaction graph
Sum of terms:
Interaction graph: Alternative sum of terms:
Problem B
Given function F and interaction graph G find the sum such that G is the interaction graph of and the error |F- | is minimal
Note: interaction graph does not define uniquely the form of approximating function, so Problem B is not trivially reducibleto Problem A
Theorem 3.1
Theorem proves that the form of the optimal approximating sum depends only on given interaction graph G, and not upon the actual values of F.
Theorem:
Characteristic function B of an optimal approximating sum is computable as follows. We have B(X
i) = 1 iff the
set of vertices Wi corresponding to the set of variables
Xi defines a complete subgraph of G.
Proof of theorem 3.1
Example
Problem B reduces to:
- Finding all complete subgraphs of graph G
- Solving problem A
Optimal approximation with a fixed amount of memory
Problem C:
Given function F find a sum whose terms can be stored as tables in no more than M cells of memory and such that the error |F- | is minimal
2 ways of storing a sum
1) The sum
reqires 2N^2 cells
2) Store 6 functions from the table, such that
if any of arguments of f1 : f
5 is zero, then the value of function is
zero and it's not stored. Total storage space:
2 ways of storing a sum
In general, first methos requires
cells, where summation is extended to all maximal sets. Second method requires
Cells, but the summation extends to all sets and is optimal, because it is exactly equal to
the number of dimentions of vector space
2 ways of storing a sum
Error
By definition,
Thus
Translation of Problem C into integer programming problem (0,1) resticted
Problem D:
Determine the integer variables yi (i=1,...,m) (0,1)
restricted such that
with the constraints
Correnpondence between Problem C and Problem D
In Problem D both the objective function and the constraints are linear. Therefore linear interger programming methods apply.