An Introduction to Randomness Extractors Ronen Shaltiel University of Haifa Daddy, how do computers...
-
Upload
bruno-bock -
Category
Documents
-
view
212 -
download
0
Transcript of An Introduction to Randomness Extractors Ronen Shaltiel University of Haifa Daddy, how do computers...
An Introduction to Randomness Extractors
Ronen ShaltielUniversity of Haifa
Daddy, how do
computers get random
bits?
Randomized algorithms and protocols
Randomized algorithms/protocols: Receive stream of independent
unbiased coin tosses. Necessary for Crypto.
deterministic algorithm
output
input
Provably help in distributed settings. Randomized algorithms are often simpler and more
efficient than known deterministic ones. (even though we conjecture that BPP=P).
Randomized
“weak source of randomness”
Randomized
algorithm
Computers can sample from: Electro-magnetic noise (Intel) Key strokes of user (Unix) Timing of past events (Unix)These distributions are “somewhat
random” but not “truly random”.Paradigm: randomness extractorsInput: one sample from arbitrary
“weak source of randomness”. Output: independent coin tosses.
How do computers obtain random coin tosses?
RandomnessExtractor
Extensively studied area, dates back to von-Neumann in 1951.
output
input
Coins may be biased and correlated
How do computers obtain random coin tosses?
“weak source of randomness”
Randomized
algorithm
RandomnessExtractor
output
input
Computers can sample from: Electro-magnetic noise (Intel) Key strokes of user (Unix) Timing of past events (Unix)These distributions are “somewhat
random” but not “truly random”.Paradigm: randomness extractorsInput: one sample from arbitrary
“weak source of randomness”. Output: independent coin tosses.
Extensively studied area, dates back to von-Neumann in 1951.
Extractors have applications in: Randomized complexity theory. Cryptography. Network design. Ramsey theory. Coding theory. Combinatorics. Algorithm design. Data structures.
Extractors have many applications
Often not directly related to randomness!
Gives additional motivation to extractors (in addition to the initial motivation of extracting randomness for randomized algs).
“weak source of randomness”
Randomized algorithm
RandomnessExtractor
input
output
Several notions of extractors
Deterministic extractorsRestrict to specific families of “allowed sources”.
Multiple sources extractors Extractor gets samples from several independent
sources.
Seeded extractors Allow extractor to get a seed of few truly random
bits.
Deterministic extractors: Formal definition
“weak source of randomness”
RandomnessExtractor
Distribution X from CDfn: Let C be a set of distributions over {0,1}n (family of “allowed sources”).
E:{0,1}n ! {0,1}m is an extractor for C if 8X2C, random variable E(X) is uniform over {0,1}m.
Two distributions Y,Z over the same domain are ²-close if 8 event A, |Pr[Y 2 A]-Pr[Z 2 A]| ≤ ².
Goal: Design efficiently computable extractors for interesting and general families of sources. Maximize number of extracted bits. Minimize error ².
²-close to
Example: von-Neumann’s sources and extractor (1951!)
Let 0<p≤½ be a parameter (e.g., p=1/10). A vN-source is a distribution X=(X1,..,Xn) s.t.
X1,..,Xn i.i.d. p ≤ Pr[Xi=1] ≤ 1-p.
vN extractor E(x) (extracts one bit): on input x2{0,1}n Scan input bits from left to right. If you see pair “01” stop and output “0”. If you see pair “10” stop and output “1”.
Observation: Pr[“01”] = Pr[“10”] (implies correctness).Subsequent work on extracting many bits [Elias72,Peres92].
X has entropy ≥ pn.
Impossibility of extraction from Santha-Vazirani sources
Let 0<p≤½ be a parameter (e.g., p=1/10). A vN-source is a distribution X=(X1,..,Xn) s.t.
X1,..,Xn i.i.d. p ≤ Pr[Xi=1] ≤ 1-p.
An SV-source is a distribution X=(X1,..,Xn) s.t. Source bits can be correlated. Every next bit is somewhat unpredictable. More formally, 8i, 8x1,..,xi-12{0,1},
p ≤ Pr[Xi=1|X1=x1,..,Xi-1=xi-1] ≤ 1-p.
Thm: [SanthaVazirani86] No extractors for such sources.Historically => research on other notions of extractors.
X has entropy ≥ pn.
Bit-fixing sources [ChorGoldreichFriedmanHastadRudichSmolensky85]
Let k be a parameter.
A k-bit-fixing source is a distribution X=(X1,..,Xn) s.t. k bits are uniformly distributed. remaining n-k bits are fixed to arbitrary values.
Easy to extract one bit: E(X1,..,Xn)=parity(X1,..,Xn)
Thm: [CGFHRS] Impossible to extract 2 bits with zero error for k<n/3.
Probably not a good example for “extraction story”.Naturally arise in cryptographic scenarios.
x1
x2
x3
xn
k random bits
(Non-interactive) Privacy amplification
Alice and Bob share a uniformly chosen key Z2{0,1}n.
Can use random key to encrypt communication on public channel.
Eve somehow learns n-k bits of key.
Alice and Bob don’t know which bits.
Eve’s view: Z is a k-bit fixing-source.
Eve’s view: E(Z) is (close to) uniform. E(Z) is a new and secure key.
Motivates extractors: Extract many bits (hopefully k bits). Explicit (poly-time computable).
Alice
Bob
Eavesdropper
public channel
Z2R{0,1}n
Z2R{0,1}n
Use bit-fixing source
extractor .
E(Z) E(Z)
k random bits
From my point of view Z is distributed like:
Extract m=(1-o(1))k bits
[CGFHRS85][CohenWigderson89][KampZuckerman07]
[GabizonRazShaltiel06][Rao09]
Affine sources Let F be a finite field (typically F2={0,1}).
An affine source is a distribution that is uniform over some affine subspace with dimension k of Fn.
Affine sources generalize bit-fixing sources. Extractor is E:Fn!{0,1} is in particular “anti-linear”:
non-constant on any affine subspace of dimension k. (In extractor jargon, this is called a “disperser”).
Exist for k=O(log n) by probabilistic method. Explicit constructions: (poly-time computable)
Extractor : k=o(n) [Bourgain07]. Disperser: k=no(1) (“anti-linear function”) [Shaltiel11].
Feasibly samplable sources [Blum86,TrevisanVadhan00].
Sources defined by considering an allowed “sampling process”.
Source distribution = Sampler(uniform bits). Restrictions on complexity of sampler
induces family of sources. Small space, Small circuits, Constant depth
circuits… [TV00,KampRaoVadhanZuckerman06, KonigMaurer05,Shaltiel06,Viola11,DeWatson11].
Orthogonal notion of “Feasibly recognizable sources” suggested in [Shaltiel09].
Source uniform on {x:P-1(x)=1} for some procedure P. Restrictions on complexity of procedures induce family.
Several notions of extractors
Deterministic extractorsRestrict to specific families of “allowed sources”.
Multiple sources extractors Extractor gets samples from several independent
sources.
Seeded extractors Allow extractor to get a seed of few truly random
bits.
Multiple sources extractors No deterministic extractors for SV-sources. Possible if you get samples from two independent
sources!
Can allow a more general family than SV-sources. C={distributions X with “high entropy”}. Best we can hope for.
X
n n
Y
2-sourceextractor
Dfn: (min-entropy)
X has min-entropy ≥ k if ∀x: Pr[X=x] ≤ 2-k
“Can hope to extract k random bits from X”.
Seen examples of sources with min-entropy ≥ k.
vN-sources. SV-sources. Bit-fixing sources. Affine sources.
Another example: flat distributions: X uniformly distributed on a subset of size 2k of {0,1}n.
subset flat distribution
Measuring the entropy of the source distribution
“weak source of randomness”
Distribution X over n bits
]Pr[
1logmin)(H
}1,0{ xXX nx
]Pr[
1logExp)(H
}1,0{ xXX nx
size 2k
{0,1}n
A more stringent variant of Shannon entropy
X
Formal definition of Multiple sources extractors
Definition: (emerged from [SanthaVazirani86]) A (k,ε)-2-source-extractor is a function E(x,y) s.t. for every two independent dist. X,Y over n bit strings each having min-entropy ≥ k, E(X,Y) is ε-close to uniform.
Realistic model for generating random bits.Unfortunately, we don’t have good explicit
constructions.
X
n n
Y
2-sourceextractor
Can be generalized to
t>2 sources.
Explicit 2-source extractors imply explicit Ramsey graphs
2-source extractor E(x,y) that outputs one bit is a matrix (w.l.o.g. symmetric)
Property: Every X x Y rectangle of size 2k is balanced.
Þ Every X x X rectangle of size 2k is not monochromatic.
Þ Adjac. matrix of a 2k -Ramsey graph: Graph with no 2k -clique or 2k -independent set.
Explicitly constructing r-Ramsey graphs for small r is a longstanding open problem.
0 0 1 0 0 0 0 1 0
0 1 0 0 1 1 1 0 1
1 0 1 1 1 0 0 1 0
0 1 1 0 0 0 0 1 0
0 1 0 1 0 0 1 1 0
1 0 1 0 0 0 1 0 0
0 1 0 1 0 1 1 0 1
1 0 1 0 1 1 0 1 0
0 1 0 1 0 0 1 0 0
2n
X
Y
x
yX
Explicit constructions of 2-source extractors and Ramsey graphs
2k-Ramsey graphs on 2n nodes
Erdős 47: Probabilstic method achieves k≈log n
Frankl and Wilson 81: Explicit construction k≈(n log n)½
[BKSSW05,BRSW06]: Explicit construction k=no(1)
(Extractor techniques).
Construct bipartite Ramsey graphs (stronger than Ramsey graphs but weaker than 2-source extractors).
(k,ε)-2-source extractors
Probabilstic method achieves k≈log n
Chor and Goldreich 88: E(x,y)=<x,y>mod 2 works for k ≥ n/2.
Bourgain 05: Explicit construction k=0.4999n.
Progress on t-source extractors [BIW04,BKSSW05,Rao06].
Rao06: extract from log n/log k sources with min-ent k.
Several notions of extractors
Deterministic extractorsRestrict to specific families of “allowed sources”.
Multiple sources extractors Extractor gets samples from several independent
sources.
Seeded extractors Allow extractor to get a seed of few truly random
bits.
We allow an extractor to also receive an additional seed of (few) independent random bits.
Makes sense as long as: # bits extracted > seed length.
Handle all high min-entropy sources!
Seeded extractors [NisanZuckerman92]
source dist. X on n bits
Extractorseed
Y
random output
Randomness
Definition: A (k,ε)-extractor is a function E(x,y) s.t. for every dist. X with min-entropy ≥ k, E(X,Y) is ε-close to uniform .
Lower bounds [RadhakrishnanTaShma98]: seed length ≥ log(n-k) + 2log(1/ε)
Probabilistic method: Exists optimal extractor which matches lower bound and extracts all the k random bits in the source distribution.
Explicit constructions: E(x,y) can be computed in poly-time.
uniformly distributed
Current milestones in explicit constructions: [LuReingoldVadhanWigderson03, GuruswamiUmansVadhan07,DvirWigderson08,DvirKoppartySarafSudan09].“Optimal up to constants”: seed = O(log(n) + log(1/ε)) output (k) bits.For constant error: seed = O(log(n)) output (1-o(1))∙k bits.
Simulating randomized algorithms using weak random sources
Goal: Run rand algorithm with a weak random source of randomness.
Where can we get a seed?Idea: Go over all seeds. Given sample X from
source. ∀y compute zy= E(X,y) Compute Alg(input,zy) Answer majority vote.
seed=O(log n)=>poly-time.Explicit constructions.
Unsuitable for crypto protocols.
Randomized algorithm
input
output
random coins
RandomnessExtractor
seed
source dist. X on n bits
Something about the tools used in explicit constructions
2-wise independent hash functions [ImpagliazzoLevinLuby89,NisanZuckerman92]. E(x,h)=h(x),h where h is chosen from small family of 2-
wise independent hash functions. Disadvantage: huge seed.
List decodable error correcting codes [Trevisan99]. E(x,y)=Enc(x)y,y where Enc is a binary list decodable error
correcting code. (also works vice-versa). Rate ≥ 1/poly(n) => logarithmic seed. Disadvantage: extract only one additional bit. Can try and exploit properties of specific codes
[TaShmaZuckeramanSafra01,ShaltielUmans01,GuruswamiUmansVadhan07
]. Various composition methods […]
long seedextractor
Composing short seed extractor with long output extractor
x1
x2
x3
xn
k bits of min-entropy
short seedextractor
Short random output
Seeded Extractors are only guaranteed to work when the source and seed are independent.
correlated!
!long random output
Nevertheless, many constructions make this “go through” by modifying initial extractors to have additional properties.
Seeded extractors as graphs with “volume expansion”.
Extractor is a bipartite graph.
Given extractor E(x,y) N=2n (# of inputs) M=2m (# of outputs) K=2k (# of source
elements) D=2d (# of seeds)
Connect x to E(x,1),..,E(x,D).
Small seed length d ~ log n => small deg D ~ log N.
D=2d edges
x
N≈{0,1
}n
M≈{0,1}m
E(x,1)
E(x,D)
..
Extractor graphs: volume expansion property
Extractor property:∀dist X of min-
entropy≥k,E(X,Y) ε-close to
uniform.
=>“expansion” property:
∀set X of size K=2k ,|Γ)X)| ≥ (1-ε)M.
Such graph/function is called “Disperser”.
X
N≈{0,1
}n
M≈{0,1}m
K=2k Γ(X)
(1-ε)M
Extractors and Expander graphs
X
N≈{0,1
}n
M≈{0,1}m
Γ(X)
(1-ε)M
Extractor
N≈{0,1
}n
X Γ(X)
D=2d edges
(1+δ-)Expander
(1+δ)KK
N≈{0,1
}n
K=2k
Volume expansion:
K -> (1-ε)M
K/N -> (1-ε)
Extractors and Expander graphs
X
N≈{0,1
}n
M≈{0,1}m
Γ(X)
(1-ε)M
Extractor
N≈{0,1
}n
X Γ(X)
(1+δ-)Expander
(1+δ)K
N≈{0,1
}n Size expansion:
K -> (1+δ)K
K K=2k
Extractors produce better results in some applications of
expanders
Expanders with expansion that beat the eigenvalue bound [WigdersonZuckerman93]
Goal: Construct low deg expanders with huge expansion.
Line up two low degree extractors.
∀set X of size K , (where K<<N)|Γ)X)| ≥ (1-ε)M > M/2.∀sets X,X’ of size KX and X’ have common
neighbour. Contract middle layer. Bipartite graph in which
every set of size K sees N-K vertices.
Trivially degree ≥ (N-K)/K ≈ N/K.
Obtain low degree ND2/K. Eigenvalue methods cannot
yield graphs with such parameters.
N≈{0,1
}n
N≈{0,1
}n
X
X’
v1
v2 v3
vD
Randomness efficient (oblivious) sampling using expanders [AjtaiKormlosSzemeredi87]
Random walk variables v1..vD behave like i.i.d:
∀A of size ½M Hitting property:
Pr[∀i : vi∊A] ≤ δ = 2-Ω(D). Chernoff style property:
Pr[#i : vi∊A far from exp.] ≤ δ = 2-Ω(D).
# of random bits used for walk: m+O(D)=m+O(log(1/δ))
# of random bits for i.i.d. m∙D=m ∙ O(log(1/δ))
M≈{0,1}m
Random walk on constant degree
expander
Randomness efficient (oblivious) sampling using extractors [Sipser86,Zuckerman96]
Given parameters m,δ: Use E with k=m,
n=m+log(1/δ) ε<½ and small seed d.
Choose random x: m+log(1/δ) random bits.
Set vi=E(x,i)
Expansion property ⇒ Hitting prop.
∀A of size ½MCall x bad if ∀i: E(x,i) inside A.# of bad x’s < K=2k
Pr[x is bad] < 2k/2n = δ
D edges
x
N≈{0,1
}n
M≈{0,1}m
E(x,1)
E(x,D)
..
bad x’s
(1-ε)M
A
Every (oblivious) sampling scheme yields an extractor
An (oblivious) sampling scheme uses a random n bit string x to generated D random variables.
Thm: [Zuckerman06] if the scheme has sampling property then the derived graph is an extractor.
Extractors oblvs Sampling
D=2d edges
x
N≈{0,1
}n
M≈{0,1}m
E(x)1
E(x)D
..
Extractors come in several flavors and have many applications in diverse fields.
Goal: Explicitly construct extractors with parameters that match existential bounds.
Many open problems.
See article in proceedings for more details.
Conclusion
“weak source of randomness”
Randomized algorithm
RandomnessExtractor
input
output
Thank You…
Daddy, can you tell me that story
again?