When is Randomness Extraction Possible?

David Zuckerman

University of Texas at Austin

Randomness in Computer Science

• Many uses of randomness in CS.– Randomized algorithms– Cryptography– Distributed computing

• But: Natural sources may be defective.– Clock drift, thermal noise, Zener diode.

What is minimal randomness requirement?

• Can we eliminate randomness completely?• If not:

– Can we minimize quantity of randomness?– Can we minimize quality of randomness?

• What does this mean?

What is minimal randomness requirement?

• Can we eliminate randomness completely?• If not:

– Can we minimize quantity of randomness?• Pseudorandom generator

– Can we minimize quality of randomness?• Randomness extractor

Pseudorandom Numbers

• Computers rely on pseudorandom generators:

PRG71294 141592653589793238

short random string

long “random-enough”string

What does “random enough” mean?

Modern Approach to PRGs[Blum-Micali, Yao]

Alg

Alg

random

pseudorandom

≈ samebehavior

Require PRG to “fool” all efficient algorithms.

Using Defective (Weak) Randomness

• Simulate randomized algorithms• Stronger: extract high-quality randomness:

• Which models admit such extraction?

Ext n bits m bits

≈ uniform

Simple example:

extractor

random bit

Ext(x1,…,xn) = Parity(x1,…,xn)

`bit-fixing’ distribution (don’t know where rand. bit is)

1 0 1 0 0

Harder when input bits dependent.

Modeling General Weak Sources

• Source = random variable X on {0,1}n.• Attempt #1: Shannon Entropy

9

Problem:D: with prob. .99 0n

with prob. .01 uniform on n bits

Min-Entropy:

Min-Entropy

X

• (n,k)-source: X on {0,1}n with min-entropy k.

• Min-entropy k iff all strings have probability ≤ 2-k.

• Special Case: X uniform on set of size 2k.

• General Case: Enough to handle special case (Chor-Goldreich 88).

10

Can Arise in Different Ways

• Physical source of randomness.• Cryptography: condition on adversary’s

information, e.g. bounded storage model.

• Pseudorandom generators (for space s machines): condition on TM configuration.

Goal: Extract Randomness

Ext n bits m bits

statistical error

Problem: Impossible, even for k=n-1, m=1, ε<1/2.

Impossibility Proof

• Suppose f:{0,1}n {0,1} satisfies sources X ∀with H∞(X) ≥ n-1, f(X) ≈ U.

f-1(0)f-1(1)

Take X=f-1(0)

What if More Structure?

• Semirandom sources [Santha-Vazirani ‘84]– δ < Pr[Xi|X1=x1,…,Xi-1=xi-1] < 1-δ

• Extraction impossible.• But can simulate randomized algorithms

[Vazirani-Vazirani ‘85].• Can simulate even in general setting [Z ‘91].

Goal: Extract randomness with minimal assumptions on source distribution.

Outline• Extractors for Structured Sources

– Algebraic sources: bit-fixing, affine, additive– Complexity-theoretic sources

• Seeded Extractors– Gives simulation of randomized algorithms– Other applications

• Independent-Source Extractors• Network extractor Protocols• Conclusions

Extractors for Structured Sources

• Probabilistic Method: If ≤ sources of min-entropy k:

Can extract m=(1-α)k bits with error 2-αk/3.• Algebraic sources:

– Bit-fixing, affine, additive, polynomial, variety.• Complexity-theoretic sources:

– AC0 sources, small-space sources.• Independent sources.

Oblivious Bit-Fixing Source

• Example: ?0010?111??11.– ? = uniform on {0,1}.– (n-k) bits fixed by adversary; k uniform bits.– Parity extracts 1 bit.

• For k≥logc n, can extract k-o(k) bits [GRS, Rao].• Application: Exposure Resilient Cryptography.

– Adversary learns many bits of secret key.– Can still do cryptography.

Non-Oblivious Bit-Fixing Source

• Adversarial bits may depend on random bits.– k uniform bits; (n-k) bits fixed by adversary.

• Parity fails even when k=n-1.• Extraction impossible when k≤n-cn/log n.• Majority extracts when k≥n-c√n.• Ajtai-Linial: extractor for k≥n-cn/log2 n.

Affine Source

• Random vector from (unknown) affine subspace.• Generalizes oblivious bit-fixing sources.• Large fields: dimension>0 [Gabizon-Raz 2005].• Over F2: extractor for min-entropy αn, any α>0

[Bourgain 2007].• New extractor for min-entropy k≥logc n

[Li 2015, building on Chattopadhyay-Z 2015]• Affine extractors used for other extractors.• Gives circuit lower bound [Demenkov-Kulikov‘11]

Minimum additive structure?[Bhowmick-Gabizon-Le-Z 2015]

• Attempt 1:• A is an additive set if |A+A|≤2|A|• Additive source: uniform on additive set.

• Claim: No extractor f for such sources.• Proof: A:= Larger of f-1(0) and f-1(1).• |A+A|≤2|A|, but f(A) constant.

• For smaller A, intersect f-1(0) with B: |B+B|≤2|B|.– |A+A|≤4|A|

Symmetric Sets• A = subset of additive group G.

• SYM(A): elements of G that can be written in many ways as difference of elements of A.

• x= a1-b1 = a2-b2 =a3-b3 =..

• If A is a subgroup/subspace:Any x in A can be written in |A| ways.

Extractors for Additive Sources

• SYM0.5(A) , {x in G | x can be written in |A|/2 ways as x= a-b , a,b in A}

• Dfn: A is an additive set if:

- |A+A| ≤ |A|1.1

- SYM0.5(A) > |A|/2

• Thm [BGLZ]: For large p, any constant δ>0 :Explicit extractor for additive sources in Zp

and (Zp) n with entropy rate δ.

Complexity-Theoretic Sources

• X=f(Uniform), complexity(f) small.• Deterministic extraction possible under

assumptions [Trevisan-Vadhan ‘00].• No assumptions:

– NC0 [De-Watson ‘11, Viola ‘11]– AC0 [Viola ‘11]– Proofs reduce to low-weight affine extractors [Rao

‘09].

Small Space Sources• Space s source: min-entropy k source

generated by width 2s branching program.

n+1 layers

1 1 0 1 0 0

1/, 0

1-1/, 0 1,10.1,0

0.8,1

0.1,0

0.3,0

0.5,10.1,1

0.1,0

1

width 2s

Bit Fixing Sources can be modelled by Space 0 sources

? 1 ? ? 0 1

0.5,1 0.5,1 0.5,1

0.5,0 0.5,0 0.5,0

1,1 1,0 1,1

Extractors for Small Space Sources

• For k ≥ n1-δ, space n1-3δ, can extract k-o(k) bits [Kamp-Rao-Vadhan-Z ‘06].

• Proof idea:– Condition on intermediate states.– Reduces to variants of independent sources.

Seeded Extractor[Nisan-Z ‘93,…, Guruswami-Umans-Vadhan ’07,…]

Ext n bits m =.99k bits

statistical error

d=O(log (n/ε)) random bit seed Y

Strong extractor: (Ext(X,Y),Y) ≈ Uniform

Simulating Randomized Algorithms• Randomized algorithm R using m random bits.• Assume no high-quality randomness available.

– Available random source X has H∞(X)≥k>m.

• Given extractor for H∞(X)≥k– seed length d=O(log n), output length m.

• Simulate with factor 2d blowup:– Run R with random string Ext(x,y1),…,Ext(x,y2d).– Take majority vote or median.

Applications of Extractors

• PRGs for Space-Bounded Computation [Nisan-Z]• PRGs for Random Sampling [Z]• Cryptography [Lu, Vadhan, CDHKS, Dodis-Smith]• Expander graphs and superconcentrators [Wigderson-Z]• Coding theory [Ta-Shma- Z]• Hardness of approximation [Z, Umans, Mossel-Umans]• Efficient deterministic sorting [Pippenger]• Time-space tradeoffs [Sipser]• Data structures [Fiat-Naor, Z, BMRV, Ta-Shma]

Use in Privacy Amplification[Bennett, Brassard, Robert 1985]

• Goal: convert weak shared secret X to uniform secret.• Unbounded passive adversary.

public

Pick Y

Shared secret = Ext(X,Y). Correct by strong extractor definition.

Graph-Theoretic View: “Expansion”

(1-)M K=2k

D=2d

N=2n

M=2m

Can use this to constructexpanders beatingeigenvalue bound [WZ]

x y Ext(x,y)

output uniform

Alternate View

S

BADS

D=2d

N=2n M=2m

x

Other direction:ErrorS ≤ |BADS|2-k + ε

Averaging Sampler via Alternate View [Z ‘96]

• Goal: Estimate mean μ of– Black box access to f.

Algorithm: Pick x randomly in {0,1}n. Sample f at Γ(x) = {x1,…,xD}.

Output μf.

Pr[error > ε] = |BADf|/2n.

Use 1.01m random bits: Pr[error >1/poly]=2-Ω(m).

Independent Sources

n bits n bits

Ext

m =Ω(k) bits statistical error

2-Source Extractors

• Inner product extracts for min-entropy > n/2.• Bourgain 2005: min-entropy .49n.• Chattopadhyay-Z ‘15: min-entropy polylog(n)

– Uses non-malleable extractors and extractors for non-oblivious bit-fixing sources.

Interleaved Sources

• Independent sources interleaved arbitrarily– e.g. X1X2Y1X3Y2Y3Y4X4

• Raz-Yehudayoff 2011: Extractor for min-entropy .99n each.

• Chattopadhyay-Z 2015: .99n and clog n.– Larger fields .51n and clog n.– Gives extractor for any-order small-space sources with

min-entropy .51n.• Gives lower bound on best-partition communication

complexity.

Construction Idea

• Use 2-source extractor of form f(X+Y) in Fpr.

– e.g., Quadratic character in Fpr.

• Find vectors v1,…,v2n in Fpr with span of any n

having dimension at least d. Want r lg p < 2n.• Ext(z1,…,z2n) = f(Σzivi)

• H∞(Σ’zivi) ≥ k–(n-d), where Σ’ is over i from X.– Same for Y.

Cryptography with Weak Sources

• Players have independent weak sources.• Allow Byzantine faults.• For 2 players, impossible [DOPS].• For more players, possible!

Network Extractor Protocol [Goldwasser-Sudan-Vaikunthanatan05, Dodis-

Oliveira03]

010101010

01001011011011

11010

100100101

10100

010100101

10110

011110101

11001

01010101

01001

001010101

01001

010111101

10101

Input: x1,…,xp 2 {0,1}n from independent weak random sources

Output: z1,…,zp 2 {0,1}m private nearly-uniformrandom strings (for honest parties)

Byzantine faults:can send arbitrary messages

Network Extractor Protocols

• After running network extractor protocol, run standard protocol, e.g., Byzantine Agreement.

• Naïve idea to design protocol:– A few players broadcast sources.– Remaining players apply independent-source

extractor to those sources and own source.– Problem: what if only malicious players

broadcast?

Network Extractor Constructions

• Information-theoretic setting [Kalai-Li-Rao-Z]:– For k ≥ exp(logα n), can still tolerate linear number

of faults in BA and leader election, any α>0.• Computational setting [Kalai-Li-Rao]:

– Under certain crypto assumptions, for k = αn, secure multiparty computation if ≥ 2 honest players.

Conclusions

• Extraction possible for:– Algebraic: Oblivious bit-fixing; affine; additive.– Complexity: AC0; small space.

• Extraction impossible for:– Non-oblivious bit-fixing (unless k>n - n/log2 n).– SV sources.

• Can extract from general sources if add:– O(log n) uniform bits.– A second weak source.

Thank you!