Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation...

59
ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) EE210A: Adaptation and Learning Professor Ali H. Sayed

Transcript of Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation...

Page 1: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED)

EE210A: Adaptation and Learning Professor Ali H. Sayed

Page 2: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2

MAIN RESULT FROM LAST LECTURE

Page 3: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 3

LECTURE #06

CONSTRAINED LINEAR ESTIMATION Sections in order: 6.1-6.3

Page 4: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 4

A MATRIX RESULT

Page 5: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 5

A MATRIX RESULT

Page 6: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 6

MOTIVATION

Page 7: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 7

EXAMPLE

Page 8: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 8

UNKOWN CONSTANT

Page 9: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 9

MINIMUM-VARIANCE UNBIASED EST.

Page 10: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 10

MINIMUM-VARIANCE UNBIASED EST.

Page 11: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 11

MVUE PROBLEM FORMULATION

Page 12: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 12

MVUE OR BLUE PROBLEM

Page 13: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 13

INTERPRETATION

Page 14: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 14

INTUITION

Page 15: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 15

GAUSS-MARKOV THEOREM

Carl F. Gauss (1777-1855)

Andrey A. Markov (1856-1922)

Page 16: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 16

ARGUMENT

Page 17: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 17

CONSTRAINED OPTIMIZATION

Page 18: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 18

EXAMPLE

Page 19: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 19

EXAMPLE

Page 20: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 20

CHANNEL AND NOISE ESTIMATION

Page 21: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 21

COLLECTING DATA

Page 22: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 22

COMPARING ESTIMATORS

Page 23: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 23

NOISE POWER ESTIMATION

Page 24: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 24

DECISION-FEEDBACK EQUALIZATION

Page 25: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 25

CHANNEL AND EQUALIZER MODELS

Page 26: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 26

DFEs

Page 27: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 27

EQUALIZER DESIGN

Page 28: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 28

EQUALIZER DESIGN

Page 29: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 29

EQUALIZER DESIGN

Page 30: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 30

EQUALIZER DESIGN

Page 31: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 31

ASSUMPTIONS

Page 32: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 32

PROBLEM FORMULATION

Page 33: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 33

FORWARD FILTER

Page 34: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 34

BACKWARD FILTER

Page 35: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 35

BACKWARD FILTER

Page 36: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 36

CHANNEL MODEL

Page 37: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 37

CHANNEL MODEL

Page 38: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 38

CHANNEL MODEL

Page 39: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 39

CHANNEL MODEL

Page 40: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 40

CHANNEL MODEL

Page 41: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 41

CHANNEL MODEL

Page 42: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 42

EXAMPLE

Page 43: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 43

EXAMPLE

Page 44: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 44

EXAMPLE

Page 45: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 45

COMPUTER PROJECT

Page 46: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 46

COMPUTER PROJECT

Page 47: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 47

COMPUTER PROJECT

Page 48: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 48

COMPUTER PROJECT

Page 49: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 49

COMPUTER PROJECT

Page 50: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 50

COMPUTER PROJECT

Page 51: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 51

COMPUTER PROJECT

Page 52: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 52

COMPUTER PROJECT

Page 53: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 53

COMPUTER PROJECT

Page 54: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 54

COMPUTER PROJECT

Page 55: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 55

COMPUTER PROJECT

Page 56: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 56

COMPUTER PROJECT

Page 57: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 57

COMPUTER PROJECT

Page 58: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 58

COMPUTER PROJECT

Page 59: Professor Ali H. Sayed - ASL · 2019-01-13 · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 15 . gauss-markov theorem . carl f. gauss (1777-1855)

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 59

COMPUTER PROJECT