9.3 Complex Numbers; The Complex Plane; Polar Form of Complex Numbers.
Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ......
Transcript of Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ......
![Page 1: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/1.jpg)
Linear System Theory, Complex Numbers, Convolution
T H E U N I V E R S I T Y of T E X A S
H E A L T H S C I E N C E C E N T E R A T H O U S T O N
S C H O O L of H E A L T H I N F O R M A T I O N S C I E N C E S
For students of HI 5323 “Image Processing”
Willy Wriggers, Ph.D.School of Health Information Sciences
http://biomachina.org/courses/processing/06.html
![Page 2: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/2.jpg)
InvarianceIf some operation on a signal commutes with a particular transformation, that operation is invariant to that transformation:
U(Tf) = T(Uf)
The operation U is invariant to transformation T
![Page 3: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/3.jpg)
Examples of Invariance
Non-uniform Amp.
Uniform Amplification
Warping
Scaling
Rotation
Translation
Intensity DifferenceDirectionDistanceOperation
Tran
sfor
mat
ion
Yes Yes YesYes No YesNo Yes YesNo No YesYes Yes NoYes Yes No
![Page 4: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/4.jpg)
Linearity: Revisited
A function f is linear iff (if and only if):
f(ax + by) = af(x) + bf(y)
This can be broken down into two components
1. f(ax) = af(x) (scalar multiplication)
2. f(x + y) = f(x) + f(y) (addition)
![Page 5: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/5.jpg)
Shift InvarianceShift invariance: an operation is invariant to translation
Implication: shifting the input produces the same output with an equal shift
if x(t) → y(t)
then x(t + T) → y(t + T)
![Page 6: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/6.jpg)
SystemsLinearity and shift invariance are nice properties for a signal-processing operation to have
– Input devices
– Output devices
– Processing
In signal processing, a transformation that is linear and shift invariant is called a system.
![Page 7: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/7.jpg)
Reality CheckNo physical device is really a system:
• Linearity is limited by saturation
• Shift invariance is limited by sampling duration or field of view
• Random noise isn’t linear
![Page 8: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/8.jpg)
ImpulsesOne way of probing what a system does is to test it on a single input point (a single spike in the signal, a single point of light, etc.)
Mathematically, a perfect single-point input is written as:
δ(t) =
and
δ(t)dt = 1
This is called the Dirac delta function
⎧⎨⎩
∞ if t = 00 otherwise
∞
⌠⌡-∞
![Page 9: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/9.jpg)
Impulses (cont.)
Multiplying a delta function by a constant multiplies the integrated area:
a δ(t)dt = a∞
⌠⌡-∞
![Page 10: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/10.jpg)
Impulse ResponseBecause a system is shift-invariant, it responds the same everywhere:
δ(t) → h(t)
implies
δ(t + T) → h(t + T)
This response h(t) is called the impulse response or point spread function
![Page 11: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/11.jpg)
Impulse ResponseBecause a system is linear, the response to a multiplied impulse is the same as the multiple times the response:
δ(t) → h(t)
implies
a δ(t) → a h(t)
![Page 12: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/12.jpg)
Impulse ResponseBecause a system is linear, the response to two impulses is the same as the sum of the two responses individually:
δ(t) → h(t)
δ(t + T) → h(t + T)
Implies
δ(t) + δ(t + T) → h(t) + h(t + T)
![Page 13: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/13.jpg)
Impulse ResponsePutting it all together:
δ(t) → h(t)
implies
a δ(t) + b δ(t + T) → a h(t) + b h(t + T)
Implication: If you know the impulse response at any point, you know everything there is to know about the system!
![Page 14: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/14.jpg)
Complex Numbers: Review
A complex number is one of the form:
a + bi
where
i = √-1a: real part
b: imaginary part
![Page 15: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/15.jpg)
Complex ArithmeticWhen you add two complex numbers, the real and imaginary parts add independently:
(a + bi) + (c + di) = (a + c) + (b + d)i
When you multiply two complex numbers, you cross-multiply them like you would polynomials:
(a + bi) × (c + di) = ac + a(di) + (bi)c + (bi)(di)
= ac + (ad + bc)i + (bd)(i2)
= ac + (ad + bc)i - bd
= (ac - bd) + (ad + bc)i
![Page 16: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/16.jpg)
Polynomial Multiplication
p1(x) = 3 x2 + 2 x + 4p2(x) = 2 x2 + 5 x + 1
p1(x) p2(x) = ____x4 + ____x3 + ____x2 + ____x + ____
![Page 17: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/17.jpg)
The Complex PlaneComplex numbers can be thought of as vectors in the complex plane with basis vectors (1, 0) and (0, i):
Real
Imaginary
1-1
i
-i
i
![Page 18: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/18.jpg)
Magnitude and PhaseThe length of a complex number is its magnitude:
|a + bi| = √a2 + b2
The angle from the real-number axis is its phase:
φ (a + bi) = tan-1(b / a)
When you multiply two complex numbers, their magnitudes multiply
|z1z2| = |z1||z2|
And their phases add
φ (z1z2) = φ (z1) + φ (z1)
![Page 19: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/19.jpg)
The Complex Plane: Magnitude and Phase
Real
Imaginary
1-1
i
-i
z1
z2z1z2
Real
Imaginary
1-1
i
-i
i
![Page 20: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/20.jpg)
Complex ConjugatesIf z = a + bi is a complex number, then its complex conjugate is:
z* = a - bi
The complex conjugate z* has the same magnitude but opposite phase
When you add z to z*, the imaginary parts cancel and you get a real number:
(a + bi) + (a - bi) = 2a
When you multiply z to z*, you get the real number equal to |z|2:
(a + bi)(a - bi) = a2 – (bi)2 = a2 + b2
![Page 21: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/21.jpg)
Complex DivisionIf z1 = a + bi, z2 = c + di, z = z1 / z2, the division can be accomplished by multiplying the numerator and denominator by the complex conjugate of the denominator:
2 2 2 2
( )( )( )( )a bi c di ac bd bc adz ic di c di c d c d
+ − + −⎛ ⎞ ⎛ ⎞= = +⎜ ⎟ ⎜ ⎟+ − + +⎝ ⎠ ⎝ ⎠
![Page 22: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/22.jpg)
Euler’s Formula
• Remember that under complex multiplication:– Magnitudes multiply– Phases add
• Under what other quantity/operation does multiplication result in an addition?– Exponentiation: cacb = ca + b (for some constant c)
• If we have two numbers of the form m·ca (where c is some constant), then multiplying we get:
(m·ca ) (n·cb) = m·n·ca + b
• What constant c can represent complex numbers?
![Page 23: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/23.jpg)
Euler’s Formula• Any complex number can be represented using Euler’s
formula:z = |z|eiφ (z) = |z|cos(φ ) + |z|sin(φ )i = a + bi
Real
Imaginary
1-1
i
-i
b
aφ
a = |z|cos(φ )b = |z|sin(φ )
![Page 24: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/24.jpg)
Powers of Complex NumbersSuppose that we take a complex number
z = |z|ei φ (z)
and raise it to some powerzn = [|z|ei φ (z)]n
= |z|n ei n φ (z)
zn has magnitude |z|n and phase n φ (z)
![Page 25: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/25.jpg)
Real
Imaginary
1-1
i
-i
Powers of Complex Numbers: Example
• What is in for various n?
, 4n = 0
, 5n = 1
n = 2, 6 , 8
n = 3, 7
![Page 26: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/26.jpg)
Real
Imaginary
1-1
i
-i
Powers of Complex Numbers: Example
• What is (eiπ/4)n for various n?
, 8n = 0
, 9
n = 2
n = 4
n = 6
n = 1n = 3
n = 5n = 7
![Page 27: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/27.jpg)
Real
Imaginary
1-1
i
-i
Harmonic Functions• What does x(t) = eiωt look like?• x(t) is a harmonic function (a building block for later analysis)
eiωt
Angular frequency
Time
![Page 28: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/28.jpg)
Harmonic Functions as Sinusoids
sin(ωt)cos(ωt)
ℑ(eiωt)ℜ(eiωt)Imaginary PartReal Part
![Page 29: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/29.jpg)
Harmonics and SystemsIf we present a harmonic input (function)
x1(t) = eiωt
to a shift-invariant linear system, it produces the response
x1(t) → y1(t)
y1(t) = K(ω, t) x1(t) = K(ω, t) eiωt
where, for now, we simply define
tietytK ωω )(),( 1=
![Page 30: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/30.jpg)
Harmonics and Systems: Shifted Input
Suppose we create a harmonic input (function) by shifting the original input
x2(t) = x1(t – T) = eiω(t – T)
The response, x2(t) → y2(t), to this shifted input is
y2(t) = K(ω, t – T) x2(t) = K(ω, t – T) eiω(t – T)
![Page 31: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/31.jpg)
Harmonics and Systems: Shifted Input
However, note that
x2(t) = eiω(t – T) = eiωt e–iωT = x1(t) e–iωT
Thus, the response can be written
x2(t) → y1(t) e–iωT
y2(t) = y1(t) e–iωT = K(ω, t) x1(t) e–iωT =
K(ω, t) x2(t)
![Page 32: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/32.jpg)
Harmonics and Systems: Shifted Input
Now we have both
y2(t) = K(ω, t) x2(t)
y2(t) = K(ω, t – T) x2(t)Thus,
K(ω, t – T) = K(ω, t) So, K is just a constant function with respect to t:
K(ω)
![Page 33: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/33.jpg)
Harmonics and SystemsThus, for any harmonic function
x(t) = eiωt
we have
x(t) → y(t)
y(t) = K(ω) x(t) = K(ω) eiωt
Implication: When a system (a shift-invariant linear transformation)is applied to a harmonic signal, the result is the same harmonicsignal multiplied by a constant that depends only on the frequency
![Page 34: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/34.jpg)
Transfer FunctionsWe now have a second way to characterize systems:1: If you know the impulse response at any point, you know
everything there is to know about the system 2: The function K(ω) defines the degree to which harmonic inputs
transfer to the outputK(ω) is the called the transfer function
![Page 35: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/35.jpg)
Transfer FunctionsExpressing K(ω) in polar (magnitude-phase) form:
K(ω) = A(ω) eiφ(ω)
Recall that the magnitudes multiply and the phases add:K(ω) eiωt = A(ω) ei[ωt + φ(ω)]
A(ω) is called the Modulation Transfer Function (MTF)– Magnitude of the transfer function– Indicates how much the system amplifies or attenuates input
φ(w) is called the Phase Transfer Function (PTF)– Phase of the transfer function– Only effect is to shift the time origin of the input function
![Page 36: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/36.jpg)
Impulse ResponseRemember that we can entirely characterize a system by its impulse response:
δ (t) → h(t)
Problem: given an input signal x(t), how do we determine the output y(t)
![Page 37: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/37.jpg)
Linearity and Shift InvarianceBecause a system is linear:
a δ (t) → a h(t)
Because a system is shift invariant:δ (t – k) → h(t – k)
![Page 38: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/38.jpg)
Response to an Entire SignalA signal x(t) can be thought of as the sum of a lot of weighted, shifted impulse functions:
where − δ (τ – t) is the delta function at τ– x(t) is the weight of that delta function
(Read the integral simply as summation)
( ) ( )x t x t dτ δ( τ) τ∞
−∞
= −∫
![Page 39: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/39.jpg)
Response to an Entire Signal (cont.)
Because of linearity, each impulse goes through the system separately:
x(τ) δ (t – τ) → x(τ) h(t – τ)This means
( ) ( )x t d x h t dτ δ( τ) τ τ ( τ) τ∞ ∞
−∞ −∞
− → −∫ ∫
![Page 40: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/40.jpg)
Response to an Entire Signal (cont.)
So,
This operation is called the convolution of x and h
( ) ( )y t x h t dτ ( τ) τ∞
−∞
= −∫
![Page 41: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/41.jpg)
ConvolutionConvolution of an input x(t) with the impulse response h(t) is written as
x(t) * h(t)
That is to say,
ττ)(τ dthxthtx ∫∞
∞−
−=∗ )()()(
![Page 42: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/42.jpg)
Response to an Entire SignalSo, the response of a system with impulse response h(t)to input x(t) is simply the convolution of x(t) and h(t):
ττ)(τ dthxthtxtytx ∫∞
∞−
−=∗=→ )()()()()(
![Page 43: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/43.jpg)
Convolution of Discrete FunctionsFor a discrete function x[j] and impulse response h[j]:
∑ −⋅=∗k
kjhkxjhjx ][][][][
![Page 44: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/44.jpg)
One Way to Think of Convolution
Think of it this way:
– Shift a copy of h to each position t (or discrete position k)
– Multiply by the value at that position x(t) (or discrete sample x[k])
– Add shifted, multiplied copies for all t (or discrete k)
∑ −⋅=∗k
kjhkxjhjx ][][][][
ττ)(τ dthxthtx ∫∞
∞−
−=∗ )()()(
![Page 45: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/45.jpg)
Example: Convolution – One wayx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]
x[0] h[j – 0]= [ __ __ __ __ __ __ __ __ __ ]x[1] h[j – 1]= [ __ __ __ __ __ __ __ __ __ ]x[2] h[j – 2]= [ __ __ __ __ __ __ __ __ __ ]x[3] h[j – 3]= [ __ __ __ __ __ __ __ __ __ ]x[4] h[j – 4]= [ __ __ __ __ __ __ __ __ __ ]
x[j] * h[j] = x[k] h[j – k]
= [ __ __ __ __ __ __ __ __ __ ]
Σk
![Page 46: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/46.jpg)
Example: Convolution – One wayx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]
x[0] h[j – 0]= [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1]= [ __ __ __ __ __ __ __ __ __ ]x[2] h[j – 2]= [ __ __ __ __ __ __ __ __ __ ]x[3] h[j – 3]= [ __ __ __ __ __ __ __ __ __ ]x[4] h[j – 4]= [ __ __ __ __ __ __ __ __ __ ]
x[j] * h[j] = x[k] h[j – k]
= [ __ __ __ __ __ __ __ __ __ ]
Σk
![Page 47: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/47.jpg)
Example: Convolution – One wayx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]
x[0] h[j – 0]= [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1]= [ __ 4 8 12 16 20 __ __ __ ] x[2] h[j – 2]= [ __ __ __ __ __ __ __ __ __ ]x[3] h[j – 3]= [ __ __ __ __ __ __ __ __ __ ]x[4] h[j – 4]= [ __ __ __ __ __ __ __ __ __ ]
x[j] * h[j] = x[k] h[j – k]
= [ __ __ __ __ __ __ __ __ __ ]
Σk
![Page 48: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/48.jpg)
Example: Convolution – One wayx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]
x[0] h[j – 0]= [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1]= [ __ 4 8 12 16 20 __ __ __ ] x[2] h[j – 2]= [ __ __ 3 6 9 12 15 __ __ ]x[3] h[j – 3]= [ __ __ __ __ __ __ __ __ __ ]x[4] h[j – 4]= [ __ __ __ __ __ __ __ __ __ ]
x[j] * h[j] = x[k] h[j – k]
= [ __ __ __ __ __ __ __ __ __ ]
Σk
![Page 49: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/49.jpg)
Example: Convolution – One wayx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]
x[0] h[j – 0]= [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1]= [ __ 4 8 12 16 20 __ __ __ ] x[2] h[j – 2]= [ __ __ 3 6 9 12 15 __ __ ]x[3] h[j – 3]= [ __ __ __ 1 2 3 4 5 __ ]x[4] h[j – 4]= [ __ __ __ __ __ __ __ __ __ ]
x[j] * h[j] = x[k] h[j – k]
= [ __ __ __ __ __ __ __ __ __ ]
Σk
![Page 50: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/50.jpg)
Example: Convolution – One wayx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]
x[0] h[j – 0]= [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1]= [ __ 4 8 12 16 20 __ __ __ ] x[2] h[j – 2]= [ __ __ 3 6 9 12 15 __ __ ]x[3] h[j – 3]= [ __ __ __ 1 2 3 4 5 __ ]x[4] h[j – 4]= [ __ __ __ __ 2 4 6 8 10 ]
x[j] * h[j] = x[k] h[j – k]
= [ __ __ __ __ __ __ __ __ __ ]
Σk
![Page 51: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/51.jpg)
Example: Convolution – One wayx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]
x[0] h[j – 0]= [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1]= [ __ 4 8 12 16 20 __ __ __ ] x[2] h[j – 2]= [ __ __ 3 6 9 12 15 __ __ ]x[3] h[j – 3]= [ __ __ __ 1 2 3 4 5 __ ]x[4] h[j – 4]= [ __ __ __ __ 2 4 6 8 10 ]
x[j] * h[j] = x[k] h[j – k]
= [ 1 6 14 23 34 39 25 13 10 ]
Σk
![Page 52: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/52.jpg)
Another Way to Look at Convolution
Think of it this way:
– Flip the function h around zero
– Shift a copy to output position j
– Point-wise multiply for each position k the value of the function x and the flipped and shifted copy of h
– Add for all k and write that value at position j
∑ −⋅=∗k
kjhkxjhjx ][][][][
![Page 53: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/53.jpg)
Why Flip the Impulse Function?An input at t produces a response at t + τ of h(τ)
Suppose I want to determine the output at t
What effect does the input at t + τ have on t?
h(–τ)
![Page 54: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/54.jpg)
Convolution in Two DimensionsIn one dimension:
In two dimensions:
Or, in discrete form:
ττ)(τ dthxthtx ∫∞
∞−
−=∗ )()()(
∫ ∫∞
∞−
∞
∞−
−−=∗ yxyxyx ddyxhIyxhyxI τττ,ττ,τ )()(),(),(
∑∑ −−=∗k j
kyjxhkjIyxhyxI ],[],[],[],[
![Page 55: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/55.jpg)
Example: Two-Dimensional Convolution
____ ____ ____ ____ ____ ________ ____ ____ ____ ____ ________ ____ ____ ____ ____ ________ ____ ____ ____ ____ ________ ____ ____ ____ ____ ________ ____ ____ ____ ____ ____
1 1 2 21 1 2 21 1 2 21 1 2 2
1 1 1* 1 2 1 =
1 1 1
![Page 56: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/56.jpg)
Example: Two-Dimensional Convolution
1 2 4 5 4 22 5 9 12 10 43 7 13 17 14 63 7 13 17 14 62 5 9 12 10 41 2 4 5 4 2
1 1 2 21 1 2 21 1 2 21 1 2 2
1 1 1* 1 2 1 =
1 1 1
![Page 57: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/57.jpg)
Properties of Convolution• Commutative: f * g = g * f• Associative: f * (g * h) = (f * g) * h• Distributive over addition: f * (g + h) = f * g + f * h
• Derivative:
Convolution has the same mathematical properties as multiplication(This is no coincidence)
( )d f g f g f gdt
′ ′∗ = ∗ + ∗
![Page 58: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/58.jpg)
Useful Functions• Square: Πa(t)
• Triangle: Λa(t)
• Gaussian: G(t, s)
• Step: u(t)
• Impulse/Delta: δ (t)
• Comb (Shah Function): combh(t)
Each has their two-dimensional equivalent.
![Page 59: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/59.jpg)
Square
What does f(t) * Πa(t) do to a signal f(t)?What is Πa(t) * Πa(t)?
-a a
11 if ( )
0 otherwisea
t at
⎧ ≤⎪Π = ⎨⎪⎩
![Page 60: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/60.jpg)
Triangle
-a a
11 if
( )0 otherwise
a
t t aat⎧ − ≤⎪Λ = ⎨⎪⎩
![Page 61: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/61.jpg)
GaussianGaussian: maximum value = 1
Normalized Gaussian: area = 1
Convolving a Gaussian with another:
-σ σ
1
-σ σ
1
222( , )
tG t e σσ
−=
2221( , )
2
tG t e σσ
πσ−
=
),(),(),( 22
2121 σσσσ +=∗ tGtGtG
![Page 62: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/62.jpg)
Step Function
What is the derivative of a step function?
1
⎩⎨⎧ ≥
=otherwise 0
0 if 1)(
ttu
![Page 63: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/63.jpg)
Impulse/Delta Function• We’ve seen the delta function before:
• Shifted Delta function: impulse at t = k
• What is a function f(t) convolved with δ (t)?• What is a function f(t) convolved with δ (t - k)?
0
k0
∫∞
∞−
=⎩⎨⎧ =∞
= 1)( and otherwise 0
0 if )( dtt
tt δδ
⎩⎨⎧ =∞
=−otherwise 0
if )(
ktktδ
![Page 64: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/64.jpg)
Comb (Shah) FunctionA set of equally-spaced impulses: also called an impulse train
h is the spacingWhat is f(t) * combh(t)?
-2h h-h 2h 3h0-3h
∑ −=k
h hkttcomb )()( δ
![Page 65: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/65.jpg)
Convolution Filtering• Convolution is useful for modeling the behavior of
linear, shift-invariant devices• It is also useful to do ourselves to produce a desired
effect• When we do it ourselves, we get to choose the function
that the input will be convolved with• This function that is convolved with the input is called
the convolution kernel
![Page 66: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/66.jpg)
Convolution Filtering: Averaging
Can use a square function (“box filter”) or Gaussian to locally average the signal/image– Square (box) function: uniform averaging– Gaussian: center-weighted averaging
Both of these blur the signal or image
![Page 67: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/67.jpg)
Convolution Filtering: Unsharp Masking
To sharpen a signal/image, subtract a little bit of the blurred input:
I(x, y) + α [I(x, y) – I(x, y) * G(x, y, σ)]
This is called unsharp masking
The larger you make α, the more sharpening you get
More on filters in later sessions!
![Page 68: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/68.jpg)
Text Credits
The text for this lecture was adapted in part from the following source, in agreement with the listed copyright statements:
http://web.engr.oregonstate.edu/~enm/cs519/index.html© 2003 School of Electrical Engineering and Computer Science, Oregon State University, Dearborn Hall, Corvallis, Oregon, 97331
![Page 69: Linear System Theory, Complex Numbers, · PDF fileLinear System Theory, Complex Numbers, ... If some operation on a signal commutes with a ... • What constant c can represent complex](https://reader035.fdocuments.us/reader035/viewer/2022081514/5a6ff3a87f8b9aa7538b88b1/html5/thumbnails/69.jpg)
Resources
Textbooks:Kenneth R. Castleman, Digital Image Processing, Chapter 9