Systems Prof. Seungchul Lee Industrial AI Lab. Most slides from - - PowerPoint PPT Presentation

▶
systems
SMART_READER_LITE
LIVE PREVIEW

Systems Prof. Seungchul Lee Industrial AI Lab. Most slides from - - PowerPoint PPT Presentation

Jan 21, 2023 282 likes •1.06k views

Systems Prof. Seungchul Lee Industrial AI Lab. Most slides from (edX) Discrete Time Signals and Systems by Prof. Richard G. Baraniuk 1 Systems A discrete-time system is a transformation (a rule or formula) that maps a discrete- time

slide-1
SLIDE 1

Systems

  • Prof. Seungchul Lee

Industrial AI Lab.

1

Most slides from (edX) Discrete Time Signals and Systems by Prof. Richard G. Baraniuk

slide-2
SLIDE 2

Systems

  • A discrete-time system 𝐼 is a transformation (a rule or formula) that maps a discrete-

time input signal 𝑦 into a discrete-time output signal 𝑧

2

slide-3
SLIDE 3

Example: Systems

3

slide-4
SLIDE 4

Linear Systems

  • A system 𝐼 is linear if it satisfies the following two properties:

– Scaling: – Additivity:

4

slide-5
SLIDE 5

Linear Systems and Matrix Multiplication

  • Matrix multiplication (aka linear combination) is a fundamental signal processing system
  • Matrix multiplications are linear systems

where ℎ𝑜,𝑛 = 𝐼 𝑜,𝑛 represents the row-𝑜, column-𝑛 entry of the matrix 𝐼

  • All linear systems can be expressed as matrix multiplications

5

slide-6
SLIDE 6

Matrix Multiplication and Linear Systems in Pictures

  • System output as a linear combination of columns
  • System output as a sequence of inner products

6

slide-7
SLIDE 7

Time-Invariant Systems

  • For infinite-length signals

– A system 𝐼 processing infinite-length signals is time-invariant (shift-invariant) if a time shift of the input signal creates a corresponding time shift in the output signal

  • For finite-length signals

– A system 𝐼 processing infinite-length signals is time-invariant (shift-invariant) if a circular time shift of the input signal creates a corresponding circular time shift in the output signal

7

slide-8
SLIDE 8

Linear Time-Invariant (LTI) Systems

  • A system 𝐼 is linear time-invariant (LTI) if it is both linear and time-invariant
  • We will only consider Linear Time-Invariant (LTI) systems.

8

slide-9
SLIDE 9

Matrix Multiplication and LTI Systems (Infinite-Length Signals)

  • When the linear system is also shift invariant, 𝐼 has a special structure
  • Linear system for infinite-length signals can be expressed as
  • Enforcing time invariance implies that for all 𝑟 ∈ ℤ
  • Change of variables: 𝑜′ = 𝑜 − 𝑟 and 𝑛′ = 𝑛 − 𝑟

9

slide-10
SLIDE 10

Matrix Multiplication and LTI Systems (Infinite-Length Signals)

  • We see that for an LTI system
  • Entries on the matrix diagonals are the same - Toeplitz matrix

10

slide-11
SLIDE 11

Matrix Multiplication and LTI Systems (Infinite-Length Signals)

  • All of the entries in a Toeplitz matrix can be expressed in terms of the entries of the

– 0-th column – Time-reversed 0-th row

  • Row-𝑜, column-𝑛 entry of the matrix

11

slide-12
SLIDE 12

Matrix Multiplication and LTI Systems (Infinite-Length Signals)

  • Note the diagonals !

12

slide-13
SLIDE 13

Matrix Multiplication and LTI Systems (Finite-Length Signals)

  • Linear system for signals of length 𝑂 can be expressed as
  • Enforcing time invariance implies that for all 𝑟 ∈ ℤ
  • Change of variables: 𝑜′ = 𝑜 − 𝑟 and 𝑛′ = 𝑛 − 𝑟

13

slide-14
SLIDE 14

Matrix Multiplication and LTI Systems (Finite-Length Signals)

  • We see that for an LTI system
  • Entries on the matrix diagonals are the same + circular wraparound - Circulent matrix

14

slide-15
SLIDE 15

Matrix Multiplication and LTI Systems (Finite-Length Signals)

  • All of the entries in a circulent matrix can be expressed in terms of the entries of the

– 0-th column – Time-reversed 0-th row

  • Row-𝑜, column-𝑛 entry of the matrix

15

slide-16
SLIDE 16

Matrix Multiplication and LTI Systems (Finite-Length Signals)

  • Note the diagonals and circulent shifts !

16

slide-17
SLIDE 17

Impulse Response

17

slide-18
SLIDE 18

Impulse Response

  • We will Illustrate it with an infinite-length signal
  • The 0-th column of the matrix 𝐼 has a special interpretation
  • Compute the output when the input is a delta function (impulse)
  • This suggests that we call ℎ the impulse response of the system
  • Output of system to delta function (impulse) is ℎ. So, We call ℎ the impulse response of the system

18

slide-19
SLIDE 19

Impulse Response

  • From ℎ, we can build matrix 𝐼

– Columns/rows of 𝐼 are the shifted versions of the 0-th column/row – ℎ contains all the information of the LTI system

  • LTI systems are Toeplitz matrices (infinite-length signals)

– Entries on the matrix diagonals are the same

  • Let the input 𝜀[𝑜] and compute 𝑧[𝑜]
  • The impulse response characterizes an LTI system (that is, carries all of the information contained in

matrix 𝐼)

19

slide-20
SLIDE 20

Examples (Infinite-Length Signals)

  • Impulse response of the scaling system

20

slide-21
SLIDE 21

Examples (Infinite-Length Signals)

  • Impulse response of the shift system

21

slide-22
SLIDE 22

Examples (Infinite-Length Signals)

  • Impulse response of the moving average system

22

slide-23
SLIDE 23

Examples (Infinite-Length Signals)

  • Impulse response of the recursive average system

23

slide-24
SLIDE 24

Examples (Finite-Length Signals)

  • Entries on the matrix diagonals are the same + circular wraparound

24

slide-25
SLIDE 25

Examples (Finite-Length Signals)

  • Impulse response of the shift system

25

slide-26
SLIDE 26

Examples (Finite-Length Signals)

  • Impulse response of the moving average system

26

slide-27
SLIDE 27

Examples (Finite-Length Signals)

  • Impulse response of the recursive average system

27

slide-28
SLIDE 28

Time Response to Arbitrary Input: Convolution

28

slide-29
SLIDE 29

Convolution

  • Convolution is defined as the integral of the product of the two signals after one is reversed and

shifted

  • Output 𝑧[𝑜] came out by convolution of input 𝑦[𝑜] and system ℎ[𝑜]

– Time reverse the impulse response ℎ and shift it 𝑜 time steps to the right (delay) – Compute the inner product between the shifted impulse response and the input vector 𝑦

29

slide-30
SLIDE 30

Convolution For Infinite-Length Signals

  • Toeplitz Matrices
  • It is an inner product of ℎ vectors and 𝑦

30

slide-31
SLIDE 31

Convolution For Infinite-Length Signals

  • Convolution is product of matrix 𝐼 and 𝑦

31

slide-32
SLIDE 32

Convolution using Toeplitz Matrix

  • LTI systems are Toeplitz matrices (infinite-length signals)

– Entries on the matrix diagonals are the same

32

slide-33
SLIDE 33

Superposition (Linear) and Time-Invariant

  • Think about convolution in time

– Break input into additive parts and sum the responses to the parts

Source: Prof. Denny Freeman at MIT 33

slide-34
SLIDE 34

Superposition (Linear) and Time-Invariant

  • Think about convolution in time

– You are standing at time 𝑜

Source: Prof. Denny Freeman at MIT 34

slide-35
SLIDE 35

Convolution

  • If a system is linear and time-invariant (LTI) then its output is the sum of weighted and shifted unit-

sample responses.

Source: Prof. Denny Freeman at MIT 35

slide-36
SLIDE 36

1D Convolution

Source: Dr. Francois Fleuret at EPFL

1 3 2 3

  • 1

1 2 2 1

Input

36

slide-37
SLIDE 37

1D Convolution

Source: Dr. Francois Fleuret at EPFL

1 3 2 3

  • 1

1 2 2 1 1 3

  • 1

Input h[-n] Output

37

slide-38
SLIDE 38

1D Convolution

Source: Dr. Francois Fleuret at EPFL

1 3 2 3

  • 1

1 2 2 1 1 3

  • 1

Input h[-n] Output

7

38

slide-39
SLIDE 39

1D Convolution

Source: Dr. Francois Fleuret at EPFL

1 3 2 3

  • 1

1 2 2 1 1 3

  • 1

Input h[-n] Output

7 9

39

slide-40
SLIDE 40

1D Convolution

Source: Dr. Francois Fleuret at EPFL

1 3 2 3

  • 1

1 2 2 1 1 3

  • 1

Input h[-n] Output

7 9 12

40

slide-41
SLIDE 41

1D Convolution

Source: Dr. Francois Fleuret at EPFL

1 3 2 3

  • 1

1 2 2 1 1 3

  • 1

Input h[-n] Output

7 9 12 2

41

slide-42
SLIDE 42

1D Convolution

Source: Dr. Francois Fleuret at EPFL

1 3 2 3

  • 1

1 2 2 1 1 3

  • 1

Input h[-n] Output

7 9 12 2

  • 1

42

slide-43
SLIDE 43

1D Convolution

Source: Dr. Francois Fleuret at EPFL

1 3 2 3

  • 1

1 2 2 1 1 3

  • 1

Input h[-n] Output

7 9 12 2

  • 1

43

slide-44
SLIDE 44

1D Convolution

Source: Dr. Francois Fleuret at EPFL

1 3 2 3

  • 1

1 2 2 1 1 3

  • 1

w Input h[-n] Output

7 9 12 2

  • 1

6

44

slide-45
SLIDE 45

Structure of Convolution

Source: Prof. Denny Freeman at MIT 45

slide-46
SLIDE 46

Structure of Convolution

Source: Prof. Denny Freeman at MIT 46

slide-47
SLIDE 47

Structure of Convolution

Source: Prof. Denny Freeman at MIT 47

slide-48
SLIDE 48

Structure of Convolution

Source: Prof. Denny Freeman at MIT 48

slide-49
SLIDE 49

Structure of Convolution

Source: Prof. Denny Freeman at MIT 49

slide-50
SLIDE 50

Structure of Convolution

Source: Prof. Denny Freeman at MIT 50

slide-51
SLIDE 51

Structure of Convolution

Source: Prof. Denny Freeman at MIT 51

slide-52
SLIDE 52

Structure of Convolution

Source: Prof. Denny Freeman at MIT 52

slide-53
SLIDE 53

Structure of Convolution

Source: Prof. Denny Freeman at MIT 53

slide-54
SLIDE 54

Structure of Convolution

Source: Prof. Denny Freeman at MIT 54

slide-55
SLIDE 55

Structure of Convolution

Source: Prof. Denny Freeman at MIT 55

slide-56
SLIDE 56

Structure of Convolution

Source: Prof. Denny Freeman at MIT 56

slide-57
SLIDE 57

Structure of Convolution

Source: Prof. Denny Freeman at MIT 57

slide-58
SLIDE 58

Structure of Convolution

Source: Prof. Denny Freeman at MIT 58

slide-59
SLIDE 59

Graphical Illustration

Source: Applied Digital Signal Processing, Theory and Practice 59

slide-60
SLIDE 60

Discrete-Time Convolution: Summary

  • Representing an LTI system by a single signal
  • Unit-impulse response ℎ[𝑜] is a complete description of an LTI system
  • Given ℎ[𝑜], one can compute the response to any arbitrary input signal 𝑦[𝑜]

60

slide-61
SLIDE 61

Convolution: Commutative

  • Convolution is commutative

→ Signal = System

61

slide-62
SLIDE 62

Convolution in MATLAB

  • For finite-length signals

62

slide-63
SLIDE 63

Convolution in MATLAB

  • For finite-length signals

63

slide-64
SLIDE 64

Convolution Function

  • You have to include conv_m function file in the path

64

slide-65
SLIDE 65

Convolution in MATLAB

65

slide-66
SLIDE 66

Convolution For Finite-Length Signals

  • Circular convolution

– Circularly time reverse the impulse response ℎ and circularly shift it 𝑜 time steps to the right (delay) – Compute the inner product between the shifted impulse response and the input vector 𝑦

66

slide-67
SLIDE 67

Circular Convolution in MATLAB

67

slide-68
SLIDE 68

Convolution For Finite-Length Signals

  • Think about circular convolution in time

– Circularly time reverse the impulse response ℎ and circularly shift it 𝑜 time steps to the right (delay) – Compute the inner product between the shifted impulse response and the input vector 𝑦

68

slide-69
SLIDE 69

Convolution For Finite-Length Signals

  • Think about circular convolution in time

– Circularly time reverse the impulse response ℎ and circularly shift it 𝑜 time steps to the right (delay) – Compute the inner product between the shifted impulse response and the input vector 𝑦

69

slide-70
SLIDE 70

Circular Convolution in MATLAB

70

slide-71
SLIDE 71

Convolution Examples

71

slide-72
SLIDE 72

De-noising

72

slide-73
SLIDE 73

Edge Detection

73

slide-74
SLIDE 74

Smoothing and Detection of Abrupt Changes

74

slide-75
SLIDE 75

Example: Convolution on Audio

Source: Prof. Allen Downey at Olin 75

slide-76
SLIDE 76

Example: Convolution on Audio

Source: Prof. Allen Downey at Olin 76

slide-77
SLIDE 77

Example: Convolution on Audio

Source: Prof. Allen Downey at Olin 77