CTP431- Music and Audio Computing Audio Signal Processing (Part #1) - - PowerPoint PPT Presentation

CTP431- Music and Audio Computing Audio Signal Processing (Part #1)

Graduate School of Culture Technology KAIST Juhan Nam

1

Types of Audio Signal Processing

§ Amplitude

– Gain, fade in/out, automation curve, compressor

§ Timbre

– EQ, distortion, modulation, chorus, flanger

§ Spatial effect

– Delay, reverberation

§ Pitch

– Pitch shifting (e.g. auto-tune)

§ Duration

– Time stretching

§ Playback Rate Conversion (resampling)

– pitch-shifting /time-stretching / timbre change

2

Filter

§ A system that takes signals and modifies them in some way § In particular, we are interested in digital filters that takes discrete number sequences as input and output

3

Input Output Filter 𝑦 𝑜 𝑧 𝑜

Basic Operations in Digital Filters

§ Multiplication: 𝑧 𝑜 = 𝑐& ' 𝑦 𝑜 § Delaying: 𝑧 𝑜 = 𝑦 𝑜 − 1 § Summation: 𝑧 𝑜 = 𝑦 𝑜 + 𝑦 𝑜 − 1

4

Input Output Filter 𝑦 𝑜 𝑧 𝑜

Linear Time-Invariant (LTI) Digital Filters

§ Linearity

– Homogeneity: if 𝑦 𝑜 → 𝑧 𝑜 , then a ' 𝑦 𝑜 → a ' 𝑧 𝑜 – Superposition: if 𝑦- 𝑜 → 𝑧- 𝑜 and 𝑦. 𝑜 → 𝑧. (n), then 𝑦- 𝑜 + 𝑦. 𝑜 → 𝑧- 𝑜 + 𝑧. 𝑜

§ Time-Invariance

– If 𝑦 𝑜 → 𝑧 𝑜 , then 𝑦 𝑜 − 𝑂 → 𝑧 𝑜 − 𝑂 for any 𝑂 – This means that the system does not change its behavior over time

5

Input Output Filter 𝑦 𝑜 𝑧 𝑜

LTI Digital Filters

§ A LTI digital filters performs a combination of the three operations

– 𝑧 𝑜 = 𝑐& ' 𝑦 𝑜 + 𝑐- ' 𝑦 𝑜 − 1 + 𝑐. ' 𝑦 𝑜 − 2 + ⋯ + 𝑐2 ' 𝑦 𝑜 − 𝑁

§ This is a general form of Finite Impulse Response (FIR) filter

6

The Simplest Lowpass Filter

7

𝑧 𝑜 = 𝑦 𝑜 + 𝑦(𝑜 − 1) 𝑨7- 𝑦 𝑜 𝑧 𝑜

+

§ Difference equation § Signal flow graph

“Delay Operator”

The Simplest Lowpass Filter: Sine-Wave Analysis

§ Measure the amplitude and phase changes given a sinusoidal signal input

8

The Simplest Lowpass Filter: Frequency Response

§ Plot the amplitude and phase change over different frequency

– The frequency sweeps from 0 to the Nyquist rate

9

The Simplest Lowpass Filter: Frequency Response

§ Mathematical approach

– Use complex sinusoid as input: 𝑦 𝑜 = 𝑓9:; – Then, the output is: 𝑧 𝑜 = 𝑦 𝑜 + 𝑦 𝑜 − 1 = 𝑓9:; + 𝑓9:(;7-) = 1 + 𝑓79: ' 𝑓9:; = 1 + 𝑓79: ' 𝑦(𝑜) – Frequency response: 𝐼 𝜕 = 1 + 𝑓79: = 𝑓9>

? + 𝑓79> ? 𝑓79> ? = 2cos

(

: .)𝑓79>

?

– Amplitude response: 𝐼(𝜕) = 2 cos

: .

– Phase response: ∠𝐼 𝜕 = −

: .

10

The Simplest Highpass Filter

§ Difference equation: 𝑧 𝑜 = 𝑦 𝑜 − 𝑦(𝑜 − 1) § Frequency response

11

Impulse Response

§ The filter output when the input is a unit impulse

– 𝑦 𝑜 = 𝜀 𝑜 = 1, 0, 0, 0, … → 𝑧 𝑜 = ℎ(𝑜)

§ Characterizes the digital system as a sequence of numbers

– A system is represented just like audio samples!

12

Input Output Filter 𝑦 𝑜 = 𝜀 𝑜 𝑧 𝑜 = ℎ(𝑜) ℎ 𝑜

Examples: Impulse Response

§ The simplest lowpass filter

– h 𝑜 = 1, 1

§ The simplest highpass filter

– h 𝑜 = 1, −1

§ Moving-average filter (order=5)

– h 𝑜 =

• J ,
• J ,
• J ,
• J ,
• J

§ General FIR Filter

– h 𝑜 = 𝑐&, 𝑐-, 𝑐., … , 𝑐2 à A finite length of impulse response

13

§ The output of LTI digital filters is represented by convolution operation between 𝑦 𝑜 and ℎ 𝑜 § Deriving convolution

– The input can be represented as a time-ordered set of weighted impulses

• 𝑦 𝑜 = 𝑦&, 𝑦-, 𝑦., … , 𝑦2 = 𝑦& ' 𝜀 𝑜 + 𝑦- ' 𝜀 𝑜 − 1 + 𝑦. ' 𝜀 𝑜 − 2 + ⋯ + 𝑦2 ' 𝜀 𝑜 − 𝑁

– By the linearity and time-invariance

• 𝑧 𝑜 = 𝑦& ' ℎ 𝑜 + 𝑦- ' ℎ 𝑜 − 1 + 𝑦. ' ℎ 𝑜 − 2 + ⋯ + 𝑦2 ' ℎ 𝑜 − 𝑁 = ∑

𝑦(𝑗) ' ℎ(𝑜 − 𝑗)

2 MN&

Convolution

14

𝑧 𝑜 = 𝑦 𝑜 ∗ ℎ 𝑜 = P 𝑦(𝑗) ' ℎ(𝑜 − 𝑗)

2 MN&

Convolution In Practice

§ The practical expression of convolution

– This represents input 𝑦 𝑜 as a streaming data to the filter ℎ 𝑜

§ Matlab Animation Demo

– http://mac.kaist.ac.kr/~juhan/ctp431/convolution_demo.html

§ The length of convolution output

– If the length of 𝑦 𝑜 is M and the length of ℎ 𝑜 is N, the length of 𝑧 𝑜 is M+N-1

15

𝑧 𝑜 = 𝑦 𝑜 ∗ ℎ 𝑜 = P 𝑦 𝑗 ' ℎ 𝑜 − 𝑗

2 MN&

= P ℎ(𝑗) ' 𝑦(𝑜 − 𝑗)

2 MN&

Feedback Filter

§ LTI digital filters allow to use the past outputs as input

– Past outputs: 𝑧 𝑜 − 1 , 𝑧 𝑜 − 2 , … , 𝑧 𝑜 − 𝑂

§ The whole system can be represented as

– 𝑧 𝑜 = 𝑐& ' 𝑦 𝑜 + 𝑏- ' 𝑧 𝑜 − 1 + 𝑏. ' 𝑧 𝑜 − 2 + ⋯ + 𝑏R ' 𝑧 𝑜 − 𝑂 – This is a general form of Infinite Impulse Response (IIR) filter

16

Input Output Filter

+

𝑦 𝑜 𝑧 𝑜 Delay

§ Difference equation § Signal flow graph

– When 𝑏 is slightly less than 1, it is called “Leaky Integrator”

A Simple Feedback Lowpass Filter

17

𝑧 𝑜 = 𝑦 𝑜 + 𝑏 ' 𝑧(𝑜 − 1) 𝑨7- 𝑦 𝑜 𝑧 𝑜

+

𝑏

A Simple Feedback Lowpass Filter: Impulse Response

§ Impulse response

– 𝑧 0 = 𝑦 0 = 1 – 𝑧 1 = 𝑦 1 + 𝑏 ' 𝑧 0 = 𝑏 – 𝑧 2 = 𝑦 2 + 𝑏 ' 𝑧 1 = 𝑏. – … – 𝑧 𝑜 = 𝑦 𝑜 + 𝑏 ' 𝑧 𝑜 − 1 = 𝑏;

§ Stability!

– If 𝑏 < 1, the filter output converges (stable) – If 𝑏 = 1, the filter output oscillates (critical) – If 𝑏 > 1, the filter output diverges (unstable)

18

A Simple Feedback Lowpass Filter: Frequency Response

§ More dramatic change than the simplest lowpass filter (FIR)

– Phase response is not linear

19

𝑧 𝑜 = 𝑦 𝑜 + 0.9 ' 𝑧(𝑜 − 1)

Reson Filter

20

§ Difference equation § Signal flow graph 𝑧 𝑜 = 𝑦 𝑜 + 2𝑠 ' cos𝜄 ' 𝑧 𝑜 − 1 − 𝑠. ' 𝑧 𝑜 − 2 𝑨7- 𝑦 𝑜 𝑧 𝑜

+

2𝑠 ' cos𝜄

+

𝑨7-

−𝑠.

+

Reson Filter: Frequency Response

§ Generate resonance at a particular frequency

– Control the peak height by 𝑠 and the peak frequency by 𝜄

21

For stability: 𝑠 < 1

§ Difference equation § Signal flow graph

Comb Filter

22

𝑧 𝑜 = 𝑦 𝑜 + 𝑦 𝑜 − 𝑁 𝑨72 𝑦 𝑜 𝑧 𝑜

+

Feedforward

𝑧 𝑜 = 𝑦 𝑜 + 𝑏 ' 𝑧(𝑜 − 𝑂)

(Feedforward) (Feedback)

𝑨7R 𝑦 𝑜 𝑧 𝑜

+

𝑏

Feedback

Comb Filter: Frequency Response

§ ”Combs” become shaper in the feedback type

23

𝑧 𝑜 = 𝑦 𝑜 + 𝑦(𝑜 − 8) 𝑧 𝑜 = 𝑦 𝑜 + 0.9 ' 𝑧(𝑜 − 8)

Perception of Time Delay

§ The 30 Hz transition

– Given a repeated click sound (e.g. impulse train):

• If the rate is less than 30Hz, they are perceived as discrete events.
• As the rate is above 30 Hz, they are perceive as a tone

– Demo: http://auditoryneuroscience.com/?q=pitch/click_train

§ Feedback comb filter: 𝑧 𝑜 = 𝑦 𝑜 + 𝑏 ' 𝑧(𝑜 − 𝑂)

– If N <

[\ ]& (𝐺 _: sampling rate): models sound propagation and reflection with energy

loss on a string (Karplus-strong model) – If N <

[\ ]& (𝐺 _: sampling rate): generate a looped delay

24

General Filter Form

§ The general form of digital Filters

– 𝑧 𝑜 = 𝑐& ' 𝑦 𝑜 + 𝑐- ' 𝑦 𝑜 − 1 + 𝑐. ' 𝑦 𝑜 − 2 + … + 𝑐R ' 𝑦 𝑜 − 𝑁 +𝑏- ' 𝑧 𝑜 − 1 + 𝑏. ' 𝑧 𝑜 − 2 + ⋯ + 𝑏R ' 𝑧 𝑜 − 𝑂

25

Z-1 x(n)

+

Z-1 Z-1 . . . b1 b2 bM b0 x(n-1) x(n-2) x(n-M) Z-1 y(n) Z-1 Z-1 . . .

• a1
• a2
• aN

y(n-1) y(n-2) y(n-N)

Frequency Response

§ Sine-wave Analysis

– 𝑦 𝑜 = 𝑓9:; à 𝑦 𝑜 − 𝑛 = 𝑓9:(;7a) = 𝑓79:a𝑦 𝑜 for any 𝑛 – Let’s assume that 𝑧 𝑜 = 𝐻 𝜕 𝑓9(:;cd : ) à 𝑧 𝑜 − 𝑛 = 𝑓79:a𝑧 𝑜 for any 𝑛

§ Putting this into the different equation

26

𝑧 𝑜 = 𝑐& + 𝑐- ' 𝑓79: + 𝑐. ' 𝑓79.: + … + 𝑐2 ' 𝑓79:2 1 + 𝑏- ' 𝑓79: + 𝑏. ' 𝑓79.: + … + 𝑏R ' 𝑓79:R 𝑦(𝑜) 𝑧 𝑜 = 𝑐& ' 𝑦 𝑜 + 𝑐- ' 𝑓79: ' 𝑦 𝑜 + 𝑐. ' 𝑓79.: ' 𝑦 𝑜 + … + 𝑐2 ' 𝑓79:2 ' 𝑦 𝑜 +𝑏- ' 𝑓79: ' 𝑧 𝑜 + 𝑏. ' 𝑓79.: ' 𝑧 𝑜 + ⋯ + 𝑏R ' 𝑓79:R ' 𝑧 𝑜

𝐼(𝜕) = 𝑐& + 𝑐- ' 𝑓79: + 𝑐. ' 𝑓79.: + …+ 𝑐2 ' 𝑓79:2 1 + 𝑏- ' 𝑓79: + 𝑏. ' 𝑓79.: + …+ 𝑏R ' 𝑓79:R

𝐼(𝜕) : frequency response 𝐻 𝜕 = 𝐼(𝜕) : amplitude response 𝜄 𝜕 = ∠𝐼(𝜕) : phase response

Z-Transform

§ 𝑎-transform

– Define z to be a variable in complex plane: we call it z-plane – When z = ejω (on unit circle), the frequency response is a particular case of the following form – We call this 𝑨-transform or the transfer function of the filter – z-1 corresponds to one sample delay: delay operator or delay element – Filters are often expressed as 𝑨-transform: polynomial of 𝑨7-

27

𝐼 𝑨 = 𝐶(𝑨) 𝐵(𝑨) = 𝑐& + 𝑐- ' 𝑨7- + 𝑐. ' 𝑨7. + …+ 𝑐2 ' 𝑨72 1 + 𝑏- ' 𝑨7- + 𝑏. ' 𝑨7. + …+ 𝑏R ' 𝑨7R

Z-Transform

§ Poles and Zeros

– H(z) can be factorized and we can find roots for each of polynomials – Zeros: the numerator roots – Poles: the denominator roots

§ We can analyze frequency response of filters more easily with poles and zeros than numerator or denominator coefficient

28

H(z) = B(z) A(z) = (1− q1z−1)(1− q2z−1)(1− q3z−1)...(1− qMz−1) (1− p1z−1)(1− p2z−1)(1− p3z−1)...(1− pNz−1)

Practical Filters

§ One-pole one-zero filters

– Leaky integrator – Moving average – DC-removal filters – Bass / treble shelving filter

§ Biquad filters

– Reson filter – Band-pass / notch filters – Equalizer: a set of biquad filters

§ Any high-order filter can be factored into a combination of one-pole one- zero filters or bi-quad filters!

29

H(z) = b0 + b

1z−1

a0 + a1z−1 H(z) = b0 + b

1z−1 + b2z−2

a0 + a1z−1 + a2z−2