Viterbi Algorithm Saravanan Vijayakumaran sarva@ee.iitb.ac.in - - PowerPoint PPT Presentation

▶
viterbi algorithm
SMART_READER_LITE
LIVE PREVIEW

Viterbi Algorithm Saravanan Vijayakumaran sarva@ee.iitb.ac.in - - PowerPoint PPT Presentation

Oct 02, 2022 916 likes •1.27k views

Viterbi Algorithm Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay October 30, 2015 1 / 33 Encoder State Diagram 1 + D 2 1 + D + D 2 G ( D ) = 1 + D 0/000 v ( 0 )

slide-1
SLIDE 1

Viterbi Algorithm

Saravanan Vijayakumaran sarva@ee.iitb.ac.in

Department of Electrical Engineering Indian Institute of Technology Bombay

October 30, 2015

1 / 33

slide-2
SLIDE 2

Encoder State Diagram

G(D) =

  • 1 + D

1 + D2 1 + D + D2

+ + + u v(0) v(1) v(2)

S0 S1 S3 S2 1/111 0/000 1/010 0/101 0/110 1/001 1/100 0/011

2 / 33

slide-3
SLIDE 3

Encoder Trellis Diagram

S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S1 S3 S3 S3 S3 S3

· · · · · · · · · · · · 1 2 3 4 5 6 t =

000 000 000 000 000 000 001 001 001 001 111 111 111 111 111 111 011 011 011 011 010 010 010 010 010 101 101 101 101 101 100 100 100 100 110 110 110 110

  • The initial state of the encoder is the all-zeros state
  • Every path in the trellis starting from the initial state gives a codeword

3 / 33

slide-4
SLIDE 4

Terminated Trellis Diagram

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

000 000 000 000 000 000 000 001 001 001 111 111 111 111 111 011 011 011 011 011 010 010 010 010 101 101 101 101 101 100 100 100 110 110 110 110

  • The inputs are chosen to terminate the trellis in the all-zeros state
  • Every path from the initial state to the final state is a codeword

4 / 33

slide-5
SLIDE 5

Terminating the Trellis of Feedforward Encoders

+ + + u v(0) v(1) v(2)

  • Two consecutive zero input bits will drive the above

encoder to the all-zeros state

  • In a feedforward encoder with memory order m, m

consecutive zero input bits in each of the k inputs will terminate the trellis

5 / 33

slide-6
SLIDE 6

Terminating the Trellis of Feedback Encoders

wi−1 wi−2 wi−m + + + + + + + + f0 f1 f2 fm−1 fm q1 q2 qm−1 qm

· · · · · · · · ·

ui wi vi

  • To reach the all-zeros state, the input to the shift register

has to be zero for m time units wi = 0 = ⇒ ui =

m

  • j=1

qjwi−j

6 / 33

slide-7
SLIDE 7

Maximum Likelihood Decoder for BSC

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

000 000 000 000 000 000 000 001 001 001 111 111 111 111 111 1 1 1 1 1 1 1 1 1 1 010 010 010 010 101 1 1 1 1 1 1 1 1 1 100 100 110 110 110 110

  • Let r =
  • 110

110 110 111 010 101 101

  • be the BSC output
  • The ML decoder will output a codeword v such that dH(r, v) is minimum

7 / 33

slide-8
SLIDE 8

Maximum Likelihood Decoder for BSC

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 2 2 3 1 2 2 3 2 2 1 1 1 2 2 1 1 2 2 1 1 2 2 2 1 3 1 2 2 1 1 2

  • Branch metric is the Hamming distance between the codeword bits of a

state transition and the corresponding received bits

  • Path metric is the sum of all the branch metrics along a path

8 / 33

slide-9
SLIDE 9

Viterbi Algorithm (1)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 3 3 4 2 2 2 3 1 2 2 3 2 2 1 1 1 2 2 1 1 2 2 1 1 2 2 2 1 3 1 2 2 1 1 2

  • At t = 2, there is only path entering each state
  • The shortest path has one of these four paths as a prefix

9 / 33

slide-10
SLIDE 10

Viterbi Algorithm (2)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 3 3 4 2 2 2 3 1 2 2 3 2 2 1 1 1 2 2 1 1 2 2 1 1 2 2 2 1 3 1 2 2 1 1 2

  • At t = 3, there are two paths entering each state
  • The shortest path has one of these eight paths as a prefix

10 / 33

slide-11
SLIDE 11

Viterbi Algorithm (3)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 3 3 4 2 2 2 3 1 2 2 3 2 2 1 1 1 2 2 1 1 2 2 1 1 2 2 2 1 3 1 2 2 1 1 2

  • At t = 3, the two paths entering state S3 have path metrics 5 and 4
  • We eliminate the path with metric 5

11 / 33

slide-12
SLIDE 12

Viterbi Algorithm (4)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 3 3 4 4 2 2 2 3 1 2 2 2 2 1 1 1 2 2 1 1 2 2 1 1 2 2 2 1 3 1 2 2 1 1 2

  • The state S3 at time t = 3 is now labelled with its partial path metric

12 / 33

slide-13
SLIDE 13

Viterbi Algorithm (5)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 3 3 4 4 2 2 2 3 1 2 2 2 2 1 1 1 2 2 1 1 2 2 1 1 2 2 2 1 3 1 2 2 1 1 2

  • At t = 3, the two paths entering state S1 have path metrics 4 and 5
  • We eliminate the path with metric 5

13 / 33

slide-14
SLIDE 14

Viterbi Algorithm (6)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 3 3 4 4 4 2 2 2 3 1 2 2 2 2 1 1 2 2 1 1 2 2 1 1 2 2 2 1 3 1 2 2 1 1 2

  • The state S1 at time t = 3 is now labelled with its partial path metric

14 / 33

slide-15
SLIDE 15

Viterbi Algorithm (7)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 3 3 4 4 4 2 2 2 3 1 2 2 2 2 1 1 2 2 1 1 2 2 1 1 2 2 2 1 3 1 2 2 1 1 2

  • At t = 3, the two paths entering state S2 have path metrics 2 and 5
  • We eliminate the path with metric 5

15 / 33

slide-16
SLIDE 16

Viterbi Algorithm (8)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 3 3 4 4 4 2 2 2 2 3 1 2 2 2 2 1 1 2 2 1 1 2 2 1 1 2 2 1 3 1 2 2 1 1 2

  • The state S2 at time t = 3 is now labelled with its partial path metric

16 / 33

slide-17
SLIDE 17

Viterbi Algorithm (9)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 3 3 4 4 4 2 2 2 2 3 1 2 2 2 2 1 1 2 2 1 1 2 2 1 1 2 2 1 3 1 2 2 1 1 2

  • At t = 3, the two paths entering state S0 have path metrics 5 and 6
  • We eliminate the path with metric 6

17 / 33

slide-18
SLIDE 18

Viterbi Algorithm (10)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

2 3 3 4 4 4 2 5 2 2 3 1 2 2 2 2 1 1 2 2 1 1 2 2 1 1 2 2 1 3 1 2 2 1 1 2

  • The state S0 at time t = 3 is now labelled with its partial path metric

18 / 33

slide-19
SLIDE 19

Viterbi Algorithm (11)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

4 4 2 5 2 3 1 2 2 2 2 1 1 2 2 1 1 2 2 1 1 2 2 1 3 1 2 2 1 1 2

  • At t = 3, there is now only one surviving path entering each state
  • The shortest path has one of these four paths as a prefix

19 / 33

slide-20
SLIDE 20

Viterbi Algorithm (12)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

4 4 2 5 6 2 3 1 2 2 2 1 1 2 2 1 1 2 2 1 1 2 2 1 3 1 2 2 1 1 2

  • We eliminate one of the paths with metric 6 entering S3 at t = 4

20 / 33

slide-21
SLIDE 21

Viterbi Algorithm (13)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

4 4 2 5 6 4 2 3 1 2 2 2 1 1 2 2 1 1 2 2 1 1 2 2 1 3 1 2 2 1 1 2

  • We eliminate the path with metric 5 entering S1 at t = 4

21 / 33

slide-22
SLIDE 22

Viterbi Algorithm (14)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

4 4 2 5 6 4 5 2 3 1 2 2 2 1 1 2 2 1 1 2 2 1 1 2 2 3 1 2 2 1 1 2

  • We eliminate one of the paths with metric 5 entering S2 at t = 4

22 / 33

slide-23
SLIDE 23

Viterbi Algorithm (15)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

4 4 2 5 6 4 5 3 2 1 2 2 2 1 1 2 2 1 1 2 2 1 1 2 2 3 1 2 2 1 1 2

  • We eliminate the path with metric 8 entering S0 at t = 4

23 / 33

slide-24
SLIDE 24

Viterbi Algorithm (16)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

6 4 5 3 2 1 2 2 2 1 1 2 1 1 2 2 1 1 2 2 3 1 2 2 1 1 2

  • At t = 4, there is now only one surviving path entering each state
  • The shortest path has one of these four paths as a prefix

24 / 33

slide-25
SLIDE 25

Viterbi Algorithm (17)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

6 4 5 3 4 5 7 4 2 1 2 2 1 1 2 1 2 2 1 1 2 2 1 2 1 1 2

  • Using the same procedure, we eliminate one path per state at t = 5

25 / 33

slide-26
SLIDE 26

Viterbi Algorithm (18)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

4 5 7 4 1 2 2 1 2 1 2 2 1 2 2 1 2 1 2

  • At t = 5, there is now only one surviving path entering each state
  • The shortest path has one of these four paths as a prefix

26 / 33

slide-27
SLIDE 27

Viterbi Algorithm (19)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

4 5 7 4 5 6 1 2 2 1 2 1 2 1 2 2 1 2 1

  • Using the same procedure, we eliminate one path per state at t = 6

27 / 33

slide-28
SLIDE 28

Viterbi Algorithm (20)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

5 6 1 2 2 1 2 1 2 1

  • At t = 6, there is now only one surviving path entering each state
  • The shortest path has one of these two paths as a prefix

28 / 33

slide-29
SLIDE 29

Viterbi Algorithm (21)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

5 6 1 2 1 2 1 2 1

  • We eliminate the path with metric 8 entering S0 at t = 7

29 / 33

slide-30
SLIDE 30

Viterbi Algorithm (22)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

7 1 2 1 2 1

  • A single path from state S0 at t = 0 to state S0 at t = 7 has survived

30 / 33

slide-31
SLIDE 31

Viterbi Algorithm (23)

S0 S0 S0 S0 S0 S0 S0 S0 S2 S2 S2 S2 S2 S1 S1 S1 S1 S1 S3 S3 S3 S3

1 2 3 4 5 6 7 t =

111 111 1 1 1 1 010 1 1 110

  • ˆ

v =

  • 111

010 110 011 111 101 011

  • is the output of the

Viterbi algorithm

31 / 33

slide-32
SLIDE 32

Viterbi Algorithm in General

  • Consider a convolutional code with k inputs, n outputs,

memory order m and constraint length ν

  • The trellis has at most 2ν states at each time instant
  • At t = m, there is one path entering each state
  • At t = m + 1, there are 2k paths entering each state, out of

which 2k − 1 have to be eliminated

  • At each time instant t, at most 2ν surviving paths exist
  • For a discrete memoryless channel with channel transition

probability function p(·|·), we maximize log p(r|v) =

h+m−1

  • i=0

log p(ri|vi) where h is the number of k-bit input blocks, ri and vi are n symbols long

32 / 33

slide-33
SLIDE 33

Questions? Takeaways?

33 / 33