[PPT] - The setting of the research ( s ) { 0 , 1 } 1 ( ( s )) = s s S PowerPoint Presentation

SLIDE 1

On a Redundancy of AIFV-m Codes for m = 3, 5

Ryusei Fujita†, Ken-ichi Iwata†, Hirosuke Yamamoto††

†University of Fukui ††The University of Tokyo

2020 IEEE International Symposium on Information Theory 21-26 June 2020

SLIDE 2

The setting of the research

S

Source

s ∈ S

source symbol ✲ ϕ Encoder ✲ ϕ(s) ∈ {0, 1}∗ noiseless channel

ϕ−1

Decoder allowing delay at most m bits ✲

ϕ−1(ϕ(s)) = s

uniquely decodable

S : stationary memoryless source with a probability distribution {ps, s ∈ S} Rϕ: redundancy of a code ϕ for a given source S Rϕ := Lϕ − H Lϕ :=

average codeword length of a code ϕ

H := −

s∈S

ps log2 ps

entropy of a source S We derive a new upper bound on the Redundancy of AIFV-m Codes for m = 3, 5.

2 / 44

SLIDE 3

A brief history of redundancy of the optimal FV codes

Evaluate the upper bounds on redundancy Rϕ when p1 := max

s∈S ps is known.

In 1978, Gallager [1] proved

RHuffman ≤        p1 + 0.0086

if p1 < 1

2,

2 − p1 − h(p1)

if p1 ≥ 1

2,

for Huffman codes,

h(x) := −x log2 x − (1 − x) log2(1 − x)

if 0 < x < 1. In 2017, Hu, Yamamoto, Honda [2] proposed AIFV-m codes, and proved

RAIFV−2 ≤            2 − 2p1 + p2

1 − h(p1)

if 1

2 ≤ p1 ≤ √ 5−1 2

, 2 + p1 − 2p2

1

1 + p1 − h(p1)

if

√ 5−1 2

≤ p1 < 1

for AIFV-2 codes,

RAIFV−m = 1/m

if m ∈ {2, 3, 4}. They conjectured that RAIFV−m ≤ 1/m for any positive integer m. This research evaluates an upper bound on RAIFV−3 of AIFV-3 when p1 is known, and proves the worst-case redundancy of AIFV-5 codes, RAIFV−5 = 1/5.

3 / 44

SLIDE 4

Upper Bounds on Redundancy RHuffman when p1 is known

RHuffman := LHuffman − H ≤ 1

1

1⁄2 1⁄10 2⁄10 3⁄10 4⁄10 1⁄2 6⁄10 7⁄10 8⁄10 9⁄10

1

p1 := max

s∈S ps

[1] R. Gallager, “Variations on a Theme by Huffman,” IEEE Trans. Inf. Theory, 1978. [6] D. Manstetten, “Tight Bounds on the Redundancy of Huffman Codes,” IEEE Trans. Inf. Theory, 1992. ✻

Gallager [1]

RHuffman ≤ 2 − p1 − h(p1)

if 1/2 ≤ p1 ≤ 1 ✻ Manstetten [6] if 1/127 ≤ p1 ≤ 1/2

4 / 44

SLIDE 5

Upper Bounds on Redundancy RAIFV−2 when p1 is known

Redundancy

1

1⁄2 1⁄4 1⁄2

√ 5−1 2

3⁄4

1

p1 := max

s∈S ps

✻ Gallager [1], RHuffman ≤ 2 − p1 − h(p1) Hu, et al. [2], RAIFV−2 [2] W. Hu, H. Yamamoto, and J. Honda, “Worst-case Redundancy of Optimal Binary AIFV Codes and Their Extended Codes,” IEEE Trans. Inf. Theory, 2017. Theorem [2, Theorem 4]

RAIFV−2 ≤            2 − 2p1 + p2

1 − h(p1)

if 1

2 ≤ p1 ≤ √ 5−1 2 2+p1−2p2

1

1+p1

− h(p1)

if

√ 5−1 2

≤ p1 < 1 ≤ 1/2

for 0 < p1 < 1 ❳❳❳ ✲ ❳❳ ❳ ✏✏✏✏ ✏ ✶

5 / 44

SLIDE 6

Upper Bounds on Redundancy RAIFV−3 when p1 is known

Redundancy

1⁄2 1⁄3 1⁄3 1⁄4 1⁄6 1⁄2

√ 5−1 2

β1 ≈ 0.6889

3⁄4

β2 ≈ 0.8287

1

RAIFV−2 RAIFV−3 p1

Theorem 3 in the paper [New]

RAIFV−3 ≤                                               

3−3p1+p2

1

2−p1

− h(p1)

if 1

2 ≤ p1 ≤ √ 5−1 2

,

4−p1−3p2

1+p3 1

(2−p1)(1+p1) − h(p1)

if

√ 5−1 2

< p1 ≤ β1,

3−p1−2p2

1+p3 1

1+p1

− h(p1)

if β1 ≤ p1 ≤ 3

4, 9−5p2

1

4(1+p1) − h(p1)

if 3

4 < p1 ≤ β2, 23+24p1−35p3

1

12(1+p1+p2

1) − h(p1)

if β2 ≤ p1 < 1,

6 / 44

SLIDE 7

The worst case redundancy RAIFV−3 of AIFV-3 codes

Redundancy

1⁄2 1⁄3 1⁄4 1⁄2

RAIFV−2

✻

f3(p1)

1

p1

Theorem [Hu et al, Theorem 7] If p1 ≥ 1/2, then

RAIFV−3 ≤ min{RAIFV−2, f3(p1)} ≤ 1/3. f3(p1) := 2 − p1 + −p13−2 p12−p1+2

p12+p1+1

− h(p1)

given by an AIFV-3 code:

7 / 44

SLIDE 8

The worst case redundancy RAIFV−3 of AIFV-3 codes

Redundancy

1⁄2 1⁄3 1⁄4 1⁄2

RAIFV−2

✻

f3(p1)

1

p1

Theorem [Hu et al, Theorem 7] If p1 ≥ 1/2, then

RAIFV−3 ≤ min{RAIFV−2, f3(p1)} ≤ 1/3.

Theorem [Hu et al, Theorem 5] If p1 ≤ 1/2, then

RAIFV−2 ≤ 1/4,

and RAIFV−3 ≤ RAIFV−2 ≤ 1/4 ≤ 1/3.

8 / 44

SLIDE 9

The worst case redundancy RAIFV−3 of AIFV-3 codes

Redundancy

1⁄2 1⁄3 1⁄4 1⁄2

RAIFV−2

✻

f3(p1)

1

p1

Theorem [Hu et al, Theorem 7] If p1 ≥ 1/2, then

RAIFV−3 ≤ min{RAIFV−2, f3(p1)} ≤ 1/3.

Theorem [Hu et al, Theorem 5] If p1 ≤ 1/2, then

RAIFV−2 ≤ 1/4,

and RAIFV−3 ≤ RAIFV−2 ≤ 1/4 ≤ 1/3.

RAIFV−3 ≤ 1/3

for 0 < p1 < 1

9 / 44

SLIDE 10

The worst case redundancy RAIFV−3 of AIFV-3 codes

Redundancy

1⁄2 1⁄3 1⁄4 1⁄2

RAIFV−2

✻

f3(p1)

1

p1

Theorem [Hu et al, Theorem 7] If p1 ≥ 1/2, then

RAIFV−3 ≤ min{RAIFV−2, f3(p1)} ≤ 1/3.

Theorem [Hu et al, Theorem 5] If p1 ≤ 1/2, then

RAIFV−2 ≤ 1/4,

and RAIFV−3 ≤ RAIFV−2 ≤ 1/4 ≤ 1/3.

RAIFV−3 ≤ 1/3 and limp1→1 RAIFV−3 = 1/3

The worst case redundancy of AIFV-3 codes is 1/3

10 / 44

SLIDE 11

The worst case redundancy RAIFV−3 of AIFV-3 codes

Redundancy

1⁄2 1⁄3 1⁄4 1⁄2

RAIFV−2

✻

f3(p1)

1

p1

Theorem [Hu et al, Theorem 7] If p1 ≥ 1/2, then

RAIFV−3 ≤ min{RAIFV−2, f3(p1)} ≤ 1/3.

Theorem [Hu et al, Theorem 5] If p1 ≤ 1/2, then

RAIFV−2 ≤ 1/4 1/5.

We need a new lemma to prove RAIFV−5 ≤ 1/5.

11 / 44

SLIDE 12

The worst case redundancy RAIFV−3 of AIFV-3 codes

Redundancy

1⁄2 1⁄3 1⁄4 1⁄2

RAIFV−2 RAIFV−3

1

p1

In our research, Theorem 3 in the paper shows

RAIFV−3 ≤ 1/3

if p1 ≥ 1/2 using new upper bounds on RAIFV−3.

12 / 44

SLIDE 13

The worst case redundancy RAIFV−3 of AIFV-3 codes

Redundancy

1⁄2 1⁄3 1⁄4 1⁄2

RAIFV−2 RAIFV−3

1

p1

In our research, Theorem 3 in the paper shows

RAIFV−3 ≤ 1/3

if p1 ≥ 1/2 using new upper bounds on RAIFV−3. Lemma 1 in the paper shows

RAIFV−3 ≤ 16

9 − log2 3

if p1 ≤ 1/2,

13 / 44

SLIDE 14

The worst case redundancy RAIFV−3 of AIFV-3 codes

Redundancy

1⁄2 1⁄3 1⁄4 1⁄2

RAIFV−2 RAIFV−3

1

p1

In our research, Theorem 3 in the paper shows

RAIFV−3 ≤ 1/3

if p1 ≥ 1/2 using new upper bounds on RAIFV−3. Lemma 1 in the paper shows

RAIFV−3 ≤ 16

9 − log2 3

if p1 ≤ 1/2,

≤ 1/5 ≤ 1/3.

14 / 44

SLIDE 15

The worst case redundancy RAIFV−3 of AIFV-3 codes

Redundancy

1⁄2 1⁄3 1⁄4 1⁄2

RAIFV−2 RAIFV−3

1

p1

In our research, Theorem 3 in the paper shows

RAIFV−3 ≤ 1/3

if p1 ≥ 1/2 using new upper bounds on RAIFV−3. Lemma 1 in the paper shows

RAIFV−3 ≤ 16

9 − log2 3

if p1 ≤ 1/2,

≤ 1/5 ≤ 1/3.

Lemma 1 shows RAIFV−5 ≤ RAIFV−3 ≤ 1/5 if p1 ≤ 1/2.

15 / 44

SLIDE 16

The worst case redundancy RAIFV−4 of AIFV-4 codes

Redundancy

1⁄2 1⁄4 1⁄2

RAIFV−2

✻

f4(p1)

1

p1

Theorem [2, Theorem 8]

RAIFV−4 ≤ min{RAIFV−2, f4(p1)} ≤ 1/4 if p1 ≥ 1/2 RAIFV−4 ≤ RAIFV−2 ≤ 1/4

if p1 ≤ 1/2

f4(x) := 2 − x − h(x) + (x+1)(3−4x)+x2(2−3x)+x3(1−x)

x3+x2+x+1

given in [2, Eq. (52)] using an AIFV-4 code.

16 / 44

SLIDE 17

The worst case redundancy RAIFV−5 of AIFV-5 codes

Redundancy

1⁄2 1⁄4 1⁄5 1⁄6 1⁄2

RAIFV−3

✻

f5(p1)

1

p1

Theorem [New]

RAIFV−5 ≤ min{RAIFV−3, f5(p1)} ≤ 1/5 if p1 ≥ 1/2 RAIFV−5 ≤ RAIFV−3 ≤ 1/5

if p1 ≤ 1/2

f5(x) := 2 − x − h(x) + (x+1)(4−5x)+x2(3−4x)+x3(2−3x)+x4(1−x)

x4+x3+x2+x+1

given using an AIFV-5 code.

17 / 44

SLIDE 18

Proof of Gallager’s Bound (1. Notation )

n := |S|, the size of the source alphabet S

Each source probability pi, i = 1, 2, · · · , n is assigned to each leaf of tHuffman. Simple example of Huffman tree tHuffman with n = 4

S ∼

a

b c d

1/2 1/4 1/8 1/8

p1

p2 p3 p4

1 1 1

tHuffman p1 [1/2] p2 [1/4] p3 [1/8] p4 [1/8]

18 / 44

SLIDE 19

Proof of Gallager’s Bound (1. Notation )

n := |S|, the size of the source alphabet S

Each source probability pi, i = 1, 2, · · · , n is assigned to each leaf of tHuffman. Each internal node has the sum of the probabilities of its two children. Numbering all the nodes except the root in order of decreasing probability and increasing depth starting from 1 to 2n − 2.

qk denotes the probability of the node k of tHuffman, and q1 ≥ q2 ≥ · · · ≥ q2n−2.

Simple example of Huffman tree tHuffman with n = 4

S ∼

a

b c d

1/2 1/4 1/8 1/8

p1

p2 p3 p4 tHuffman p1 q1 [1/2] q2 [1/2] p2 q3 [1/4] q4 [1/4] p3 q5 [1/8] p4 q6 [1/8]

19 / 44

SLIDE 20

Proof of Gallager’s Bound (1. Notation )

n := |S|, the size of the source alphabet S

Each source probability pi, i = 1, 2, · · · , n is assigned to each leaf of tHuffman. Each internal node has the sum of the probabilities of its two children. Numbering all the nodes except the root in order of decreasing probability and increasing depth starting from 1 to 2n − 2.

qk denotes the probability of the node k of tHuffman, and q1 ≥ q2 ≥ · · · ≥ q2n−2.

Simple example of Huffman tree tHuffman with n = 4 Average codeword length:

LHuffman =

n−1

k=1

(q2k−1 + q2k)

Entropy:

H =

n−1

k=1

(q2k−1 + q2k) h

q2k

q2k−1 + q2k

tHuffman

p1 q1 q2 p2 q3 q4 p3 q5 p4 q6

20 / 44

SLIDE 21

Proof of Gallager’s Bound (1. Notation )

n := |S|, the size of the source alphabet S

Each source probability pi, i = 1, 2, · · · , n is assigned to each leaf of tHuffman. Each internal node has the sum of the probabilities of its two children. Numbering all the nodes except the root in order of decreasing probability and increasing depth starting from 1 to 2n − 2.

qk denotes the probability of the node k of tHuffman, and q1 ≥ q2 ≥ · · · ≥ q2n−2.

Using abridged notation

Qk := q2k−1 + q2k

and U := {1, 2, · · · , n − 1} Average length:

LHuffman =

n−1

k=1

(q2k−1 + q2k) =

k∈U

Qk

Entropy:

H =

n−1

k=1

(q2k−1 + q2k) h

q2k

q2k−1 + q2k

=
k∈U

Qk h(q2k/Qk)

21 / 44

SLIDE 22

Proof of Gallager’s Bound (2. Inequalities)

RHuffman = LHuffman − H =

k∈U

Qk −

k∈U

Qkh(q2k/Qk) = 1 − h(q1) +

k∈U\{1}

(Qk − Qkh(q2k/Qk))

because Q1 = q1 + q2 = 1, and h(q2) = h(1 − q2) = h(q1)

q2 = 1 − q1

✓ ✏ 1

tHuffman q1 q2 . . . [1 − q1]

✒ ✑

22 / 44

SLIDE 23

Proof of Gallager’s Bound (2. Inequalities)

RHuffman = LHuffman − H =

k∈U

Qk −

k∈U

Qkh(q2k/Qk) = 1 − h(q1) +

k∈U\{1}

(Qk − Qkh(q2k/Qk)) ≤ 1 − h(q1) +

k∈U\{1}

(q2k−1 − q2k)

follows from

−Qkh(q2k/Qk) < −2q2k

and Qk = q2k−1 + q2k

x h(x) > 2x

23 / 44

SLIDE 24

Proof of Gallager’s Bound (2. Inequalities)

RHuffman = LHuffman − H =

k∈U

Qk −

k∈U

Qkh(q2k/Qk) = 1 − h(q1) +

k∈U\{1}

(Qk − Qkh(q2k/Qk)) ≤ 1 − h(q1) +

k∈U\{1}

(q2k−1 − q2k) : Telescoping series ≤ 1 − h(q1) + q2 ✘✘✘✘

✘

−q3 + q4 ✘✘✘✘

✘

−q5 + q6 · · ·✘✘✘

✘

−q2n−2

24 / 44

SLIDE 25

Proof of Gallager’s Bound (2. Inequalities)

RHuffman = LHuffman − H =

k∈U

Qk −

k∈U

Qkh(q2k/Qk) = 1 − h(q1) +

k∈U\{1}

(Qk − Qkh(q2k/Qk)) ≤ 1 − h(q1) +

k∈U\{1}

(q2k−1 − q2k) ≤ 1 − h(q1) + q2 = 1 − h(q1) + 1 − q1 = 2 − q1 − h(q1) = 2 − p1 − h(p1)

if p1 ≥ 1/2 If p1 ≥ 1/2, then p1 = q1 ✓ ✏ 1

tHuffman p1 = q1 q2 . . . [1 − q1]

✒ ✑

25 / 44

SLIDE 26

Proof of Gallager’s Bound (2. Inequalities)

RHuffman = LHuffman − H =

k∈U

Qk −

k∈U

Qkh(q2k/Qk) = 1 − h(q1) +

k∈U\{1}

(Qk − Qkh(q2k/Qk)) ≤ 1 − h(q1) +

k∈U\{1}

(q2k−1 − q2k) ≤ 1 − h(q1) + q2 = 1 − h(q1) + 1 − q1 = 2 − q1 − h(q1) = 2 − p1 − h(p1)

if p1 ≥ 1/2 Redundancy

1

1⁄2 1⁄2 3⁄4

1

p1

✻ Gallager [1], RHuffman ≤ 2 − p1 − h(p1)

26 / 44

SLIDE 27

Proof of Bound on RAIFV−2 (1. Make two AIFV-2 codes (t(a)

0 , t(a) 1 ), (t(b) 0 , t(b) 1 )) 3 steps to prove an upper bound on RAIFV−2: (1st) Make a code tree tHuff of Huffman code for S . (2nd) Transform tHuff into a tree tBase as follow: for k = 2, 3, . . . , n − 1 do if node 2k − 1 is a leaf and q2k−1 > 2q2k then

1 q2k−1 q2k q2k−1 [Qk] q2k [q2k] Convert from Renew K ← K ∪ {k} to

(3rd) Make two types of AIFV-2 codes (t(a)

0 , t(a) 1 ),

(t(b)

0 , t(b) 1 ) from tBase, and evaluate those

redundancy R(a)

AIFV−2, R(b) AIFV−2. Then, we have

RAIFV−2 ≤ min{R(a)

AIFV−2, R(b) AIFV−2}.

Redundancy

1

1⁄2 1⁄4 1⁄2

√ 5−1 2

3⁄4

1

p1

Hu et al [2], RAIFV−2 27 / 44

SLIDE 28

Proof of Bound on RAIFV−2 (1. Make two AIFV-2 codes (t(a)

0 , t(a) 1 ), (t(b) 0 , t(b) 1 )) 3 steps to prove an upper bound on RAIFV−2: (1st) Make a code tree tHuff of Huffman code for S . (2nd) Transform tHuff into a tree tBase as follow: for k = 2, 3, . . . , n − 1 do if node 2k − 1 is a leaf and q2k−1 > 2q2k then

1 q2k−1 q2k q2k−1 [Qk] q2k [q2k] Convert from Renew K ← K ∪ {k} to

(3rd) Make two types of AIFV-2 codes (t(a)

0 , t(a) 1 ),

(t(b)

0 , t(b) 1 ) from tBase, and evaluate those

redundancy R(a)

AIFV−2, R(b) AIFV−2. Then, we have

RAIFV−2 ≤ min{R(a)

AIFV−2, R(b) AIFV−2}.

1 t(a) := tBase q1 q2 Evaluate R(a)

AIFV−2 of AIFV-2 code (t(a) 0 , t(a) 1 )

1 1 t(a)

1

q1 q2 t(b) 1 q2 Evaluate R(b)

AIFV−2 of AIFV-2 code (t(b) 0 , t(b) 1 )

1 1 t(b)

1

q1 q2

28 / 44

SLIDE 29

Calculate average codeword length LAIFV−2(t(a)

0 , t(a) 1 )

U := {1, 2, . . . , n − 1}, K = {k ∈ U \ {1} | q2k−1 is leaf and 2q2k < q2k−1} LHuffman =

k∈U

Qk, Qk := q2k−1 + q2k, LtBase =

k∈U\K

Qk +

k∈K

2q2k, Lt(a)

0 = LtBase,

Lt(a)

1 = LBase + q2,

π(a)

0 = 1 −

k∈K

q2k−1, π(a)

1 =

k∈K

q2k−1, and π(a)

0 + π(b) 1 = 1

Average codeword length of : (t(a)

0 , t(a) 1 )

LAIFV-2(t(a)

0 , t(a) 1 ) := π(a) 0 L(a) 0 + π(a) 1 L(a) 1

=

k∈U\K

Qk +

k∈K

(2q2k + q2q2k−1)

✓ ✏

1 q2k−1 q2k q2k−1 [Qk] q2k [q2k]

tBase :

k ∈ U \ K k ∈ K

✒ ✑ ✓ ✏

1

(t(a)

0 , t(a) 1 ) :

t(a)

0 := tBase

q1 q2 1 1

t(a)

1

q1 q2

✒ ✑

29 / 44

SLIDE 30

Proof of Bound on RAIFV−2 (2. Inequalities)

R(a)

AIFV-2

= LAIFV-2(t(a)

0 , t(a) 1 ) − H

=

k∈U\K

Qk(1 − h( q2k/Qk

1/3≤q2k/Qk≤1/2

)) +

k∈K

(2q2k + q2q2k−1 − Qkh( q2k/Qk

0≤q2k/Qk≤1/3

)) ≤ 1 − h(q1) +

k∈U\(K∪{1})

Qk

1 −

q2k 2Qk + 3 4

+
k∈K

(2q2k + q2q2k−1 − 11q2k/4) < 2 − 2p1 − p2

1 − h(p1).

1

3⁄4 1⁄3 1⁄2

x h(x) ≥            1 2 x + 3 4

if 1/3 ≤ x ≤ 1/2

11 4 x

if 0 ≤ x ≤ 1/3

30 / 44

SLIDE 31

Upper Bounds on Redundancy RAIFV−2 when p1 is known

L R 1 t(a) := tBase q1 q2 L 1 R 1 t(a)

1

q1 q2 L t(b) 1 q2 L 1 1 t(b)

1

q1 q2

Redundancy

1

1⁄2 1⁄4

01⁄2

√ 5−1 2

3⁄4

1

✠
✠

p1

Hu, et al. [2], RAIFV−2

RAIFV-2 ≤ min

LAIFV-2(t(a)

0 , t(a) 1 ), LAIFV-2(t(b) 0 , t(b) 1 )

− H,

Theorem [2, Theorem 4]

RAIFV−2 ≤                          2 − 2p1 + p2

1 − h(p1) if 1 2 ≤ p1 ≤ √ 5−1 2

using AIFV-2 code (t(a)

0 , t(a) 1 ) 2+p1−2p2

1

1+p1

− h(p1) if

√ 5−1 2

≤ p1 < 1

using AIFV-2 code (t(b)

0 , t(b) 1 )

❳❳ ✲ ✏✏✏✏ ✏ ✶

31 / 44

SLIDE 32

Proof of Bound on RAIFV−3 (1. Make four type ofAIFV-3 codes)

3 steps to prove an upper bound on RAIFV−3: (1st) Make a code tree tHuff of Huffman code for S . (2nd) Transform tHuff into a tree tB3 as follow: for k = 2, 3, . . . , n − 1 do if node 2k − 1 is a leaf and 2 < q2k−1 ≤ 4q2k then Convert he sibling pair (2k − 1, 2k) of Fig.(0) to Fig.(1), and renew K1 ← K1 ∪ {k} else if node 2k − 1 is a leaf and 4q2k < q2k−1 then Convert he sibling pair (2k − 1, 2k) of Fig.(0) to Fig.(2), and renew K2 ← K2 ∪ {k} 1

q2k−1 q2k q2k−1 [Qk] q2k [q2k] q2k−1 [Qk] q2k [q2k] [q2k]

(0) (1) 2k − 1 is a master-1 node (2) 2k − 2 is a master-2 node

32 / 44

SLIDE 33

Proof of Bound on RAIFV−3 (1. Make four type ofAIFV-3 codes)

3 steps to prove an upper bound on RAIFV−3: (1st) Make a code tree tHuff of Huffman code for S . (2nd) Transform tHuff into a tree tB3 . (3rd) Make four types of AIFV-3 codes (t(α)

0 , t(α) 1 , t(α) 2 ), α ∈ {a, b, c, d} from tB3, and

evaluate RAIFV−3 ≤ min

R(a)

AIFV−3, R(b) AIFV−3, R(c) AIFV−3, R(d) AIFV−3

.

a b

1

t(a)

0 := tB3

q1 q2 a t(b) = t(c) q1 q2 a t(d) q1 q2 a

1

b

1

t(a)

1 , α ∈ {a, . . . , d}

q1 q2

1 1

b

1

t(a)

2 = t(b) 2

q1 q2 a

1

t(c)

2

= t(d)

2

q1 q2

33 / 44

SLIDE 34

Proof of Bound on RAIFV−3 (1. Make four type ofAIFV-3 codes)

3 steps to prove an upper bound on RAIFV−3: (1st) Make a code tree tHuff of Huffman code for S . (2nd) Transform tHuff into a tree tB3 (3rd) Make four types of AIFV-3 codes (t(α)

0 , t(α) 1 , t(α) 2 ), α ∈ {a, b, c, d} from tB3, and

evaluate RAIFV−3 ≤ min

R(a)

AIFV−3, R(b) AIFV−3, R(c) AIFV−3, R(d) AIFV−3

.

a b

1

t(a) q1 q2 a t(b)

0 = t(c)

q1 q2 a t(d) q1 q2 a

1

b

1

t(b)

1 , α ∈ {a, . . . , d}

q1 q2

1 1

b

1

t(a)

2

=t(b)

2

q1 q2 a

1

t(c)

2

= t(d)

2

q1 q2

34 / 44

SLIDE 35

Proof of Bound on RAIFV−3 (1. Make four type ofAIFV-3 codes)

3 steps to prove an upper bound on RAIFV−3: (1st) Make a code tree tHuff of Huffman code for S . (2nd) Transform tHuff into a tree tB3 (3rd) Make four types of AIFV-3 codes (t(α)

0 , t(α) 1 , t(α) 2 ), α ∈ {a, b, c, d} from tB3, and

evaluate RAIFV−3 ≤ min

R(a)

AIFV−3, R(b) AIFV−3, R(c) AIFV−3, R(d) AIFV−3

.

a b

1

t(a) := tB3 q1 q2 a t(b) =t(c) q1 q2 a t(d) q1 q2 a

1

b

1

t(c)

1 , α ∈ {a, . . . , d}

q1 q2

1 1

b

1

t(a)

2

= t(b)

2

q1 q2 a

1

t(c)

2 = t(d) 2

q1 q2

35 / 44

SLIDE 36

Proof of Bound on RAIFV−3 (1. Make four type ofAIFV-3 codes)

3 steps to prove an upper bound on RAIFV−3: (1st) Make a code tree tHuff of Huffman code for S . (2nd) Transform tHuff into a tree tB3 (3rd) Make four types of AIFV-3 codes (t(α)

0 , t(α) 1 , t(α) 2 ), α ∈ {a, b, c, d} from tB3, and

evaluate RAIFV−3 ≤ min

R(a)

AIFV−3, R(b) AIFV−3, R(c) AIFV−3, R(d) AIFV−3

.

a b

1

t(a) := tB3 q1 q2 a t(b) = t(c) q1 q2 a t(d) q1 q2 a

1

b

1

t(d)

1 , α ∈ {a, . . . , d}

q1 q2

1 1

b

1

t(a)

2

= t(b)

2

q1 q2 a

1

t(c)

2

=t(d)

2

q1 q2

36 / 44

SLIDE 37

Subdivide AIFV-3 code (t(α)

0 , t(α) 1 , t(α) 2 ) into 3 cases If p1 ≥ 1/2, then we subdivide AIFV-3 code (t(a)

0 , t(a) 1 , t(a) 2 ) into 3 cases

            

AIFV-3 code (t(a)

0 , t(a) 1 , t(a) 2 ) and node of index 2 ( and q2) is complete node, 2 K1 ∪ K2

AIFV-3 code (t(a)

0 , t(a) 1 , t(a) 2 ) and node of index 2 ( and q2) is master-1 node, 2 ∈ K1

AIFV-3 code (t(a)

0 , t(a) 1 , t(a) 2 ) and node of index 2 ( and q2) is master-2 node, 2 ∈ K2 tL tR

1 t(a) q1 q2 tL 1 tR 1 t(a)

L

q1 q2

AIFV-3 code (t(a)

0 , t(a) 1 , t(a) 2 )

t′′

L

1 t′

L

1 tR 1 t(a)

2

q1 ¯ q ˆ q

✓ ✏

t′

L

t′′

L

1 q2 ¯ q ˆ q tL q2 ˆ q tL q2 ˆ q complete master-1 master-2 node case, node case, node case, 2 K1 ∪ K2 2 ∈ K1 2 ∈ K2

✒ ✑

37 / 44

SLIDE 38

Bound the redundancy R(a)

3

f AIFV-3 code (t(a)

0 , t(a) 1 , t(a) 2 ) Average codeword length LB3 of code tree tB3, which is tranformed from tHuff:

LB3 =

k∈U\(K1∪K2)

Qk +

k∈K1

2q2k +

k∈K2

3q2k, K1 := {k ∈ U \ {1} | node 2k − 1 is a leaf and 2q2k < q2k−1 ≤ 4q2k}, K2 := {k ∈ U \ {1} | node 2k − 1 is a leaf and 4q2k < q2k−1}.

For AIFV-3 code (t(a)

0 , t(a) 1 , t(a) 2 ), the redundancy R(a) 3

≤ max{R(a,0)

3

, R(a,1)

3

, R(a,2)

3

}                          R(a,0)

3

= LB3 + q2

k∈K1

q2k−1 + q4

k∈K2

q2k−1 − H

if complete node 2 K1 ∪ K2,

R(a,1)

3

= LB + q2

k∈K1

q2k−1 + q2c−1(1 − q2)

k∈K2

q2k−1 − H

if master-1 node 2 ∈ K1,

R(a,2)

3

= LB + q2

k∈K1

q2k−1 + q2c−1 − q2c 1 + q2c−1

k∈K2

q2k−1 − H

if master-2 node 2 ∈ K2.

38 / 44

SLIDE 39

Upper bound on R(a,0)

3

if node of index 2 is complete node

R(a,0)

3

≤ 1 − h(q1) +

k∈U\{1}

Qk(1 − h( q2k/Qk

1/3≤q2k/Qk≤1/2

)) +

k∈K1

(2q2k + q2q2k−1 − Qkh( q2k/Qk

1/3≤q2k/Qk≤1/5

)) +

k∈K2

(3q2k + q4q2k−1 − Qkh( q2k/Qk

0≤q2k/Qk≤1/5

)) ≤ 1 − h(q1) +

k∈U\{1}

Qk

1 − q2k

2Qk − 3 4

+
k∈K1

(q2q2k−1 − Qk/4) +

k∈K2

(3q2k + q4q2k−1 − 10q2k/3) ≤ · · · ≤ 5 − q1 4 − h(q1).

1

3⁄4 1⁄4

0 0

1⁄2

h(x) ≥             

1 2 x + 3 4 1 3 ≤ x ≤ 1 2

2x + 1

4 1 5 ≤ x ≤ 1 3 10 3 x

0 ≤ x ≤ 1

5

39 / 44

SLIDE 40

Upper bound on R(a,2)

3

if node of index 2 is master-2 node

R(a,2)

3

≤ 1 − h(q1) +

k∈U\{1}

Qk(1 − h( q2k/Qk

1/3≤q2k/Qk≤1/2

)) +

k∈K1

(2q2k + q2q2k−1 − Qkh( q2k/Qk

1/3≤q2k/Qk≤1/5

)) +

k∈K2

(3q2k + q2c−1 − q2c 1 + q2c−1 q2k−1 − Qkh( q2k/Qk

0≤q2k/Qk≤1/5

)) ≤ · · · ≤ 3 − 3q1 + q2

1

2 − q1 . R(a)

3

≤ max{R(a,0)

3

, R(a,1)

3

, R(a,2)

3

} = 3 − 3q1 + q2

1

2 − q1 . R(b)

AIFV−3, R(c) AIFV−3, R(d) AIFV−3 can be bounded similarly, and RAIFV−3 ≤ min

R(α)

3 , α ∈ {a, b, c, d}

.

1

3⁄4 1⁄4

0 0

1⁄2

h(x) ≥             

1 2 x + 3 4 1 3 ≤ x ≤ 1 2

2x + 1

4 1 5 ≤ x ≤ 1 3 10 3 x

0 ≤ x ≤ 1

5

40 / 44

SLIDE 41

Upper Bounds on Redundancy RAIFV−3 when p1 ≥ 1/2 is known

Redundancy

1⁄2 1⁄3 1⁄3 1⁄6 1⁄2

√ 5−1 2

β1 ≈ 0.6889

3⁄4

β2 ≈ 0.8287

1

p1

Theorem 3 in the paper

RAIFV−3 ≤                                               

3−3p1+p2

1

2−p1

− h(p1)

if 1

2 ≤ p1 ≤ √ 5−1 2

,

4−p1−3p2

1+p3 1

(2−p1)(1+p1) − h(p1)

if

√ 5−1 2

< p1 ≤ β1,

3−p1−2p2

1+p3 1

1+p1

− h(p1)

if β1 ≤ p1 ≤ 3

4, 9−5p2

1

4(1+p1) − h(p1)

if 3

4 < p1 ≤ β2, 23+24p1−35p3

1

12(1+p1+p2

1) − h(p1)

if β2 ≤ p1 < 1,

41 / 44

SLIDE 42

Upper Bounds on Redundancy RAIFV−3 when p1 ≤ 1/2 is known

If p1 ≤ 1/2, then 1/2 ≤ q1 ≤ 2/3 from the construction of Huffman tree tHuffman. If p1 ≤ 1/2, then 1/2 ≤ q1 ≤ 2/3.

RAIFV-3 ≤ max

R(a)

3 , R(e) 3

=

max

1/2≤q1≤2/3

       3 − 3q1 + q2

1

2 − q1 , 2 + q1 − q2

1

2        − h(q1) = 16 9 − log2 3 < 1/5

b

1

t(e) := tB3 q1 q2 Case (e) node 2 is a leaf

1

b

1

t(e)

1

q1 q2 bL

1

bR

1 1

t(e)

2

q2 q1

1⁄5

16 9 − log2 3

1⁄6 1⁄8 1⁄2 2⁄3

R(e)

3

≤

2+q1−q2

1

2

− h(q1) ✻ ✻ R(a)

3

≤

3−3q1+q2

1

2−q1

− h(q1)

q1

42 / 44

SLIDE 43

Conclusion

RAIFV−3 ≤                                               

3−3p1+p2

1

2−p1

− h(p1)

if 1

2 ≤ p1 ≤ √ 5−1 2

,

4−p1−3p2

1+p3 1

(2−p1)(1+p1) − h(p1)

if

√ 5−1 2

< p1 ≤ β1,

3−p1−2p2

1+p3 1

1+p1

− h(p1)

if β1 ≤ p1 ≤ 3

4, 9−5p2

1

4(1+p1) − h(p1)

if 3

4 < p1 ≤ β2, 23+24p1−35p3

1

12(1+p1+p2

1) − h(p1)

if β2 ≤ p1 < 1, RAIFV−3 ≤ max

1/2≤q1≤2/3

3−3q1+q2

1

2−q1

− h(q1),

2+q1−q2

1

2

− h(q1)

= 16

9 − log2 3 < 1 5

if p1 ≤ 1

2.

Theorem

RAIFV−5 = 1 5.

43 / 44

SLIDE 44

Typo: Theorem 3, Equeation (3) on page 1 in the paper

Please correct the red 2 in the exponent of p1 in the bottom line: Before the error correction

Theorem 3. RAI-3 ≤                                             

3−3p1+p2

1

2−p1

− h(p1) if 1

2 ≤ p1 ≤ √ 5−1 2

,

4−p1−3p2

1+p3 1

(2−p1)(1+p1) − h(p1)

if

√ 5−1 2

< p1 ≤ β1,

3−p1−2p2

1+p3 1

1+p1

− h(p1) if β1 ≤ p1 ≤ 3

4,

(3)

9−5p2

1

4(1+p1) − h(p1)

if 3

4 < p1 ≤ β2, 23+24p2

1−35p3 1