Sum Rate of Gaussian Multiterminal Source Coding Pramod Viswanath - - PowerPoint PPT Presentation

Sum Rate of Gaussian Multiterminal Source Coding

Pramod Viswanath University of Illinois, Urbana-Champaign March 19, 2003

Gaussian Multiterminal Source Coding

ENCODER 2 ENCODER 1 ENCODER K E D E C O D R

yK = ht

Kn

n2 n1 nN

n =

ˆ

n

y2 = ht

2n

y1 = ht

1n

mK m2 m1

Sum of rates of encoders R and distortion metric d (n, ˆ

n)

Quadratic Gaussian CEO Problem

ENCODER K ENCODER 1 ENCODER 2 E R D O C E D

y1 = n + z1 n ˆ n yK = n + zK x2 = n + z2 mK m2 m1

Sum of rates of encoders R and distortion metric d (n, ˆ n)

Result: Quadratic Gaussian CEO

• quadratic distortion metric d (n, ˆ

n) = (n − ˆ n)2

• For large number of encoders K,

R(D) = 1 2 log+

• σ2

n

D

• + σ2

z

2σ2

n

• σ2

n

D − 1

+

– Second term is loss w.r.t. cooperating encoders

Outline

• Problem Formulation:

Tradeoff between sum rate R and distortion (metric d (n, ˆ

n)).

• Main Result:

Characterize a class of distortion metrics for which no loss in sum rate compared with encoder cooperation – A multiple antenna test channel

Random Binning of Slepian-Wolf

∆ O ∆ × × × O O O × ∆ ∆ ∆ × × × ∆ ∆ ∆ O O O y1 y2 × ∆ × O O ×

• Rate is number of quantizers

Encoding in Slepian-Wolf

∆ O ∆ × × × O O O × ∆ ∆ ∆ × × × ∆ ∆ ∆ O O O y1 y2 × ∆ × O O ×

• Quantizer closest to realization

Decoding in Slepian-Wolf

• Decoder knows joint distribution of y1, y2
• It is given the two quantizer numbers from the encoders
• Picks the pair of points in the quantizers which best matches

the joint distribution – For jointly Gaussian y1, y2 nearest neighbor type test

• R1 + R2 = H (y1, y2) is sufficient for zero distortion

Deterministic Broadcast Channel

y2 = g(x) x y1 = f(x)

• Pick distribution on x such that y1, y2 have desired joint dis-

tribtion (Cover 98)

Slepian-Wolf code for Broadcast Channel

• Encoding: implement Slepian-Wolf decoder

– given two messages, find the appropriate pair y1, y2 in the two quantizers – transmit x that generates this pair y1, y2.

Slepian-Wolf code for Broadcast Channel

• Encoding: implement Slepian-Wolf decoder

– given two messages, find the appropriate pair y1, y2 in the two quantizers – transmit x that generates this pair y1, y2.

• Decoding: implement Slepian-Wolf encoder

– quantize y1, y2 to nearest point – messages are the quantizer numbers

Lossy Slepian-Wolf Source Coding

∆ O ∆ × × × O O O × ∆ ∆ ∆ × × × ∆ ∆ ∆ O O O u1 u2 × ∆ × O O ×

• Approximate y1, y2 by u1, u2

Lossy Slepian-Wolf Source Coding

• Encoding:

Find ui that matches source yi, separately for each i – For jointly Gaussian r.v. s, nearest neighbor calculation – Each encoder sends quantizer number containing the u picked

Lossy Slepian-Wolf Source Coding

• Encoding:

Find ui that matches source yi, separately for each i – For jointly Gaussian r.v. s, nearest neighbor calculation – Each encoder sends quantizer number containing the u picked

• Decoding: Reconstruct the u’s picked by the encoders

– reconstruction based on joint distribution of u’s – Previously ui = yi were correlated – Here u’s are independently picked

Lossy Slepian-Wolf

• We require

p [u1, . . . , uK|y1, . . . , yK] = ΠK

i=1p [ui|yi]

Lossy Slepian-Wolf

• We require

p [u1, . . . , uK|y1, . . . , yK] = ΠK

i=1p [ui|yi]

• Generate ˆ

n1, . . . , ˆ nK deterministically from reconstructed u’s.

• Need sum rate

Rsum = I (u1, . . . , uK; y1, . . . , yK) .

• Distortion equal to

E [d (n, ˆ

n)] .

Marton Coding for Broadcast Channel

Dec1 Dec2

w ∼ N (0, I) Ht

y

ˆ u1 ˆ u2

x

u1, u2 Enc

• Reversed encoding and decoding operations
• Sum rate I (u1, . . . , uK; y1, . . . , yK) .
• No use for the Markov property

p [u1, . . . , uK|y1, . . . , yK] = ΠK

i=1p [ui|yi]

Achievable Rates: Costa Precoding

m1 u1 w x Ht m2 u2 ˆ m1 ˆ m2

• Users’ data modulated onto spatial signatures u1, u2

Stage 1: Costa Precoding

Dec1

m1 u1 w x Ht m2 u2 ˆ m1

• Encoding for user 1 treating signal from user 2 as known

interference at transmitter

Stage 2

Dec2

m1 u1 w x Ht m2 u2 ˆ m2

• Encode user 2 treating signal for user 1 as noise

Adaptation to Lossy Slepian-Wolf

Dec

Ht z ∼ N (0, Kz)

x

u1, u2 Enc

y

ˆ u1 ˆ u2

• Joint distribution of u’s and y’s depends on noise z
• Performance independent of correlation in z

Noise Coloring

• Fix particular Costa coding scheme - fixes u’s and x.
• Idea:

Choose z such that p [u1, . . . , uK|y1, . . . , yK] = ΠK

i=1p [ui|yi]

and (Kz)ii = 1

• Then can adapt to Lossy Multiterminal Source Coding

Markov Condition and Broadcast Channel

• The Markov condition

p [u1, . . . , uK|y1, . . . , yK] = ΠK

i=1p [ui|yi]

• f independent interest in the broadcast channel

Markov Condition and Broadcast Channel

• The Markov condition

p [u1, . . . , uK|y1, . . . , yK] = ΠK

i=1p [ui|yi]

• f independent interest in the broadcast channel
• p [u1, . . . , uK|y1, . . . , yK] = ΠK

i=1p

ui|y1, . . . , yK, u1, u2, . . . , ui−1

Markov Condition and Broadcast Channel

• The Markov condition

p [u1, . . . , uK|y1, . . . , yK] = ΠK

i=1p [ui|yi]

• f independent interest in the broadcast channel
• p [u1, . . . , uK|y1, . . . , yK] = ΠK

i=1p

ui|y1, . . . , yK, u1, u2, . . . , ui−1

• Equivalent to:

given u1, . . . , ui−1 ui − → yi − → y1, . . . , yi−1, yi+1, . . . , yK

Implication

Dec

Ht z ∼ N (0, Kz)

x

u1, u2 Enc

y

ˆ u1 ˆ u2

• Need only y1 to decode u1
• Given u1, need only y2 to decode u2

Performance of Costa scheme equals that when receivers cooperate

Markov Condition and Noise Covariance

• The sum capacity is also achieved by such a scheme

(CS 01, YC 01, VT 02, VJG 02)

• For every Costa scheme, there is a choice of Kz such that

Markov condition holds (Yu and Cioffi, 01)

Sum Capacity

z ∼ N (0, Kz) Receivers Cooperating Ht H Ht H Sato

Enc

w w Broadcast

Dec Enc Dec Enc

w E

• xtKzx
• ≤ P

Reciprocity Reciprocity x1, x2 independent

Multiple Access Cooperating Transmitters

Sum Capacity

z ∼ N (0, Kz) Receivers Cooperating Ht H Ht H Sato

Enc

w w Broadcast

Dec Enc Dec Enc

w E

• xtKzx
• ≤ P

Reciprocity Convex Duality Reciprocity x1, x2 independent

Multiple Access Cooperating Transmitters

Gaussian Multiterminal Source Coding

ENCODER 2 ENCODER 1 ENCODER K E D E C O D R

yK = ht

Kn

n2 n1 nN

n =

ˆ

n

y2 = ht

2n

y1 = ht

1n

mK m2 m1

H = [h1, . . . , hK] Sum of rates of encoders R and distortion metric d (n, ˆ

n)

Main Result

• Distortion metric

d (n; ˆ

n) = 1

N (n − ˆ

n)t

I + Hdiag {p1, . . . , pK} Ht (n − ˆ

n)

– Here p1, . . . , pK - powers of users in reciprocal MAC

• Rate distortion function

R(D) = Sum rate of MAC − N log D = Sum rate of Broadcast Channel − N log D = log det

• I + HDHt

− N log D

Bells and Whistles

• For quadratic distortion metric

d (n; ˆ

n) = 1

N (n − ˆ

n)t (n − ˆ n)

set of H can be characterized

• Analogy with CEO problem:

For large number of encoders and random H characterization of R(D) almost surely

Discussion

• A “connection” made between coding schemes for multiter-

minal source and channel coding

Discussion

• A “connection” made between coding schemes for multiter-

minal source and channel coding

• Connection somewhat superficial

– relation between source coding and broadcast channel through a common random coding argument (PR 02, CC 02) – relation between source coding and multiple access chan- nel through a change of variable (VT 02, JVG 01)

Discussion

• A “connection” made between coding schemes for multiter-

minal source and channel coding

• Connection somewhat superficial

– relation between source coding and broadcast channel through a common random coding argument – relation between source coding and multiple access chan- nel through a change of variable

• Connection is suggestive

– a codebook level duality