Covers Open Problem: The Capacity of the Relay Channel Ayfer Ozg - - PowerPoint PPT Presentation

Cover’s Open Problem: “The Capacity of the Relay Channel”

Ayfer ¨ Ozg¨ ur

Stanford University

Advanced Networks Colloquia Series University of Maryland, March 2017 Joint work with Xiugang Wu and Leighton Pate Barnes.

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 1 / 26

Father of the Information Age

Claude Shannon (1916-2001)

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 2 / 26

The Bell System Technical Journal

• Vol. XXVII

J Illy, 1948 No.3

A Mathematical Theory of Communication

By c. E. SHANNON

IXTRODUCTION

T

HE recent development of various methods of modulation such as reM and PPM which exchange bandwidth for signal-to-noise ratio has in- tensified the interest in a general theory of communication. A basis for such a theory is contained in the important papers of Nyquist! and Hartley"

• n this subject.

In the present paper we will extend the theory to include a number of new factors, in particular the effect of noise in the channel, and the savings possible due to the sta tistiral structure of the original message and due to the nature of the final destination of the information. The fundamental problem of communication is that of reproducing at

• ne point either exactly or approximately a message selected at another

point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is

• ne selected from a set of possible messages.

The system must be designed to operate for each possible selection, not just the one which will actually be chosen since this is unknown at the time of design.

If the number of messages in the set is finite then this number or any

monotonic function of this number can be regarded as a measure of the in- formation produced when one message is chosen from the set, all choices being equally likely. As was pointed out by Hartley the most natural choice is the logarithmic function. Although this definition must be gen- eralized considerably when we consider the influence of the statistics of the message and when we have a continuous range of messages, we will in all cases use an essentially logarithmic measure. The logarithmic measure is more convenient for various reasons:

• 1. It is practically more useful.

Parameters of engineering importance

1 Nyquist, H., "Certain Factors Affecting Telegraph Speed," Belt System Tectmical J OUT-

nal, April 1924, p, 324; "Certain Topics in Telegraph Transmission Theory," A. I. E. E.

TI aIlS., v. 47, April 1928, p. 617.

2 Hartley. R. V. L.. "Transmission oi Information.' Belt System Technical Journal, July

1928, p. . 'US. 379

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 3 / 26

“A method is developed for representing any communication system geometrically...”

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 4 / 26

AWGN Channel

Transmitter Receiver

P Capacity C = log

• 1 + P

N

• Ayfer ¨

Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 5 / 26

Converse: Sphere Packing

√ nN

p n(P + N)

√ nN √ nN

Y sphere

Xn(1) Xn(2) Xn(3)

noise sphere

# of X n ≤

• Sphere
• n(P + N)
• Sphere

√ nN

• .

= 2

n 2 log 2πe(P+N)

2

n 2 log 2πeN

= 2

n 2 log(1+ P N ) Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 6 / 26

Achievability: Geometric Random Coding

Pr(∃ false X n) ≤ |Lens|

• Sphere

√ nP

• × 2nR

. = 2

n 2 log 2πe PN P+N

2

n 2 log 2πeP

× 2nR = 2− n

2 log(1+ P N ) × 2nR Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 7 / 26

The story goes...

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 8 / 26

Cover’s Open Problem

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 9 / 26

Gaussian case

In = f(Zn)

Source Relay Destination

W1 ∼ N(0, N) W2 ∼ N(0, N) Xn Y n Zn C0

C(∞) = 1 2 log

• 1 + 2P

N

• Achievability: C ∗

0 = ∞.

Cutset-bound (Cover and El Gamal’79): C ∗

0 ≥ 1

2 log

• 1 + 2P

N

• − 1

2 log

• 1 + P

N

• .

Potentially, C ∗

0 → 0 as P/N → 0.

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 10 / 26

Main Result

In = f(Zn)

Source Relay Destination

W1 ∼ N(0, N) W2 ∼ N(0, N) Xn Y n Zn C0

Theorem C ∗

0 = ∞

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 11 / 26

Upper Bound on the Capacity

0.1 0.2 0.3 0.7 0.8 0.9 1 2.5 2.6 2.7 2.8 2.9 3 3.1

SNR = 15 dB

Cut-set bound C-F Old bound New bound

0.4 0.5 0.6 C0 (bit/channel use)

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 12 / 26

Cutset Bound

In = f(Zn)

Source Relay Destination

W1 ∼ N(0, N) W2 ∼ N(0, N) Xn Y n

Zn C0

nR ≤ I(X n; Y n, In) + nǫn = I(X n; Y n) + I(X n; In|Y n) + nǫn = I(X n; Y n) + H(In|Y n)

• ≤nC0

− H(In|Y n, X n)

• ≥0

+nǫn ≤ n(I(X; Y ) + C0 + ǫn)

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 13 / 26

Cutset Bound

In = f(Zn)

Source Relay Destination

W1 ∼ N(0, N) W2 ∼ N(0, N) Xn Y n

Zn C0

nR ≤ I(X n; Y n, In) + nǫn = I(X n; Y n) + I(X n; In|Y n) + nǫn = I(X n; Y n) + H(In|Y n)

• ≤nC0

− H(In|X n)

• ≥0

+nǫn ≤ n(I(X; Y ) + C0 + ǫn)

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 13 / 26

√ nN Xn Typical set of Zn/Y n ⊆ In-th bin

Zn Y n

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26

√ nN Xn Typical set of Zn/Y n ⊆ In-th bin

Zn Y n

If H(In|X n) = 0,

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26

√ nN Xn Typical set of Zn/Y n ⊆ In-th bin

Zn Y n

If H(In|X n) = 0, then H(In|Y n) = 0.

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26

In = f(Zn)

Source Relay Destination

W1 ∼ N(0, N) W2 ∼ N(0, N) Xn Y n Zn C0

R ≤ I(X n; Y n) + H(In|Y n)

• ≤?

− H(In|X n)

• =−n log sin θn

+nǫn

Goal:

In = f (Z n) − Z n − X n

• H(In|X n)=−n log sin θn

−Y n

• H(In|Y n)≤?

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 15 / 26

H(In|X n) = 0

Multiple bins Xn √ nN Typical set of Zn

# of bins =? P(each bin) =?

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 16 / 26

From n- to nB- Dimensional Space

X, Y, Z, I : B-length i.i.d. from {(X n(b), Y n(b), Z n(b), In(b))}B

b=1.

If H(In|X n) = −n log sin θn, then for any typical (x, i) p(i|x) . = 2nB log sin θn, P(Z ∈ A(i)|x) . = 2nB log sin θn

√ nBN

x

Typical set of Z/Y Pr . = 2nB log sin θn ith bin Ax(i) |Ax(i)| . = 2nB( 1

2 log 2πeN sin2 θn)

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 17 / 26

Isoperimetric Inequalities

Isoperimetric Inequality in the Plane (Steiner 1838) Among all closed curves in the plane with a given enclosed area, the circle has the smallest perimeter.

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 18 / 26

Isoperimetric Inequalities

Isoperimetric Inequality on the Sphere (Levy 1919) Among all sets on the sphere with a given volume, the spherical cap has the smallest boundary, or the smallest volume of ω-neighborhood for any ω > 0.

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 19 / 26

Blowing-up Lemma

|Ax(i)| . = 2nB( 1

2 log 2πeN sin2 θn)

Isoperimetric Inequality on the Sphere + Measure Concentration: P(Z ∈ blow-up of Ax(i)|x) ≈ 1. ⇓ P(Y ∈ blow-up of Ax(i)|x) ≈ 1.

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 20 / 26

Geometry of Typical Sets

n-dimensional space: Almost all (X n, Y n, Z n, In) √ nN

Xn Y n Zn → In

nB-dimensional space: Almost all (x, y, i) √ nBN

z → i x

y

π 2 − θn

Information Inequality: (Wu and Ozgur, 2015) H(In|Y n) ≤ n(2 log sin θn + √2 log sin θn ln 2 log e).

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 21 / 26

A new approach

Ax(i) Y Ax(i) Y

Control the intersection of a sphere drawn around a randomly chosen Y and Ax(i).

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 22 / 26

Easy if Ax(i) is a spherical cap

√ nBN

z0

y0

x

θn

Cap(z0, θn) Cap(y0, ωn) Cap(z0, θn) ∩ Cap(y0, !n) ωn

|Cap(z0, θn) ∩ Cap(Y, ωn)| . = 2nB( 1

2 log 2πeN(sin2θn−cos2 ωn)) Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 23 / 26

Strengthening of the Isoperimetric Inequality

Strengthening of the Isoperimetric Inequality: Among all sets on the sphere with a given volume, the spherical cap has minimal intersection volume at distance ω for almost all points on the sphere for any ω > π/2 − θ. Proof: builds on the Riesz rearrangement inequality.

Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 24 / 26

A Packing Argument

Given Y, # of I ≤

• Sphere
• Y,
• nBN4sin2 ωn

2

• 2nB( 1

2 log 2πeN(sin2θn−cos2 ωn)) Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 25 / 26

Summary

Solved an problem posed by Cover and named “The Capacity of Relay Channel” in Open Problems in Communication and Computation, Springer-Verlag, 1987. Developed a converse technique that significantly deviates from standard converse techniques based on single-letterization and has some new ingredients:

◮ Typicality ◮ Measure Concentration ◮ Isoperimetric Inequality Ayfer ¨ Ozg¨ ur (Stanford) “The Capacity of the Relay Channel” March’17 26 / 26