Feedback Capacity of Finite-State Channels with Causal State - - PowerPoint PPT Presentation

Feedback Capacity of Finite-State Channels with Causal State Information Known at the Encoder

Eli Shemuel Oron Sabag Haim Permuter

Ben-Gurion University of the Negev, Israel

2020 IEEE International Symposium on Information Theory Los Angeles, California, USA June 2020

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 1 / 23

Table of Contents

1

The Setting

2

Main Results Theorem- The Capacity Theorem- Q-Graphs Examples of solving specific known problems Open problem example - Energy Harvesting Model

3

Conclusions

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 2 / 23

Table of Contents

1

The Setting

2

Main Results Theorem- The Capacity Theorem- Q-Graphs Examples of solving specific known problems Open problem example - Energy Harvesting Model

3

Conclusions

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 3 / 23

FSC with Feedback and causal state information

General Finite State Channel (FSC) p(yi, si|xi, si−1, yi−1) = P(yi, si|xi, si−1)

Remarks

There is no assumption that the state sequence (S1, S2, ...) is i.i.d. The state is input dependent.

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 4 / 23

Special Cases

The general channel model: P(yi, si|xi, si−1) = P(yi|xi, si−1)P(si|xi, si−1, yi) This setting covers the following specific cases: i.i.d state: p(si−1)p(yi|xi, si−1) Markovian state: p(si|si−1)p(yi|xi, si−1), and look-ahead. (Can feedback increase the capacity?) Unifilar FSC: si = f (xi, si−1, yi) Energy Harvesting

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 5 / 23

Table of Contents

1

The Setting

2

Main Results Theorem- The Capacity Theorem- Q-Graphs Examples of solving specific known problems Open problem example - Energy Harvesting Model

3

Conclusions

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 6 / 23

The Capacity

Theorem (Capacity)

C = lim

N→∞

1 N max

{p(ui|ui−1,yi−1)}N

i=1,

xi=f (ui,si−1) N

• i=1

I(Ui, Ui−1; Yi|Y i−1) (1) is the capacity for some f : U × S → X, with the joint distribution

n

• i=1

p(si, yi|si−1, xi)1{xi =f (ui, si−1)}p(ui|yi−1, ui−1), (2) Lack of cardinality bound of the auxiliary r.v. U. Multi-letter expression. For any fixed cardinality |U| and a fixed function f one can compute achievable rates by 2 main tools.

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 7 / 23

Achievability - Shannon Strategy

Coding is based on a Shannon strategy-like argument, conversion to a new FSC P(si, yi|ui, si−1). P(si, yi|xi, ui, si−1, yi−1) = P(si, yi|x(ui, si−1), si−1) = P(si, yi|ui, si−1)

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 8 / 23

Converse - Sketch of Proof

Any sequence of achievable block codes (2nR, n) has nR

(a)

≤ I(M; Y n) + nǫn where I(M; Y n)

(b)

=

n

• i=1

I(M; Yi|Y i−1)

(c)

=

n

• i=1

I(Ui; Yi|Y i−1) (a) follows from Fano’s inequality, where ǫn → 0 as n → ∞. (b) follows from the chain rule. (c) follows from defining Ui (M, Y i−1).

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 9 / 23

n

• i=1

I(Ui; Yi|Y i−1)

(d)

≤ max

{p(ui|ui−1,yi−1),p(xi|ui,si−1)}n

i=1

n

• i=1

I(Ui; Yi|Y i−1)

(e)

= max

{p(ui|ui−1,yi−1),p(vi),xi=fi(ui,vi,si−1)}n

i=1

n

• i=1

I(Ui; Yi|Y i−1) (d) follows because the objective is determined by {p(ui, yi, si)}n

i=1 = {p(ui, yi, si)}n i=1 due to the definition of

Ui (M, Y i−1) and from the following lemma:

Lemma

p(uk, yk, sk) is determined by {p(ui|ui−1, yi−1)p(xi|ui, si−1)}k

i=1.

(e) follows from the Functional Representation Lemma, i.e. there exists a RV, Vi, independent of (Ui, Si−1), such that Xi can be represented as a function of (Ui, Si−1) and Vi, and from a similar lemma.

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 10 / 23

max

{p(ui|ui−1,yi−1),p(vi),xi=fi(ui,vi,si−1)}n

i=1

n

• i=1

I(Ui; Yi|Y i−1)

(f )

≤ max

{p(˜ ui|˜ ui−1,yi−1),xi=fi(˜ ui,si−1)}n

i=1

n

• i=1

I(˜ Ui; Yi|Y i−1)

(g)

= max

{p(˜ ui|˜ ui−1,yi−1),x=f (si−1,˜ ui,i)}n

i=1

n

• i=1

I(˜ Ui; Yi|Y i−1) (f) follows from defining ˜ Ui (Ui, Vi), from the fact that p(˜ ui|˜ ui−1, yi−1) = p(vi|ui−1, vi−1, yi−1)p(ui|vi, ui−1, vi−1, yi−1). p(vi) and p(ui|vi, ui−1, vi−1, yi−1) are sub-domains of p(vi|ui−1, vi−1, yi−1) and p(ui|ui−1, yi−1) respectively; the mutual information increases since conditioning reduces entropy. (g) follows since there exist an invariant function f (˜ u, s, i) = fi(˜ ui, si−1).

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 11 / 23

max

{p(˜ ui|˜ ui−1,yi−1),x=f (si−1,˜ ui,i)}n

i=1

n

• i=1

I(˜ Ui; Yi|Y i−1)

(h)

≤ max

{p(u′

i |u′ i−1,yi−1),x=f (u′ i ,si−1)}n i=1

n

• i=1

I(U′

i ; Yi|Y i−1)

≤ max

{p(u′

i |u′ i−1,yi−1)},x=f (u′ i ,si−1)n i=1

n

• i=1

I(U′

i , U′ i−1; Yi|Y i−1)

(h) follows from defining U′ = (˜ Ui, T = i) where T represents the time index. Finally, by dividing both sides by n we get R ≤ 1 n max

{p(u′

i |u′ i−1,yi−1)},x=f (u′ i ,si−1)n i=1

n

• i=1

I(U′

i , U′ i−1; Yi|Y i−1)

which completes the proof.

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 12 / 23

The Q-graph

A Q-graph is an irreducible directed graph. Each node Q has |L| different labelled edges. E.g. L = {0, 1, ?} The Q-graph defines a mapping: Φi−1 : Li−1 → Q (or g : Q × L → Q) Each outputs sequence is Q-uantized.

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 13 / 23

Q-Graph Lower Bound

Theorem

(Q-graph lower bound) The feedback capacity of an FSC with causal state at the encoder is lower bounded by Cfb ≥ I(U+, U; Y |Q), ∀Q-graph (3) for any mapping X = f (U+, S) and for all aperiodic inputs PU+|U,Q ∈ Pπ that are BCJR-invariant. The BCJR-invariant property: PS+,U+|Q,Y (s+, u+|q, y) = PS+,U+|Q+(s+, u+|g(q, y)), (4) for all (s+, u+, q, y). Markov chain: (S+, U+) − Q+ − (Q, Y ). Evaluated with the stationary distribution.

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 14 / 23

Example - The Trapdoor channel

State evolution: s+ = s ⊕ x ⊕ y. The probability of the Channel p(yt|xt, st−1): xt st−1 p(yt = 0|xt, st−1) 1 1 0.5 1 0.5 1 1

Lemma

For the Trapdoor channel, there exists a policy that achieves R = log2 1 + √ 5 2 , (5) with |U| = 2 and x = u ⊕ s.

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 15 / 23

Example - The input constrained BEC - (1, ∞)-RLL

State evolution: s = x. The probability of the Channel p(yt|xt, st−1): Binary Erasure Channel (BEC) with erasure probability parameter ǫ.

Lemma

For the input-constrained BEC, there exists a policy that achieves R = max

p

H2(p) p +

1 1−ǫ

, (6) for all ǫ, with |U| = 2 and x = u ∧ ¯ s.

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 16 / 23

Example - The Energy Harvesting (EH) Model

X is constrained by the available energy at each channel use. Bmax− the battery size. Ei ∈ {0, 1}− the arrival process. i.i.d. ∼ Bern(h). If Ei = 1 then the battery is charged after the transmission up to the battery size. The battery state evolves according to: Si = min{Si−1 − Xi + Ei, Bmax}

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 17 / 23

The Binary Energy Harvesting Model (BEHM)

X = {0, 1} Y = {0, 1} Bmax = 1, S = {0, 1} If Si−1 = 0 then Xi = 0. If Si−1 = 1 then Xi ∈ {0, 1}. Noiseless channel Y = X What is the capacity?

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 18 / 23

Graph suspected to be the optimal to solve the BEHM

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 19 / 23

Outperforming the best achievable rates in the literature

Lemma

For the noiseless BEHC with a unit-sized battery and for each harvesting probability distributed i.i.d. ∼Bern(h), h ∈ {0, 0.1, ..., 1}, RQ-graph shown in the table below is an achievable rate. Arrival prob (h) RQ-graph LB UB 0.1 0.23244 0.2317 0.2600 0.2 0.35573 0.3546 0.3871 0.3 0.45539 0.4487 0.4740 0.4 0.53813 0.5297 0.5485 0.5 0.61064 0.6033 0.6164 0.6 0.67835 0.6729 0.6807 0.7 0.74350 0.7403 0.7442 0.8 0.81003 0.8088 0.8101 0.9 0.88458 0.8845 0.8846

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 20 / 23

Table of Contents

1

The Setting

2

Main Results Theorem- The Capacity Theorem- Q-Graphs Examples of solving specific known problems Open problem example - Energy Harvesting Model

3

Conclusions

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 21 / 23

Conclusions

Capacity Q-Graphs Energy Harvesting

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 22 / 23

The End

Thank you!

Shemuel, Sabag, Permuter (BGU) Los Angeles, California, USA ISIT 2020 23 / 23