Distributed Statistical Inference using Type Based Random Access - - PowerPoint PPT Presentation

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Distributed Statistical Inference using Type Based Random Access over Multi-access Fading Channels

Animashree Anandkumar and Lang Tong

Adaptive Communication and Signal Processing Group School of Electrical and Computer Engineering Cornell University, Ithaca, NY 14853. Supported by the MURI under Office of Naval Research and Army Research Laboratory CTA 03/22/2006. Anima Kumar at CISS ’06 1 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Classical Distributed Inference

• Node 1

Node 2 C1 C2 Cn Fusion Node n Center

• Sensors : Sense physical phenomenon and

transmit their local decisions.

• Fusion Center: Make inference on the

phenomenon.

• Sensor-Fusion Center Communication

Perfect (Error - free) with rate constraints.

• Typically in Radar communication.

Key Issues

• Quantization @ sensors.
• Inference @ fusion center.

Anima Kumar at CISS ’06 2 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Classical Distributed Inference

• Node 1

Node 2 C1 C2 Cn Fusion Node n Center

• Sensors : Sense physical phenomenon and

transmit their local decisions.

• Fusion Center: Make inference on the

phenomenon.

• Sensor-Fusion Center Communication

Perfect (Error - free) with rate constraints.

• Typically in Radar communication.

Key Issues

• Quantization @ sensors.
• Inference @ fusion center.

Anima Kumar at CISS ’06 2 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Inference in Large Wireless Sensor Networks

Mobile Access Cluster head

Characteristics

• Low Power and Low Rate Transmissions.
• Bandwidth Allocation.
• Multiaccess Channel with Fading.
• Energy Efficiency to prolong network

life-time.

• Faulty, sleeping or poorly placed sensors.
• Deterministic scheduling (TDMA) may

not be appropriate. Medium Access Design is a key component.

Anima Kumar at CISS ’06 3 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Inference in Large Wireless Sensor Networks

Mobile Access Cluster head

Characteristics

• Low Power and Low Rate Transmissions.
• Bandwidth Allocation.
• Multiaccess Channel with Fading.
• Energy Efficiency to prolong network

life-time.

• Faulty, sleeping or poorly placed sensors.
• Deterministic scheduling (TDMA) may

not be appropriate. Medium Access Design is a key component.

Anima Kumar at CISS ’06 3 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Random Access

• Model : Random Number of Sensors in a data collection.
• Probabilistic Wake-up : Transmit based on a coin-flip.
• Transmit only Significant Data.
• Fusion center is a Mobile Access Point : collects data from

different geographic regions.

Anima Kumar at CISS ’06 4 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Random Access

• Model : Random Number of Sensors in a data collection.
• Probabilistic Wake-up : Transmit based on a coin-flip.
• Transmit only Significant Data.
• Fusion center is a Mobile Access Point : collects data from

different geographic regions.

Anima Kumar at CISS ’06 4 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Distributed Inference over Multi-Access Channels

Detection ( Binary Hypothesis) and Estimation. Random number of sensors per collection Ni are IID with mean λ. Sensor Quantization: Xi,j quantized to M levels and Conditionally IID given θ Xij ∼ pθ = (pθ(1), · · · , pθ(M)) Multi-access model

• Flat IID fading: Hi,j
• AWGN W (t) with PSD =σ2.

Inference at Fusion Center

• Neyman Pearson or Bayesian Detection.
• Maximum Likelihood Estimation.

Physical Phenomenon

1 n 2

Sensor Sensor Sensor θ ∈ Θ

Multiple collections: i–time index, j–sensor index.

Anima Kumar at CISS ’06 5 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Distributed Inference over Multi-Access Channels

Detection ( Binary Hypothesis) and Estimation. Random number of sensors per collection Ni are IID with mean λ. Sensor Quantization: Xi,j quantized to M levels and Conditionally IID given θ Xij ∼ pθ = (pθ(1), · · · , pθ(M)) Multi-access model

• Flat IID fading: Hi,j
• AWGN W (t) with PSD =σ2.

Inference at Fusion Center

• Neyman Pearson or Bayesian Detection.
• Maximum Likelihood Estimation.

Physical Phenomenon

1 n 2

Xi,1 Xi,2 Xi,ni Sensor Sensor Sensor θ ∈ Θ

Multiple collections: i–time index, j–sensor index.

Anima Kumar at CISS ’06 5 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Distributed Inference over Multi-Access Channels

Detection ( Binary Hypothesis) and Estimation. Random number of sensors per collection Ni are IID with mean λ. Sensor Quantization: Xi,j quantized to M levels and Conditionally IID given θ Xij ∼ pθ = (pθ(1), · · · , pθ(M)) Multi-access model

• Flat IID fading: Hi,j
• AWGN W (t) with PSD =σ2.

Inference at Fusion Center

• Neyman Pearson or Bayesian Detection.
• Maximum Likelihood Estimation.

Physical Phenomenon

1

Multiaccess Channel

n 2

Xi,1 Xi,2 Xi,ni Sensor Sensor Sensor θ ∈ Θ Yi Noise W

Multiple collections: i–time index, j–sensor index.

Anima Kumar at CISS ’06 5 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Distributed Inference over Multi-Access Channels

Detection ( Binary Hypothesis) and Estimation. Random number of sensors per collection Ni are IID with mean λ. Sensor Quantization: Xi,j quantized to M levels and Conditionally IID given θ Xij ∼ pθ = (pθ(1), · · · , pθ(M)) Multi-access model

• Flat IID fading: Hi,j
• AWGN W (t) with PSD =σ2.

Inference at Fusion Center

• Neyman Pearson or Bayesian Detection.
• Maximum Likelihood Estimation.

Physical Phenomenon

1

Multiaccess Channel

n 2

Xi,1 Xi,2 Xi,ni Sensor Sensor Sensor

Detector / Estimator Fusion Center

θ ∈ Θ Yi Noise W ˆ θ

Multiple collections: i–time index, j–sensor index.

Anima Kumar at CISS ’06 5 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Spatio-Temporal Tradeoff

• Mean Transmitting Rate λ and Number of Data Collections l.
• Suppose we fix mean number of transmissions is ρ∆

=λl, (proportional to energy budget).

• Should energy be allocated to simultaneous transmissions :

large λ ?

• Or should we collect more data : large l ?
• Small λ : not enough sensors transmit , cannot counter noise.
• But, large λ : Less Observations as l is small.
• Role of multi access channel : Coherence or Cancelation ?

Number of collections l Transmission Rate λ ρ = λl

Anima Kumar at CISS ’06 6 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Spatio-Temporal Tradeoff

• Mean Transmitting Rate λ and Number of Data Collections l.
• Suppose we fix mean number of transmissions is ρ∆

=λl, (proportional to energy budget).

• Should energy be allocated to simultaneous transmissions :

large λ ?

• Or should we collect more data : large l ?
• Small λ : not enough sensors transmit , cannot counter noise.
• But, large λ : Less Observations as l is small.
• Role of multi access channel : Coherence or Cancelation ?

Number of collections l Transmission Rate λ ρ = λl

Anima Kumar at CISS ’06 6 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Spatio-Temporal Tradeoff

• Mean Transmitting Rate λ and Number of Data Collections l.
• Suppose we fix mean number of transmissions is ρ∆

=λl, (proportional to energy budget).

• Should energy be allocated to simultaneous transmissions :

large λ ?

• Or should we collect more data : large l ?
• Small λ : not enough sensors transmit , cannot counter noise.
• But, large λ : Less Observations as l is small.
• Role of multi access channel : Coherence or Cancelation ?

Number of collections l Transmission Rate λ ρ = λl

Anima Kumar at CISS ’06 6 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Fading Coherence Index γ

• Define Fading Coherence index as

γ = |E(H)|2

Var(H) .

• Non Coherent ( γ = 0 ) : Uniform phase uncertainty

(e.g.,Rayleigh.)

• Perfectly Coherent ( γ = ∞ ) : Deterministic Channel or no

fading.

Anima Kumar at CISS ’06 7 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Outline

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Anima Kumar at CISS ’06 8 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Outline

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Anima Kumar at CISS ’06 9 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Type Based Random Access

• Signal Waveform

S1(t), . . . , SM(t)—a pre-determined set of M

• rthogonal waveforms with energy constraint E.
• Sensor Encoding

Quantized Data Xi,j = x is encoded to waveform Sx(t) Si,j(t; x) = Sx(t)

• Observation @ FC:

Yi(t) =

Ni

j=1

Hi,j √ ESXi,j (t − τi,j) + Wi(t).

• Narrow band signal assumption :

Si,j(t − τi,j) ≈ Si,j(t).

• Data-centric in contrast to user-centric

schemes (TDMA, FDMA or CDMA).

Channel

Sensor Sensor Sensor

Multiaccess

θ ∈ Θ Xi,1 Xi,2 Xi,n 1 2 ni SXi,1(t) SXi,n(t)

Hi,1 Hi,n

Yi(t) ˆ θ Detector / Estimator at Fusion Center

Key: MAC adds transmissions with same data level

Anima Kumar at CISS ’06 10 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Matched Filter Output

Schematic of TBRA

X1 = 1 X2 = 2 X3 = 1 X4 = 1 X5 = 3 Sr(t) Sr(t) Sr(t) Sb(t) Sg(t) nr = 3 nb = 1 ng = 1

Ideal Conditions with λ = 5 Matched Filtering Yi

∆

= 1 √ E

• Yi(t), S1(t)
• , · · · ,
• Yi(t), SM(t)
• i

=

Ni

j=1

Hi,jeXi,j + Wi, Wi

iid

∼ N (0, σ2 E I) where eX1, · · · , eXM are basis vectors. Ideal Conditions (Deterministic Ni ≡ λ, Hi,j ≡ 1 or γ = ∞ and Wi ≡ 0) Yi =

λ

j=1

eXi,j . jth entry of Yi : no. of sensors quantizing to level j.

Yi λ

gives Type or Empirical Distribution of Xi,j.

Anima Kumar at CISS ’06 11 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Matched Filter Output

Schematic of TBRA

X1 = 1 X2 = 2 X3 = 1 X4 = 1 X5 = 3 Sr(t) Sr(t) Sr(t) Sb(t) Sg(t) nr = 3 nb = 1 ng = 1

Ideal Conditions with λ = 5 Matched Filtering Yi

∆

= 1 √ E

• Yi(t), S1(t)
• , · · · ,
• Yi(t), SM(t)
• i

=

Ni

j=1

Hi,jeXi,j + Wi, Wi

iid

∼ N (0, σ2 E I) where eX1, · · · , eXM are basis vectors. Ideal Conditions (Deterministic Ni ≡ λ, Hi,j ≡ 1 or γ = ∞ and Wi ≡ 0) Yi =

λ

j=1

eXi,j . jth entry of Yi : no. of sensors quantizing to level j.

Yi λ

gives Type or Empirical Distribution of Xi,j.

Anima Kumar at CISS ’06 11 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Matched Filter Output

Schematic of TBRA

X1 = 1 X2 = 2 X3 = 1 X4 = 1 X5 = 3 Sr(t) Sr(t) Sr(t) Sb(t) Sg(t) nr = 3 nb = 1 ng = 1

Ideal Conditions with λ = 5 Matched Filtering Yi

∆

= 1 √ E

• Yi(t), S1(t)
• , · · · ,
• Yi(t), SM(t)
• i

=

Ni

j=1

Hi,jeXi,j + Wi, Wi

iid

∼ N (0, σ2 E I) where eX1, · · · , eXM are basis vectors. Ideal Conditions (Deterministic Ni ≡ λ, Hi,j ≡ 1 or γ = ∞ and Wi ≡ 0) Yi =

λ

j=1

eXi,j . jth entry of Yi : no. of sensors quantizing to level j.

Yi λ

gives Type or Empirical Distribution of Xi,j.

Anima Kumar at CISS ’06 11 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Role of Coherence Index γ for TBRA

• MAC adds transmissions with same

data level

• Large γ : Better addition of signals

in the mean since EYi = λE(H)pθ.

• Small γ : Effect of sensor data is
• nly a second order effect (through

the Channel Variance).

Channel

Sensor Sensor Sensor

Multiaccess θ ∈ Θ Xi,1 Xi,2 Xi,n 1 2 ni SXi,1(t) SXi,n(t)

Hi,1 Hi,n

Yi(t) ˆ θ Detector / Estimator at Fusion Center

γ = |E(H)|2

Var(H) .

Anima Kumar at CISS ’06 12 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Outline

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Anima Kumar at CISS ’06 13 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Spatio-Temporal Tradeoff

• Mean no. of transmissions ρ = λl fixed.
• Performance Metric Mρ,λ.
• Optimal allocation in λ and l

λ∗(ρ) = arg sup

λ

Mρ,λ.

• For finite ρ, Mρ,λ intractable in our

setup.

Number of collections l Transmission Rate λ ρ = λl Performance Metric Mρ,λ sup

λ

Mρ,λ Optimal Allocation λ∗(ρ) Anima Kumar at CISS ’06 14 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Asymptotic Performance Metric M(λ)

• Metric : Asymptotic (in ρ) Performance.

M(λ)∆ = lim

ρ→∞ Mρ,λ.

• Goal : Existence of a optimal λ∗ such that

λ∗ = arg max

λ

M(λ).

• λ∗ is bounded : Cancelation, Avoid Interference.
• λ∗ is unbounded : Coherence, Simultaneous Transmissions.

M(λ) λ λ∗

Anima Kumar at CISS ’06 15 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Performance Metric M(λ)

Detection

• Performance Metric is Detection error

exponent M(λ)∆ = − lim

ρ→∞

1 ρ log Pe(ρ, λ), where Pe(ρ, λ) is detection error probability.

• Under Neyman-Pearson or Bayesian setting,

MNP(λ) = 1 λDλ(f0||f1), MB(λ) = 1 λCλ(f0, f1), Dλ(f0||f1) : Kullback-Leibler distance. Cλ(f0, f1) : Chernoff information.

Estimation

• Define Performance Metric

M(λ)∆ = Iλ(θ) λ , Iλ(θ) : Fisher Information of Yi.

• Cramer Rao Bound for any unbiased

estimator ˆ θ based on ρ mean no. of transmissions and transmission rate λ,

Var(ˆ

θ − θ) ≥ λ ρIλ(θ).

• Asymptotic Efficiency of ML estimator

√ρ(ˆ θMLE − θ) d → N (0, λ Iλ(θ) ).

Anima Kumar at CISS ’06 16 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Performance Metric M(λ)

Detection

• Performance Metric is Detection error

exponent M(λ)∆ = − lim

ρ→∞

1 ρ log Pe(ρ, λ), where Pe(ρ, λ) is detection error probability.

• Under Neyman-Pearson or Bayesian setting,

MNP(λ) = 1 λDλ(f0||f1), MB(λ) = 1 λCλ(f0, f1), Dλ(f0||f1) : Kullback-Leibler distance. Cλ(f0, f1) : Chernoff information.

Estimation

• Define Performance Metric

M(λ)∆ = Iλ(θ) λ , Iλ(θ) : Fisher Information of Yi.

• Cramer Rao Bound for any unbiased

estimator ˆ θ based on ρ mean no. of transmissions and transmission rate λ,

Var(ˆ

θ − θ) ≥ λ ρIλ(θ).

• Asymptotic Efficiency of ML estimator

√ρ(ˆ θMLE − θ) d → N (0, λ Iλ(θ) ).

Anima Kumar at CISS ’06 16 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Outline

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Anima Kumar at CISS ’06 17 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Theorem on Existence of optimal λ

Under Regularity Conditions in the paper, Non Coherent Channels (E(H) = 0 or γ = 0) : Existence of Bounded optimal λ∗ lim

λ→0 M(λ) =

lim

λ→∞ M(λ) = 0,

which implies that there exists 0 < λ∗ < ∞ such that sup

λ

M(λ) = 1 λ∗ Iλ∗(θ). Deterministic Channels (Var(H) = 0 or γ = ∞) : No Bounded optimizing λ∗, M(λ) = Θ(λ) , λ → ∞, M(λ) λ γ = 0 λ∗

γ = |E(H)|2

Var(H) .

Anima Kumar at CISS ’06 18 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Theorem on Existence of optimal λ

Under Regularity Conditions in the paper, Non Coherent Channels (E(H) = 0 or γ = 0) : Existence of Bounded optimal λ∗ lim

λ→0 M(λ) =

lim

λ→∞ M(λ) = 0,

which implies that there exists 0 < λ∗ < ∞ such that sup

λ

M(λ) = 1 λ∗ Iλ∗(θ). Deterministic Channels (Var(H) = 0 or γ = ∞) : No Bounded optimizing λ∗, M(λ) = Θ(λ) , λ → ∞, M(λ) λ γ = ∞

γ = |E(H)|2

Var(H) .

Anima Kumar at CISS ’06 18 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

M(λ) for different Coherence Indices.

M(λ) λ γ = 0 low γ high γ γ = ∞ ? λ∗

γ = |E(H)|2

Var(H) .

Anima Kumar at CISS ’06 19 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Numerical Plots of Performance Metric

5 10 15 20 25 0.01 0.02 0.03 0.04 0.05 0.06

γ = 0 γ = 0.05 γ = 0.5 M(λ) λ

( SNR = 7db, σ2

H = 0.1 ) 5 10 15 0.005 0.01 0.015 0.02 0.025

SNR = −4db SNR = 0db SNR = 2db

M(λ) λ

( γ = 0, σ2

H = 1 )

Normalized Chernoff Information vs. Transmission Rate.

Anima Kumar at CISS ’06 20 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Asymptotic Normal Distribution

• To analyze asymptotic behavior of M(λ) : compute

Performance metric for limiting distribtution ˜ M(λ).

• Since by continuity lim

λ→∞ M(λ) = lim λ→∞

˜ M(λ).

• CLT for Random Number of Summands :

Y − λE(H)pθ √ λ

d

→ N

• 0, Var(H)Diag(pθ)
• as

λ → ∞.

• Gaussian Metric ˜

M(λ): closed form expressions.

• For Large λ, approximate actual M(λ) by Gaussian ˜

M(λ).

Anima Kumar at CISS ’06 21 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Asymptotic Normal Distribution

• To analyze asymptotic behavior of M(λ) : compute

Performance metric for limiting distribtution ˜ M(λ).

• Since by continuity lim

λ→∞ M(λ) = lim λ→∞

˜ M(λ).

• CLT for Random Number of Summands :

Y − λE(H)pθ √ λ

d

→ N

• 0, Var(H)Diag(pθ)
• as

λ → ∞.

• Gaussian Metric ˜

M(λ): closed form expressions.

• For Large λ, approximate actual M(λ) by Gaussian ˜

M(λ).

Anima Kumar at CISS ’06 21 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Gaussian Approximation

5 10 15 0.005 0.01 0.015 0.018

SNR = −4db SNR = 0db SNR = 2db

˜ M(λ) λ

Gaussian Metric vs. Transmission Rate. ( γ = 0, σ2

H = 1 )

−6 −4 −2 2 4 6 5 10 15 20 25

Numerical Eval. Gaussian Approx.

SNR in db λ∗

Optimal λ∗ vs. SNR in db. ( γ = 0, σ2

H = 1 )

Anima Kumar at CISS ’06 22 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Outline

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Anima Kumar at CISS ’06 23 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Summary

• Introduced TBRA : removes requirement of channel coherency

and handles random number of sensors.

• Provided a general characterization of Performance metric of

estimation and provided approximate solutions.

• Proved the existence of optimal spatio-temporal allocation

scheme dependent on Channel Coherence Index.

Anima Kumar at CISS ’06 24 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Related Publication

• A. Anandkumar and L. Tong, A Large Deviation Analysis of

Detection over Multi-Access Channels with Random Number

• f Sensors, accepted to Proc. of ICASSP 06, Toulouse,

France, May 2006.

• A. Anandkumar and L. Tong, Type-Based Random Access for

Distributed Detection over Multiaccess Fading Channels, Submitted to IEEE Trans. Signal Proc., Dec. 2005.

• A. Anandkumar, L. Tong and A. Swami, Large deviation

analysis of Sequential distributed detection using Type based Random Access, To be submitted to Proc. of EUSIPCO, Sep. 2006.

Anima Kumar at CISS ’06 25 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

References

• G. Mergen, V. Naware, and L. Tong, Asymptotic Detection Performance of

Type-Based Multiple Access Over Multiaccess Fading Channels, submitted to IEEE Trans. on Signal Processing, May 2005.

• Ke Liu and A. M. Sayed, Optimal distributed detection strategies for wireless

sensor networks,, in 42nd Annual Allerton Conf. on Commun., Control and Comp., Oct. 2004.

• J.-F. Chamberland and V. V. Veeravalli, Asymptotic results for decentralized

detection in power constrained wireless sensor networks, IEEE JSAC Special Issue on Wireless Sensor Networks, 2004.

• S.A. Aldosari and J.M.F. Moura, Detection in decentralized sensor networks, in
• Proc. of ICASSP 04 Conf., Montreal, Canada.
• B. Chen, R. Jiang, T. Kasetkasem, and P.K. Varshney, Channel aware decision

fusion in wireless sensor networks, IEEE Trans. on Signal Processing, vol. 52,

• Dec. 2004.

Anima Kumar at CISS ’06 26 / 27

Type Based Random Access Performance Metric Optimal TBRA Conclusion

Thank You !

Anima Kumar at CISS ’06 27 / 27