The Shannon-McMillan theorem (AEP) for quantum sources and related - - PowerPoint PPT Presentation

The Shannon-McMillan theorem (AEP) for quantum sources and related topics

I.Bjelakovic, T.K., A. Szkola, R.Siegmund-Schultze

Motivation

• Transfer of fundamental theorems of classical

information theory to quantum information theory

• In a wider context: how a quantum ergodic theory

and quantum dynamical system theory looks like

The classical Shannon-McMillan-(Breiman) theorem

• Given (Σ,µ,σ), Σ sequence space over finite alphabet,

µ ergodic measure, σ shift-transformation, Σ x, x(n) = (x1,, x2, x3,…, xn)

• a.s. for ergodic µ: the individual information rate

equals the average information rate

• This is a law of large numbers under very mild

assumptions

( )

( )

( ) ( )

( )

n

n n w

log x n 1 lim h lim w log w n n

µ →∞ →∞ ∈Σ

− µ   − = = µ µ    

∑

Typical subspaces and data compression

Reformulation in terms of typical subspaces:

( )

{ }

( ) ( ) ( )

( )

( )

n n n+1 n n n n h n

there is a family of typical sets s.t. (filtration property) and and and

• ne

T T n T T 1 ha 1 log #T h n w T e n n s: for

µ

µ − −ε

⊂ Σ ⊃ µ → → ∀ε > µ ∈ ≤ > ε

{ } ( ) ( )

n n n nh

strong converse in other furthermore for any family s.t. it follows that ( ) : to cover a positive fraction of the whole space one needs asymptot w i

• r

ca B 1 limsup log #B h n B e lly cylinder ds

• sets

µ

µ

< µ →

• f lengt n

( )

n

Σ

( )

n n n

T typical subspace for : T 1 − µ µ > − ε

( )

n 1 +

Σ

n 1

T typical subspace

+ −

( )

n 2 +

Σ

( )

n 3 +

Σ

n 2

T typical subspace

+ − n 3

T typical subspace

+ −

( ) { }

n 1 2 n 2 kh

given a typical long symbol sequence : typical words of length x ,x ,.......,x 0,1 1 k k lo , there are abou g n h 2 Ap t plication to da typical ta compression: Codeboo words Codebooksize and n k bits kh

µ

µ µ

∈ ⇒ <

• (

)

( )

1 2 k k 1 2k n k n 1 1 code with kh bits code code n codewords k

x ,x ,...,x x ,....,x ..............x ,... Spl needed to specify a word .,x from the codebook : (only fr

• n

i act tting io

µ

+ − +

• [

]

n of blocks does not belong to the codebook) bits needed to code the whole sequence ( n kh nh h 0,1 k )

µ µ µ

⋅ = ∈ ⇒

The quantum setting

{ }

{ }

x n x x 1,2,...,n n

: matrix-a A H= C* A A x A A : A shif lgebra over Hilbert space t transformat (

• algebra)

: copy of at site norm-closure of : : positive, normed, linear functional on (me A ion i asure) nvari

κ ∞ ∈ ∞

= = ⊗ σ ϕ ϕ

• :

: is extremal among the invar an ia t erg nt f

• di

unctiona c ls ϕ = ϕ σ ϕ ϕ

( ) ( ) ( ) ( ) ( ) ( )

n

n n A n n n 1 n 1 n+1 n n n n n

for there is a s.t. and (consistency) ( partial density matrix entro : D a tr D a D tr D t py entropy rate co r n+1 S tr D lo trace with respect to site ) : ( g D 1 s lim von Neuma ) S n v nn :

+ + →∞

ϕ = ϕ ϕ = = ϕ = − ϕ = ϕ

( ) ( )

{ }

n n

: 1 lim min log t : project ering expo

• r fr

rP : P A

• m

s.t P 1 n . nent

→∞

β ε ϕ > − ε

The quantum Shannon-McMillan theorem

(Ref.: Inventiones Mathematica, 2003)

{ }

( ) ( ) ( ) { } ( ) ( )

n n n n n n n n

A Q A 1 Q 1 lim log tr Q s n i) ii Let be an ergodic state on family of orthogonal projectors s.t.: and for any sequence of minimal projectors p Q 1log p s n for any seque ) ii n e i) c o

∞ →∞

ϕ ∈ ϕ → = ϕ < − ϕ → ϕ ⇒ ∃ ⇒

{ }

( ) ( ) ( )

n n n n n

Q A 1 li f pr m lo

• jectors

s.t. g tr Q s Q n

→∞

′ ∈ ′ ϕ → ⇒ ′ < ϕ

Comments

• The theorem holds for
• lattices as well
• Covering exponent is for all ε > 0: β(ε) = s(φ)
• The typical projectors (subspaces) can be

explicitly constructed from the eigenspaces of Dn corresponding to eigenvalues of order

• The relation between the typical subspaces for

different n is still unclear

• Extensions to other group actions are possible
• The typical subspaces can be chosen to be

universal (not depending on φ but only on s(φ)) due to a result by Kalchenkov

( )

ns

e

− ϕ

ν

History

• Josza&Schumacher: typical subspace theorem for

product states (Bernoulli case, 1996)

• Petz&Mosonyi: weak version of the Shannon-

McMillan under the assumption of complete ergodicity (2001) and strong form for Gibbs states (with Hiai, 1993)

• Neshveyev&Størmer: Shannon-McMillan for

finitely generated C*-algebras but only tracial states (2002)

• Datta&Shuchov: Shannon-McMillan for spin

lattices with restrictions on the interaction (2002)

Extensions { }

( ) ( )

( )

n n, n, n, n

Let be an ergodic state on family of orthogonal projectors A Q A n n( A pointwise variant (Shan ) s.t. for : and for any non-McMillan-Brei 1 Q 1 li s man eq ): i m lo ) ii n u g tr Q s )

∞ ε ε ε →∞

ϕ ∀ε > ∈ > ε ϕ > − ε ∃ < ⇒ ϕ + ε

{ }

( ) ( )

n n, n n+1 n+1, n,

p Q 1log p s n R[tr (Q )] ence of minimal projectors (her Q R[.] e is the i ii) range projector)

ε ε ε

< − ϕ < ϕ − ε = ⇒

• The relation between the typical projectors for

different ε is unclear

• For abelian algebras (classical case) the above

theorem is equivalent to the Shannon-McMillan- Breiman theorem

A theorem for the relative entropy

(I.Bjelakovich, R.Siegmund-Schultze)

( ) ( )

( )

tr D log D log D R supp supp

• f two states and on finite dimensional algebra:

for

• therwise

: an invariant state and an invariant product stat elative entropy Relative entropy rat S e e ,

• :

ω ω τ

ω τ  − ω ≤ τ  ω τ =  ∞   ψ ϕ

( ) ( )

n

n n n A n

A 1 s , lim S , : | n n ( )

∞ →∞

ψ ϕ = ψ ϕ ϕ = ϕ

( ) ( ) ( )

{ }

( ) ( )

n ,n n n ,n n

projector s.t. For an ergodic state and an invariant product state on , : min log Q :Q A , Q 1 A 1 l Rel for equivalently for typical subspace pr ative exponent im , s ,

• ject

n :

ε ∞ ε →∞

⇒ β ψ ϕ = ϕ ∈ ψ > − ε ψ ϕ β ψ ϕ = ψ ϕ ∀ε >

{ }

( )

( )

( )

( )

( )

( )

n n s , n s , n

• rs
• f

Q e Q e ; n n

− ψ ϕ +ε − ψ ϕ −ε

⇒ ψ ≤ ϕ ≤ > ε

Relative entropy typical and untypical subspaces

ψ

( )

( )

ns , n

From point of vi : Q ew e

− ψ ϕ

ϕ ϕ

• typical subspace for ψ

( )

n

From point of view: Q 1 ψ ψ > −ε

( )

n n

Q T ψ

• ( )

n n

Q T ψ

• Complete analogy to the classical case
• The proof is similar to the one for the Shannon-

McMillan theorem but more technical involved

• New simple proof of the monotonicity of the relative

entropy can be derived from this result

• Starting point for developing a large deviation theory

(Sanov’s theorem)

Proof strategy

n n

Idea Natural c : want to u andida se abelian approximations to lift the classical results to the quantum case : algebra generated by the eigenspace projectors

• f

(density matrix corresp B t D e

• ndi

{ } ( )

( )

n n n n n n m n n

n 1,..,n B A B A B A B B B * n * n

ng to ) is : A A ...... A A A ...... A ..... A A an abelian syste ...... A B , , A m corresponds to

• n

∞

⊗ ⊗ ⊗ ⊂ ⊂ ⊂ → ∞ ∞

ϕ = ϕ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ϕ σ σ σ

( )

( )

( )

( )

( )

( )

( ) ( )

( )

n n

B * n B B * B

is isomorphic to a classical system What can be said about the

• f

? is ergodic under the assumption of

• ergodic

B , , , , 1 1 s s h s n n n properties complete ergodicity Pet , , f ( ) z

∞ µ

ϕ σ Σ µ σ ϕ ≤ ϕ σ = ≤ ϕ + ε Σ µ σ µ ϕ In the general case splits into at most . All components are isomorphic under some shift-power and have the same entropy. To prove this one needs an ergodic d k n ec ergodic co

• mposition

mpon theo ents rem µ

( )

n

A , , fo r :

∞ ϕ σ

( )

( )

{ }

( ) ( ) ( )

( )

( )

( )

( ) ( )

( )

( )

( )

n

i n 1 i k i i-1 i j n n i A

splits into ergodic components and finite i) ii) iii) iv) size entropy estimation: a A , , 1 k n k n s s ns 1 n s S s n nd for almost every ergodic com Ne ponen x s t t t

∞ ≤ ≤

ϕ σ ≤ ≤ ϕ ϕ = ϕ σ ϕ σ = ϕ σ = ϕ ∀η > → ∞ ϕ ≤ ϕ ≤ ϕ + ⇒ η

• : combining the different levels of approximation

ep

( ) ( ) ( ) ( )

{ }

n n n n ,n n

Lemma : a) B 1 log# given a sequence of probability measures

• ver finite alphabets

s.t. B C n 1 H h n 1 limsup : mi b) n log# : 1 h for ) n c

ε

µ ≤ < ∞ µ →   β = Ω µ Ω > − ε ≤ ∀ε >     ⇒

( )

( )

{ }

( )

( )

{ }

( )

,n n n h n n n n h n n n n n

for take and as the index set of the projectors correspond 1 lim h n a B : a e a B : a e ing to the eigenspaces of and apply the results about i) i t h s B D i) iii) he e

ε →∞ − −ε − +ε

∀ε > β = µ ∈ µ > → µ ∈ µ < → = ⇒ ϕ rgodic decomposition and mix everything carefully! For the proof of the relative entropy theorem one needs simultaneous good abelian approximations of the states and ψ ϕ

A coding application

(I.Bjelakovich, A.Szkoła)

( ) ( )

Is the projection onto the typical subpaces a quantum operation with asumptotic fidelity 1? A is a trace preserving completely positive map B H Que B sti H' from , : f

• n:

quantum chann inite dimension e i H l al H →

( ) ( )

{ }

( ) ( )

{ }

( )

( )

( )

( )

( ) ( )

( )

( )

n n n n n n n n n n n n

lbertspace for stationary quantum Compression sch , A , , A B H ,D : B H em B H H : B source e H B H

∞ ⊗ ⊗ ⊗ ⊗

ϕ σ ≅ = → ⊂ → C D C D

( )

( )

( ) ( )

( )

( ) ( ) ( )

( )

2 n n n ' n n n

• f two density matrices and :

(generalizes the overlap

• f vectors in a Hil

Fidelity Compression rate: F , t bert space) How r 1 1 F , tr 1 F , 2 logdim H R : limsup n F large i , D s D D ρ τ ρ τ = ρ τ ρ ψ ξ − ρ τ ≤ ρ − τ ≤ − ρ τ = =

• C

D C for given and la R rge ? n

C

( ) ( )

{ }

( )

( )

( )

( )

n n ' n n n ' n n n

, R s lim F D ,D 1 R s li there is a compression s m F D ,D cheme with s.t. any scheme with satisfies similar statements hold for stronger versio Theorem: i) ii ns of fidelity (entangl ) emen

→∞ →∞

= ϕ = < ϕ =

C C

C D t fidelity, ensemble fidelity)

Open problems

• Stronger pointwise theorem
• Estimation of entropy
• Universal coding schemes (unknown source)
• Lempel-Ziv type coding
• Rate distortion
• Coding theorems for different channels
• Large deviations, Sanov‘s theorem
• Isomorphism classes etc. ( are q-Bernoulli systems

completely classified by the entropy?)