Heat Equation Fan Cheng John Hopcroft Center Computer Science and - - PowerPoint PPT Presentation

β–Ά
heat equation
SMART_READER_LITE
LIVE PREVIEW

Heat Equation Fan Cheng John Hopcroft Center Computer Science and - - PowerPoint PPT Presentation

Mar 30, 2023 359 likes β€’631 views

[Pop-up Salon, Maths, SJTU, 2019/01/10] On the Complete Monotonicity of Heat Equation Fan Cheng John Hopcroft Center Computer Science and Engineering Shanghai Jiao Tong University chengfan@sjtu.edu.cn Overview 2 2 2 ,

slide-1
SLIDE 1

On the Complete Monotonicity of Heat Equation

Fan Cheng John Hopcroft Center Computer Science and Engineering Shanghai Jiao Tong University

chengfan@sjtu.edu.cn

[Pop-up Salon, Maths, SJTU, 2019/01/10]

slide-2
SLIDE 2

β„Ž π‘Œ + π‘’π‘Ž

πœ–2 2πœ–π‘¦2 𝑔 𝑦, 𝑒 = πœ– πœ–π‘’ 𝑔(𝑦, 𝑒) β„Ž π‘Œ = βˆ’βˆ« 𝑦log𝑦 d𝑦 𝑍 = π‘Œ + π‘’π‘Ž π‘Ž ∼ π’ͺ(0,1)

CMI = 2 (H. P. McKean, 1966) CMIβ‰₯ πŸ“. Conjecture: CMI = +∞ (Cheng, 2015)

(Heat equation) (Gaussian Channel)

Overview

slide-3
SLIDE 3

β–‘ β€œSuper-H” Theorem β–‘ Boltzmann equation and heat equation β–‘ Shannon Entropy Power Inequality β–‘ Complete Monotonicity Conjecture

Outline

slide-4
SLIDE 4

Fire and Civilization

Drill Steam engine James Watts Myth: west and east Independence of US The Wealth of Nations 1776

slide-5
SLIDE 5

Study of Heat

Heat transfer

β–‘ The history begins with the work of Joseph Fourier around 1807 β–‘ In a remarkable memoir, Fourier invented both Heat equation and the method of Fourier analysis for its solution

πœ– πœ–π‘’ 𝑔 𝑦, 𝑒 = 1 2 πœ–2 πœ–π‘¦2 𝑔(𝑦, 𝑒)

slide-6
SLIDE 6

Information Age

π‘Žπ‘’ ∼ π’ͺ(0, 𝑒)

Gaussian Channel: X and Z are mutually independent. The p.d.f of X is g(x) Y is the convolution of X and : Y = X+ The p.d.f. of Y π‘Žπ‘’ 𝑔(𝑧; 𝑒) = ∫ 𝑕(𝑦) 1 2πœŒπ‘’ 𝑓

(π‘§βˆ’π‘¦)2 2𝑒

πœ– πœ–π‘’ 𝑔(𝑧; 𝑒) = 1 2 πœ–2 πœ–π‘§2 𝑔(𝑧; 𝑒) π‘Žπ‘’ Fundamentally, Gaussian channel and heat equation are identical in mathematics (Gaussian mixture model) A mathematical theory of communication, Bell System Technical

  • Journal. 27 (3): 379–423.
slide-7
SLIDE 7

Entropy Formula

Entropy Second law of thermodynamics: one way only

slide-8
SLIDE 8

Ludwig Boltzmann

Boltzmann formula: Boltzmann equation: H-theorem:

Ludwig Eduard Boltzmann 1844-1906 Vienna, Austrian Empire 𝑇 = βˆ’π‘™πΆln𝑋 𝑇 = βˆ’π‘™π‘βˆ‘

𝑗

π‘žπ‘—lnπ‘žπ‘— 𝑒𝑔 𝑒𝑒 = (πœ–π‘” πœ–π‘’)force + (πœ–π‘” πœ–π‘’)diff + (πœ–π‘” πœ–π‘’)coll 𝐼(𝑔(𝑒))is nonβˆ’decreasing

Gibbs formula:

slide-9
SLIDE 9

β–‘ Notation

  • A function is completely monotone (CM) iff all the signs of its

derivatives are: +, -, +, -,…… (e.g.,

1 𝑒 , π‘“βˆ’π‘’)

β–‘ McKean’s Conjecture on Boltzmann equation (1966):

β–‘ 𝐼(𝑔(𝑒)) is CM in 𝑒, when 𝑔 𝑒 satisfies Boltzmann equation β–‘ False, disproved by E. Lieb in 1970s β–‘ the particular Bobylev-Krook-Wu explicit solutions, this β€œtheorem” holds true for n ≀ 101 and breaks downs afterwards

β€œSuper H-theorem” for Boltzmann Equation

  • H. P. McKean, NYU.

National Academy of Sciences

slide-10
SLIDE 10

β–‘ Heat equation: Is 𝐼(𝑔(𝑒)) CM in 𝑒, if 𝑔(𝑒) satisfies heat equation

β–‘ Equivalently, is 𝐼(π‘Œ + π‘’π‘Ž) CM in t? β–‘ The signs of the first two order derivatives were obtained β–‘ Failed to obtain the 3rd and 4th. (It is easy to compute the derivatives, it is hard to obtain their signs)

β€œSuper H-theorem” for Heat Equation

β€œThis suggests that……, etc., but I could not prove it”

  • C. Villani, 2010 Fields Medalist
slide-11
SLIDE 11

Claude E. Shannon and EPI

β–‘ Entropy power inequality (Shannon 1948): For any two independent continuous random variables X and Y, Equalities holds iff X and Y are Gaussian β–‘ Motivation: Gaussian noise is the worst noise. β–‘ Impact: A new characterization of Gaussian distribution in information theory β–‘ Comments: most profound! (Kolmogorov)

𝑓2β„Ž(π‘Œ+𝑍) β‰₯ 𝑓2β„Ž(π‘Œ) + 𝑓2β„Ž(𝑍)

Central limit theorem Capacity region of Gaussian broadcast channel Capacity region of Gaussian Multiple-Input Multiple-Output broadcast channel Uncertainty principle

All of them can be proved by Entropy power inequality (EPI)

slide-12
SLIDE 12

β–‘ Shannon himself didn’t give a proof but an explanation, which turned

  • ut to be wrong

β–‘ The first proof is given by A. J. Stam (1959), N. M. Blachman (1966) β–‘ Research on EPI

Generalization, new proof, new connection. E.g., Gaussian interference channel is

  • pen, some stronger β€œEPI’’ should exist.

β–‘ Stanford Information Theory School: Thomas Cover and his students: A. El Gamel, M. H. Costa, A. Dembo, A. Barron (1980- 1990) β–‘ Princeton Information Theory School: Sergio Verdu, etc. (2000s)

Entropy Power Inequality

Battle field of Shannon theory

slide-13
SLIDE 13

Ramification of EPI

Shannon EPI Gaussian perturbation: β„Ž(π‘Œ + π‘’π‘Ž) Fisher Information: I π‘Œ + π‘’π‘Ž =

πœ– πœ–π‘’ β„Ž(π‘Œ +

π‘’π‘Ž)/2 Fisher Information is decreasing in 𝑒 𝑓2β„Ž(π‘Œ+ π‘’π‘Ž) is concave in 𝑒 Fisher information inequality (FII):

1 𝐽(π‘Œ+𝑍) β‰₯ 1 𝐽(π‘Œ) + 1 𝐽(𝑍)

Tight Young’s inequality π‘Œ + 𝑍 𝑠 β‰₯ 𝑑 π‘Œ π‘ž 𝑍 π‘Ÿ

Status Quo: FII can imply EPI and all its

  • generalizations. However, life is always
  • hard. FII is far from enough
slide-14
SLIDE 14

On π‘Œ + π‘’π‘Ž

When t > 0, π‘Œ + π‘’π‘Ž and β„Ž(π‘Œ + π‘’π‘Ž) are infinitely differential π‘Œ is arbitrary and β„Ž(π‘Œ) may not exist When t β†’ 0, π‘Œ + π‘’π‘Ž β†’ π‘Œ. When 𝑒 β†’ ∞, π‘Œ + π‘’π‘Ž β†’Gaussian π‘Œ + π‘’π‘Ž is called mixed gaussian distribution (Gaussian Mixed Model (GMM) in machine learning) π‘Œ + π‘’π‘Ž is Gaussian channel/source in information theory Gaussian noise is the worst additive noise Gaussian distribution maximizes β„Ž(π‘Œ) Entropy power inequality, central limit theorem, etc.

slide-15
SLIDE 15

Where we take off

 Shannon Entropy power inequality  Fisher information inequality  β„Ž(π‘Œ + π‘’π‘Ž)  β„Ž 𝑔 𝑒 is CM  When 𝑔(𝑒) satisfied Boltzmann equation, disproved  When 𝑔(𝑒) satisfied heat equation, unknown  We even don’t know what CM is!

Motivation: to study some inequalities; e.g., the convexity of π’Š(𝒀 + π’‡βˆ’π’–π’‚), the concavity of

𝑱 𝒀+ 𝒖𝒂 𝒖

β€œAny progress?” β€œNone…” It is widely believed that there should be no new EPI except Shannon EPI and FII.

Information theorists got lost in the past 70 years Mathematicians ignored it

slide-16
SLIDE 16

Discovery

𝐽 π‘Œ + π‘’π‘Ž =

πœ– 2πœ–π‘’ β„Ž π‘Œ +

π‘’π‘Ž β‰₯ 0 (de Bruijn, 1958) 𝐽(1) =

πœ– πœ–π‘’ 𝐽 π‘Œ +

π‘’π‘Ž ≀ 0 (McKean1966, Costa 1985)

Observation: 𝑱(𝒀 + 𝒖𝒂) is convex in 𝒖  β„Ž π‘Œ + π‘’π‘Ž =

1 2 ln 2πœŒπ‘“π‘’, 𝐽 π‘Œ +

π‘’π‘Ž =

1 𝑒 . 𝐽 is CM: +, -, +, -…

 If the observation is true, the first three derivatives are: +, -, +  Q: Is the 4th order derivative -? Because π‘Ž is Gaussian!  The signs of derivatives of β„Ž(π‘Œ + π‘’π‘Ž) are independent of π‘Œ. Invariant!  Exactly the same problem in McKean 1966 To convince people, we must prove its convexity

slide-17
SLIDE 17

Challenge

Let π‘Œ ∼ 𝑕(𝑦)

  • β„Ž 𝑍

𝑒 = βˆ’βˆ« 𝑔(𝑧, 𝑒) ln 𝑔(𝑧, 𝑒) 𝑒𝑧, no closed form except some

special 𝑕(𝑦). 𝑔(𝑧, 𝑒) satisfies heat equation.

  • 𝐽 𝑍

𝑒 = ∫ 𝑔

1 2

𝑔 𝑒𝑧

  • 𝐽 1 𝑍

𝑒 = βˆ’βˆ« 𝑔

2

𝑔 βˆ’ 𝑔

1 2

𝑔2 2

𝑒𝑧

  • So what is 𝐽(2)? (Heat equation, integration by parts)
slide-18
SLIDE 18

Challenge (cont’d)

It is trivial to calculate derivatives. It is hard to prove their signs

slide-19
SLIDE 19

Breakthrough

Integration by parts: ∫ 𝑣𝑒𝑀 = 𝑣𝑀 βˆ’ ∫ 𝑀𝑒𝑣

First breakthrough since McKean 1966

slide-20
SLIDE 20
slide-21
SLIDE 21

GCMC

Gaussian complete monotonicity conjecture: 𝑱(𝒀 + 𝒖𝒂) is CM in 𝒖 Pointed out by C. Villani and G. Toscani the connection with McKean’s paper A general form: number partition. Hard to determine the coefficients. Conjecture: 𝐦𝐩𝐑𝑱(𝒀 + 𝒖𝒂) is convex in 𝒖

Hard to find 𝛾𝑙,π‘˜ !

slide-22
SLIDE 22

Complete monotone function

How to construct 𝑕(𝑦)?

Herbert R. Stahl, 2013

A new expression for entropy involved special functions in mathematical physics

slide-23
SLIDE 23

Complete monotone function

 A function f(t) is CM, then logf(t) is convex in t  𝐽 𝑍

𝑒 is CM in t, then log 𝐽(𝑍 𝑒) is convex in t

 A function f(t) is CM, a Schur-convex function can be obtained by f(t)  Schur-convex β†’ Majority theory

Remarks: The current tools in information theory don’t work. More sophisticated tools should be built to attack this problem. A new mathematical theory of information theory

slide-24
SLIDE 24

Potential application: Interference channel

ο‚‘

A challenge question: what is the application of GCMC?

ο‚‘

Mathematical speaking, a beautiful result on a fundamental problem will be very useful

ο‚‘

Potential Application

Central limit theorem Capacity region of Gaussian broadcast channel Capacity region of Gaussian Multiple-Input Multiple-Output broadcast channel Uncertainty principle

Where EPI works

Gaussian interference channel: open since 1970s

CM is considered to be much more powerful than EPI Where EPI fails

slide-25
SLIDE 25

Remarks

ο‚‘

If GCMC is true

ο‚‘

A fundamental breakthrough in mathematical physics, information theory and any disciplines related to Gaussian distribution

ο‚‘

A new expression for Fisher information

ο‚‘

Derivatives are an invariant

ο‚‘

Though β„Ž(π‘Œ + π‘’π‘Ž) looks very messy, certain regularity exists

ο‚‘

Application: Gaussian interference channel?

ο‚‘

If GCMC is false

ο‚‘

No Failure, as heat equation is a physical phenomenon

ο‚‘

A lucky number (e.g. 2019) where Gaussian distribution fails. Painful!