Exponential Family Lecture 07 Biostatistics 602 - Statistical - - PowerPoint PPT Presentation

▶
exponential family lecture 07 biostatistics 602
SMART_READER_LITE
LIVE PREVIEW

Exponential Family Lecture 07 Biostatistics 602 - Statistical - - PowerPoint PPT Presentation

Sep 07, 2023 299 likes •1.28k views

. . January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang January 31st, 2013 Hyun Min Kang Exponential Family Lecture 07 Biostatistics 602 - Statistical Inference . . Summary . Exponential Family . . . . . . . . . . . . . .

slide-1
SLIDE 1

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

. .

Biostatistics 602 - Statistical Inference Lecture 07 Exponential Family

Hyun Min Kang January 31st, 2013

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 1 / 20

slide-2
SLIDE 2

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Last Lecture

. . 1 What are differences between complete statistic and minimal

sufficient statistics?

. . 2 What is the relationship between complete statistic and ancillary

statistics?

. 3 What is the characteristic shared among non-constant functions of

complete statistics?

. . 4 What is the Basu’s Theorem? . . 5 Any example where Basu’s Theorem is helpful?

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 2 / 20

slide-3
SLIDE 3

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Last Lecture

. . 1 What are differences between complete statistic and minimal

sufficient statistics?

. . 2 What is the relationship between complete statistic and ancillary

statistics?

. . 3 What is the characteristic shared among non-constant functions of

complete statistics?

. . 4 What is the Basu’s Theorem? . . 5 Any example where Basu’s Theorem is helpful?

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 2 / 20

slide-4
SLIDE 4

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Last Lecture

. . 1 What are differences between complete statistic and minimal

sufficient statistics?

. . 2 What is the relationship between complete statistic and ancillary

statistics?

. . 3 What is the characteristic shared among non-constant functions of

complete statistics?

. . 4 What is the Basu’s Theorem? . . 5 Any example where Basu’s Theorem is helpful?

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 2 / 20

slide-5
SLIDE 5

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Last Lecture

. . 1 What are differences between complete statistic and minimal

sufficient statistics?

. . 2 What is the relationship between complete statistic and ancillary

statistics?

. . 3 What is the characteristic shared among non-constant functions of

complete statistics?

. . 4 What is the Basu’s Theorem? . . 5 Any example where Basu’s Theorem is helpful?

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 2 / 20

slide-6
SLIDE 6

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Last Lecture

. . 1 What are differences between complete statistic and minimal

sufficient statistics?

. . 2 What is the relationship between complete statistic and ancillary

statistics?

. . 3 What is the characteristic shared among non-constant functions of

complete statistics?

. . 4 What is the Basu’s Theorem? . . 5 Any example where Basu’s Theorem is helpful?

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 2 / 20

slide-7
SLIDE 7

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family

.

Definition 3.4.1

. . The random variable X belongs to an exponential family of distributions, if its pdf/pmf can be written in the form f x h x c exp

k j

wj tj x where

  • d

d k.

  • wj

j k are functions of alone.

  • and tj x and h x only involve data.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 3 / 20

slide-8
SLIDE 8

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family

.

Definition 3.4.1

. . The random variable X belongs to an exponential family of distributions, if its pdf/pmf can be written in the form f(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   where

  • d

d k.

  • wj

j k are functions of alone.

  • and tj x and h x only involve data.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 3 / 20

slide-9
SLIDE 9

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family

.

Definition 3.4.1

. . The random variable X belongs to an exponential family of distributions, if its pdf/pmf can be written in the form f(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   where

  • θ = (θ1, · · · , θd), d ≤ k.
  • wj

j k are functions of alone.

  • and tj x and h x only involve data.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 3 / 20

slide-10
SLIDE 10

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family

.

Definition 3.4.1

. . The random variable X belongs to an exponential family of distributions, if its pdf/pmf can be written in the form f(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   where

  • θ = (θ1, · · · , θd), d ≤ k.
  • wj(θ), j ∈ {1, · · · , k} are functions of θ alone.
  • and tj x and h x only involve data.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 3 / 20

slide-11
SLIDE 11

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family

.

Definition 3.4.1

. . The random variable X belongs to an exponential family of distributions, if its pdf/pmf can be written in the form f(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   where

  • θ = (θ1, · · · , θd), d ≤ k.
  • wj(θ), j ∈ {1, · · · , k} are functions of θ alone.
  • and tj(x) and h(x) only involve data.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 3 / 20

slide-12
SLIDE 12

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family

.

Definition 3.4.1

. . The random variable X belongs to an exponential family of distributions, if its pdf/pmf can be written in the form f(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   where

  • θ = (θ1, · · · , θd), d ≤ k.
  • wj(θ), j ∈ {1, · · · , k} are functions of θ alone.
  • and tj(x) and h(x) only involve data.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 3 / 20

slide-13
SLIDE 13

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Example of Exponential Family

.

Problem

. . Show that a Poisson(λ) (λ > 0) belongs to the exponential family .

Proof

. . . . . . . . fX x e

x

x x e exp log

x

x e exp x log Define h x x , c e , w log , and t x x, then fX x h x c exp w t x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 4 / 20

slide-14
SLIDE 14

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Example of Exponential Family

.

Problem

. . Show that a Poisson(λ) (λ > 0) belongs to the exponential family .

Proof

. . fX(x|λ) = e−λλx x! x e exp log

x

x e exp x log Define h x x , c e , w log , and t x x, then fX x h x c exp w t x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 4 / 20

slide-15
SLIDE 15

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Example of Exponential Family

.

Problem

. . Show that a Poisson(λ) (λ > 0) belongs to the exponential family .

Proof

. . fX(x|λ) = e−λλx x! = 1 x!e−λ exp (log λx) x e exp x log Define h x x , c e , w log , and t x x, then fX x h x c exp w t x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 4 / 20

slide-16
SLIDE 16

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Example of Exponential Family

.

Problem

. . Show that a Poisson(λ) (λ > 0) belongs to the exponential family .

Proof

. . fX(x|λ) = e−λλx x! = 1 x!e−λ exp (log λx) = 1 x!e−λ exp (x log λ) Define h x x , c e , w log , and t x x, then fX x h x c exp w t x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 4 / 20

slide-17
SLIDE 17

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Example of Exponential Family

.

Problem

. . Show that a Poisson(λ) (λ > 0) belongs to the exponential family .

Proof

. . fX(x|λ) = e−λλx x! = 1 x!e−λ exp (log λx) = 1 x!e−λ exp (x log λ) Define h(x) = 1/x!, c(λ) = e−λ, w(λ) = log λ, and t(x) = x, then fX x h x c exp w t x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 4 / 20

slide-18
SLIDE 18

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Example of Exponential Family

.

Problem

. . Show that a Poisson(λ) (λ > 0) belongs to the exponential family .

Proof

. . fX(x|λ) = e−λλx x! = 1 x!e−λ exp (log λx) = 1 x!e−λ exp (x log λ) Define h(x) = 1/x!, c(λ) = e−λ, w(λ) = log λ, and t(x) = x, then fX(x|λ) = h(x)c(λ) exp [w(λ)t(x)]

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 4 / 20

slide-19
SLIDE 19

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Normal Distribution Belongs to an Exponential Family

fX(x|θ = (µ, σ2)) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] exp x x Define h x , c exp , k , w , t x x, w , t x x , then fX x h x c exp

k j

wj tj x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 5 / 20

slide-20
SLIDE 20

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Normal Distribution Belongs to an Exponential Family

fX(x|θ = (µ, σ2)) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp [ − x2 2σ2 + 2µx 2σ2 − µ2 2σ2 ] Define h x , c exp , k , w , t x x, w , t x x , then fX x h x c exp

k j

wj tj x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 5 / 20

slide-21
SLIDE 21

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Normal Distribution Belongs to an Exponential Family

fX(x|θ = (µ, σ2)) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp [ − x2 2σ2 + 2µx 2σ2 − µ2 2σ2 ] Define h(x) = 1, c(θ) =

1 √ 2πσ2 exp

[ − µ2

2σ2

] , k , w , t x x, w , t x x , then fX x h x c exp

k j

wj tj x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 5 / 20

slide-22
SLIDE 22

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Normal Distribution Belongs to an Exponential Family

fX(x|θ = (µ, σ2)) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp [ − x2 2σ2 + 2µx 2σ2 − µ2 2σ2 ] Define h(x) = 1, c(θ) =

1 √ 2πσ2 exp

[ − µ2

2σ2

] , k = 2, w1(θ) = µ

σ2 , t1(x) = x, w2(θ) = − 1 2σ2 , t2(x) = x2, then

fX x h x c exp

k j

wj tj x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 5 / 20

slide-23
SLIDE 23

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Normal Distribution Belongs to an Exponential Family

fX(x|θ = (µ, σ2)) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp [ − x2 2σ2 + 2µx 2σ2 − µ2 2σ2 ] Define h(x) = 1, c(θ) =

1 √ 2πσ2 exp

[ − µ2

2σ2

] , k = 2, w1(θ) = µ

σ2 , t1(x) = x, w2(θ) = − 1 2σ2 , t2(x) = x2, then

fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)  

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 5 / 20

slide-24
SLIDE 24

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

A Specialized Normal Distribution : N(µ, µ2)

fX(x|µ) = 1 √ 2πµ2 exp [ −(x − µ)2 2µ2 ] exp x x exp exp x x Define h x , c e , k , w , t x x, w , t x x , then fX x h x c exp

k j

wj tj x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 6 / 20

slide-25
SLIDE 25

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

A Specialized Normal Distribution : N(µ, µ2)

fX(x|µ) = 1 √ 2πµ2 exp [ −(x − µ)2 2µ2 ] = 1 √ 2πµ2 exp [ − x2 2µ2 + 2µx 2µ2 − µ2 2µ2 ] exp exp x x Define h x , c e , k , w , t x x, w , t x x , then fX x h x c exp

k j

wj tj x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 6 / 20

slide-26
SLIDE 26

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

A Specialized Normal Distribution : N(µ, µ2)

fX(x|µ) = 1 √ 2πµ2 exp [ −(x − µ)2 2µ2 ] = 1 √ 2πµ2 exp [ − x2 2µ2 + 2µx 2µ2 − µ2 2µ2 ] = 1 √ 2πµ2 exp ( −1 2 ) exp [ − 1 2µ2 x2 + 1 µx ] Define h x , c e , k , w , t x x, w , t x x , then fX x h x c exp

k j

wj tj x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 6 / 20

slide-27
SLIDE 27

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

A Specialized Normal Distribution : N(µ, µ2)

fX(x|µ) = 1 √ 2πµ2 exp [ −(x − µ)2 2µ2 ] = 1 √ 2πµ2 exp [ − x2 2µ2 + 2µx 2µ2 − µ2 2µ2 ] = 1 √ 2πµ2 exp ( −1 2 ) exp [ − 1 2µ2 x2 + 1 µx ] Define h(x) = 1, c(µ) =

1 √ 2πσ2 e− 1

2 ,

k , w , t x x, w , t x x , then fX x h x c exp

k j

wj tj x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 6 / 20

slide-28
SLIDE 28

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

A Specialized Normal Distribution : N(µ, µ2)

fX(x|µ) = 1 √ 2πµ2 exp [ −(x − µ)2 2µ2 ] = 1 √ 2πµ2 exp [ − x2 2µ2 + 2µx 2µ2 − µ2 2µ2 ] = 1 √ 2πµ2 exp ( −1 2 ) exp [ − 1 2µ2 x2 + 1 µx ] Define h(x) = 1, c(µ) =

1 √ 2πσ2 e− 1

2 ,

k = 2, w1(µ) = 1

µ, t1(x) = x, w2(µ) = − 1 2µ2 , t2(x) = x2, then

fX x h x c exp

k j

wj tj x

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 6 / 20

slide-29
SLIDE 29

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

A Specialized Normal Distribution : N(µ, µ2)

fX(x|µ) = 1 √ 2πµ2 exp [ −(x − µ)2 2µ2 ] = 1 √ 2πµ2 exp [ − x2 2µ2 + 2µx 2µ2 − µ2 2µ2 ] = 1 √ 2πµ2 exp ( −1 2 ) exp [ − 1 2µ2 x2 + 1 µx ] Define h(x) = 1, c(µ) =

1 √ 2πσ2 e− 1

2 ,

k = 2, w1(µ) = 1

µ, t1(x) = x, w2(µ) = − 1 2µ2 , t2(x) = x2, then

fX(x|µ) = h(x)c(µ) exp  

k

∑

j=1

wj(µ)tj(x)  

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 6 / 20

slide-30
SLIDE 30

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Curved and Full Exponential Families

.

Definition

. . For an exponential family, if d = dim(θ) < k, then this exponential family is called curved exponential family. if d = dim(θ) = k, then this exponential family is called full exponential family. .

Examples

. . . . . . . .

  • Poisson( ),

is a full exponential family

  • is also a full exponential family
  • is also a curved exponential family

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 7 / 20

slide-31
SLIDE 31

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Curved and Full Exponential Families

.

Definition

. . For an exponential family, if d = dim(θ) < k, then this exponential family is called curved exponential family. if d = dim(θ) = k, then this exponential family is called full exponential family. .

Examples

. .

  • Poisson(λ), λ > 0 is a full exponential family
  • is also a full exponential family
  • is also a curved exponential family

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 7 / 20

slide-32
SLIDE 32

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Curved and Full Exponential Families

.

Definition

. . For an exponential family, if d = dim(θ) < k, then this exponential family is called curved exponential family. if d = dim(θ) = k, then this exponential family is called full exponential family. .

Examples

. .

  • Poisson(λ), λ > 0 is a full exponential family
  • N(µ, σ2), µ ∈ R, σ > 0 is also a full exponential family
  • is also a curved exponential family

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 7 / 20

slide-33
SLIDE 33

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Curved and Full Exponential Families

.

Definition

. . For an exponential family, if d = dim(θ) < k, then this exponential family is called curved exponential family. if d = dim(θ) = k, then this exponential family is called full exponential family. .

Examples

. .

  • Poisson(λ), λ > 0 is a full exponential family
  • N(µ, σ2), µ ∈ R, σ > 0 is also a full exponential family
  • N(µ, µ), µ ∈ R is also a curved exponential family

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 7 / 20

slide-34
SLIDE 34

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Alternative Parametrization of Exponential Families

An alternative parametrization of the exponential family of distributions in terms of ”natural” or ”canonical” parameters can be written as follows. fX x h x c exp

k j

tj x The alternative parametrization can be achieved by defining

i

wj from the following equation, fX x h x c exp

k j

wj tj x where c c w . This alternative parametrization is most often used in a GLM (Generalized Linear Model) course.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 8 / 20

slide-35
SLIDE 35

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Alternative Parametrization of Exponential Families

An alternative parametrization of the exponential family of distributions in terms of ”natural” or ”canonical” parameters can be written as follows. fX(x|η) = h(x)c∗(η) exp  

k

∑

j=1

ηtj(x)   The alternative parametrization can be achieved by defining

i

wj from the following equation, fX x h x c exp

k j

wj tj x where c c w . This alternative parametrization is most often used in a GLM (Generalized Linear Model) course.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 8 / 20

slide-36
SLIDE 36

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Alternative Parametrization of Exponential Families

An alternative parametrization of the exponential family of distributions in terms of ”natural” or ”canonical” parameters can be written as follows. fX(x|η) = h(x)c∗(η) exp  

k

∑

j=1

ηtj(x)   The alternative parametrization can be achieved by defining ηi = wj(θ) from the following equation, fX x h x c exp

k j

wj tj x where c c w . This alternative parametrization is most often used in a GLM (Generalized Linear Model) course.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 8 / 20

slide-37
SLIDE 37

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Alternative Parametrization of Exponential Families

An alternative parametrization of the exponential family of distributions in terms of ”natural” or ”canonical” parameters can be written as follows. fX(x|η) = h(x)c∗(η) exp  

k

∑

j=1

ηtj(x)   The alternative parametrization can be achieved by defining ηi = wj(θ) from the following equation, fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   where c∗(η) = c ◦ w(θ). This alternative parametrization is most often used in a GLM (Generalized Linear Model) course.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 8 / 20

slide-38
SLIDE 38

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Exponential Families

.

Theorem 6.2.10

. .

  • Let X1, · · · , Xn

i.i.d.

∼ fX(x|θ) that belongs to an exponential family

given by fX x h x c exp

k j

wj tj x where

d

d k.

  • Then the following T X is a sufficient statistic for

. T X

n j

t Xj

n j

tk Xj

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 9 / 20

slide-39
SLIDE 39

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Exponential Families

.

Theorem 6.2.10

. .

  • Let X1, · · · , Xn

i.i.d.

∼ fX(x|θ) that belongs to an exponential family

given by fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   where

d

d k.

  • Then the following T X is a sufficient statistic for

. T X

n j

t Xj

n j

tk Xj

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 9 / 20

slide-40
SLIDE 40

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Exponential Families

.

Theorem 6.2.10

. .

  • Let X1, · · · , Xn

i.i.d.

∼ fX(x|θ) that belongs to an exponential family

given by fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   where θ = (θ1, · · · , θd), d ≤ k.

  • Then the following T(X) is a sufficient statistic for θ.

T X

n j

t Xj

n j

tk Xj

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 9 / 20

slide-41
SLIDE 41

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Exponential Families

.

Theorem 6.2.10

. .

  • Let X1, · · · , Xn

i.i.d.

∼ fX(x|θ) that belongs to an exponential family

given by fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   where θ = (θ1, · · · , θd), d ≤ k.

  • Then the following T(X) is a sufficient statistic for θ.

T(X) =  

n

∑

j=1

t1(Xj), · · · ,

n

∑

j=1

tk(Xj)  

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 9 / 20

slide-42
SLIDE 42

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Normal Distribution

.

Problem

. . Let X1, · · · , Xn

i.i.d.

∼ N(µ, σ2), where µ ∈ R, and σ2 is known. Find a

sufficient statistic for µ.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 10 / 20

slide-43
SLIDE 43

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Normal Distribution - Solution

fX(x|µ) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] exp x exp exp x h x c exp w t x where h x exp

x

c exp w t x x Therefore, T X

n i

t Xi

n i

Xi is a sufficient statistic for by Theorem 6.2.10.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 11 / 20

slide-44
SLIDE 44

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Normal Distribution - Solution

fX(x|µ) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp ( − x2 2σ2 ) exp ( − µ2 2σ2 ) exp ( µ σ2 x ) h x c exp w t x where h x exp

x

c exp w t x x Therefore, T X

n i

t Xi

n i

Xi is a sufficient statistic for by Theorem 6.2.10.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 11 / 20

slide-45
SLIDE 45

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Normal Distribution - Solution

fX(x|µ) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp ( − x2 2σ2 ) exp ( − µ2 2σ2 ) exp ( µ σ2 x ) = h(x)c(µ) exp [w(µ)t(x)] where h x exp

x

c exp w t x x Therefore, T X

n i

t Xi

n i

Xi is a sufficient statistic for by Theorem 6.2.10.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 11 / 20

slide-46
SLIDE 46

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Normal Distribution - Solution

fX(x|µ) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp ( − x2 2σ2 ) exp ( − µ2 2σ2 ) exp ( µ σ2 x ) = h(x)c(µ) exp [w(µ)t(x)] where            h(x) =

1 √ 2πσ2 exp

[ − x2

2σ2

] c exp w t x x Therefore, T X

n i

t Xi

n i

Xi is a sufficient statistic for by Theorem 6.2.10.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 11 / 20

slide-47
SLIDE 47

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Normal Distribution - Solution

fX(x|µ) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp ( − x2 2σ2 ) exp ( − µ2 2σ2 ) exp ( µ σ2 x ) = h(x)c(µ) exp [w(µ)t(x)] where            h(x) =

1 √ 2πσ2 exp

[ − x2

2σ2

] c(µ) = exp ( − µ2

2σ2

) w t x x Therefore, T X

n i

t Xi

n i

Xi is a sufficient statistic for by Theorem 6.2.10.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 11 / 20

slide-48
SLIDE 48

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Normal Distribution - Solution

fX(x|µ) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp ( − x2 2σ2 ) exp ( − µ2 2σ2 ) exp ( µ σ2 x ) = h(x)c(µ) exp [w(µ)t(x)] where            h(x) =

1 √ 2πσ2 exp

[ − x2

2σ2

] c(µ) = exp ( − µ2

2σ2

) w(µ) = µ/σ2 t x x Therefore, T X

n i

t Xi

n i

Xi is a sufficient statistic for by Theorem 6.2.10.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 11 / 20

slide-49
SLIDE 49

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Normal Distribution - Solution

fX(x|µ) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp ( − x2 2σ2 ) exp ( − µ2 2σ2 ) exp ( µ σ2 x ) = h(x)c(µ) exp [w(µ)t(x)] where            h(x) =

1 √ 2πσ2 exp

[ − x2

2σ2

] c(µ) = exp ( − µ2

2σ2

) w(µ) = µ/σ2 t(x) = x Therefore, T X

n i

t Xi

n i

Xi is a sufficient statistic for by Theorem 6.2.10.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 11 / 20

slide-50
SLIDE 50

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Sufficient Statistic for Normal Distribution - Solution

fX(x|µ) = 1 √ 2πσ2 exp [ −(x − µ)2 2σ2 ] = 1 √ 2πσ2 exp ( − x2 2σ2 ) exp ( − µ2 2σ2 ) exp ( µ σ2 x ) = h(x)c(µ) exp [w(µ)t(x)] where            h(x) =

1 √ 2πσ2 exp

[ − x2

2σ2

] c(µ) = exp ( − µ2

2σ2

) w(µ) = µ/σ2 t(x) = x Therefore, T(X) = ∑n

i=1 t(Xi) = ∑n i=1 Xi is a sufficient statistic for µ by

Theorem 6.2.10.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 11 / 20

slide-51
SLIDE 51

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Theorem 5.2.11

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX x h x c exp

k j

wj tj x is a member of an exponential family. Define a statistic T X by T X

n j

t Xj

n j

tk Xj If the set w wk contains an open subset of

k,

then the distribution of T X is an exponential family of the form fT u uk H u uk c

n exp k j

wj ui

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 12 / 20

slide-52
SLIDE 52

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Theorem 5.2.11

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   is a member of an exponential family. Define a statistic T X by T X

n j

t Xj

n j

tk Xj If the set w wk contains an open subset of

k,

then the distribution of T X is an exponential family of the form fT u uk H u uk c

n exp k j

wj ui

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 12 / 20

slide-53
SLIDE 53

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Theorem 5.2.11

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   is a member of an exponential family. Define a statistic T(X) by T X

n j

t Xj

n j

tk Xj If the set w wk contains an open subset of

k,

then the distribution of T X is an exponential family of the form fT u uk H u uk c

n exp k j

wj ui

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 12 / 20

slide-54
SLIDE 54

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Theorem 5.2.11

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   is a member of an exponential family. Define a statistic T(X) by T(X) =  

n

∑

j=1

t1(Xj), · · · ,

n

∑

j=1

tk(Xj)   If the set w wk contains an open subset of

k,

then the distribution of T X is an exponential family of the form fT u uk H u uk c

n exp k j

wj ui

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 12 / 20

slide-55
SLIDE 55

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Theorem 5.2.11

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   is a member of an exponential family. Define a statistic T(X) by T(X) =  

n

∑

j=1

t1(Xj), · · · ,

n

∑

j=1

tk(Xj)   If the set {w1(θ), · · · , wk(θ), ∀θ ∈ Θ} contains an open subset of Rk, then the distribution of T(X) is an exponential family of the form fT u uk H u uk c

n exp k j

wj ui

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 12 / 20

slide-56
SLIDE 56

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Theorem 5.2.11

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   is a member of an exponential family. Define a statistic T(X) by T(X) =  

n

∑

j=1

t1(Xj), · · · ,

n

∑

j=1

tk(Xj)   If the set {w1(θ), · · · , wk(θ), ∀θ ∈ Θ} contains an open subset of Rk, then the distribution of T(X) is an exponential family of the form fT(u1, · · · , uk|θ) = H(u1, · · · , uk)[c(θ)]n exp  

k

∑

j=1

wj(θ)ui  

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 12 / 20

slide-57
SLIDE 57

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Theorem 6.2.25

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX x h x c exp

k j

wj tj x is a member of an exponential family. Then the statistic T X T X

n j

t Xj

n j

tk Xj is complete as long as the parameter space contains an open set in

k

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 13 / 20

slide-58
SLIDE 58

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Theorem 6.2.25

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   is a member of an exponential family. Then the statistic T X T X

n j

t Xj

n j

tk Xj is complete as long as the parameter space contains an open set in

k

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 13 / 20

slide-59
SLIDE 59

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Theorem 6.2.25

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   is a member of an exponential family. Then the statistic T(X) T(X) =  

n

∑

j=1

t1(Xj), · · · ,

n

∑

j=1

tk(Xj)   is complete as long as the parameter space contains an open set in

k

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 13 / 20

slide-60
SLIDE 60

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Theorem 6.2.25

Suppose X1, · · · , Xn is a random sample from pdf or pmf fX(x|θ) where fX(x|θ) = h(x)c(θ) exp  

k

∑

j=1

wj(θ)tj(x)   is a member of an exponential family. Then the statistic T(X) T(X) =  

n

∑

j=1

t1(Xj), · · · ,

n

∑

j=1

tk(Xj)   is complete as long as the parameter space Θ contains an open set in Rk

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 13 / 20

slide-61
SLIDE 61

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

What is the ”open set”?

.

Definition : Open Set

. . A set A is open in Rk of for every x ∈ A, there exists a ϵ-ball B(x, ϵ) around x such that B(x, ϵ) ⊂ A. B x y y x y

k

.

Examples (from Wolfram MathWorld)

. . . . . . . .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 14 / 20

slide-62
SLIDE 62

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

What is the ”open set”?

.

Definition : Open Set

. . A set A is open in Rk of for every x ∈ A, there exists a ϵ-ball B(x, ϵ) around x such that B(x, ϵ) ⊂ A. B(x, ϵ) = {y : |y − x| < ϵ, y ∈ Rk} .

Examples (from Wolfram MathWorld)

. . . . . . . .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 14 / 20

slide-63
SLIDE 63

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

What is the ”open set”?

.

Definition : Open Set

. . A set A is open in Rk of for every x ∈ A, there exists a ϵ-ball B(x, ϵ) around x such that B(x, ϵ) ⊂ A. B(x, ϵ) = {y : |y − x| < ϵ, y ∈ Rk} .

Examples (from Wolfram MathWorld)

. .

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 14 / 20

slide-64
SLIDE 64

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Examples of open set test

  • A = (−1, 1) : A is open in R
  • A

: A is not open in

  • A

: A is open in

  • A

: A is not open in

  • A

x y x y : A is not open in

  • A

x y x y x : A is not open in

  • A

x y x y : A is open in

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 15 / 20

slide-65
SLIDE 65

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Examples of open set test

  • A = (−1, 1) : A is open in R
  • A = [−1, 1] : A is not open in R
  • A

: A is open in

  • A

: A is not open in

  • A

x y x y : A is not open in

  • A

x y x y x : A is not open in

  • A

x y x y : A is open in

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 15 / 20

slide-66
SLIDE 66

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Examples of open set test

  • A = (−1, 1) : A is open in R
  • A = [−1, 1] : A is not open in R
  • A = (−∞, 0) × R : A is open in R2
  • A

: A is not open in

  • A

x y x y : A is not open in

  • A

x y x y x : A is not open in

  • A

x y x y : A is open in

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 15 / 20

slide-67
SLIDE 67

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Examples of open set test

  • A = (−1, 1) : A is open in R
  • A = [−1, 1] : A is not open in R
  • A = (−∞, 0) × R : A is open in R2
  • A = (−∞, 0] × R : A is not open in R2
  • A

x y x y : A is not open in

  • A

x y x y x : A is not open in

  • A

x y x y : A is open in

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 15 / 20

slide-68
SLIDE 68

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Examples of open set test

  • A = (−1, 1) : A is open in R
  • A = [−1, 1] : A is not open in R
  • A = (−∞, 0) × R : A is open in R2
  • A = (−∞, 0] × R : A is not open in R2
  • A = {(x, y) : x ∈ (−1, 1), y = 0} : A is not open in R2
  • A

x y x y x : A is not open in

  • A

x y x y : A is open in

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 15 / 20

slide-69
SLIDE 69

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Examples of open set test

  • A = (−1, 1) : A is open in R
  • A = [−1, 1] : A is not open in R
  • A = (−∞, 0) × R : A is open in R2
  • A = (−∞, 0] × R : A is not open in R2
  • A = {(x, y) : x ∈ (−1, 1), y = 0} : A is not open in R2
  • A = {(x, y) : x ∈ R, y ∈ x2} : A is not open in R2
  • A

x y x y : A is open in

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 15 / 20

slide-70
SLIDE 70

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Examples of open set test

  • A = (−1, 1) : A is open in R
  • A = [−1, 1] : A is not open in R
  • A = (−∞, 0) × R : A is open in R2
  • A = (−∞, 0] × R : A is not open in R2
  • A = {(x, y) : x ∈ (−1, 1), y = 0} : A is not open in R2
  • A = {(x, y) : x ∈ R, y ∈ x2} : A is not open in R2
  • A = {(x, y) : x2 + y2 < 1} : A is open in R2

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 15 / 20

slide-71
SLIDE 71

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ fX(x|θ) = θxθ−1 where 0 < x < 1, θ > 0. Is ∏n

i=1 Xi (1) a

sufficient statistic? (2) a complete statistic? (3) a minimal sufficient statistic? .

How to solve it

. . . . . . . .

  • Show that fX x

is a member of an exponential family.

  • Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to

n i

Xi.

  • Apply Theorem 6.2.25 to show that it is complete.
  • If they are both sufficient and complete, Theorem 6.2.28 will imply

that it is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 16 / 20

slide-72
SLIDE 72

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ fX(x|θ) = θxθ−1 where 0 < x < 1, θ > 0. Is ∏n

i=1 Xi (1) a

sufficient statistic? (2) a complete statistic? (3) a minimal sufficient statistic? .

How to solve it

. .

  • Show that fX(x|θ) is a member of an exponential family.
  • Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to

n i

Xi.

  • Apply Theorem 6.2.25 to show that it is complete.
  • If they are both sufficient and complete, Theorem 6.2.28 will imply

that it is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 16 / 20

slide-73
SLIDE 73

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ fX(x|θ) = θxθ−1 where 0 < x < 1, θ > 0. Is ∏n

i=1 Xi (1) a

sufficient statistic? (2) a complete statistic? (3) a minimal sufficient statistic? .

How to solve it

. .

  • Show that fX(x|θ) is a member of an exponential family.
  • Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to ∏n

i=1 Xi.

  • Apply Theorem 6.2.25 to show that it is complete.
  • If they are both sufficient and complete, Theorem 6.2.28 will imply

that it is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 16 / 20

slide-74
SLIDE 74

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ fX(x|θ) = θxθ−1 where 0 < x < 1, θ > 0. Is ∏n

i=1 Xi (1) a

sufficient statistic? (2) a complete statistic? (3) a minimal sufficient statistic? .

How to solve it

. .

  • Show that fX(x|θ) is a member of an exponential family.
  • Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to ∏n

i=1 Xi.

  • Apply Theorem 6.2.25 to show that it is complete.
  • If they are both sufficient and complete, Theorem 6.2.28 will imply

that it is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 16 / 20

slide-75
SLIDE 75

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ fX(x|θ) = θxθ−1 where 0 < x < 1, θ > 0. Is ∏n

i=1 Xi (1) a

sufficient statistic? (2) a complete statistic? (3) a minimal sufficient statistic? .

How to solve it

. .

  • Show that fX(x|θ) is a member of an exponential family.
  • Apply Theorem 6.2.10 to obtain a sufficient statistic and see if it is

equivalent to or related to ∏n

i=1 Xi.

  • Apply Theorem 6.2.25 to show that it is complete.
  • If they are both sufficient and complete, Theorem 6.2.28 will imply

that it is also a minimal sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 16 / 20

slide-76
SLIDE 76

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

fX(x|θ) belong to an exponential family

fX(x|θ) = θxθ−1I(0 < x < 1) I x x x I x x exp log x I x x exp log x h x c exp w t x where h x I x x c w t x log x Therefore, fX x belongs to an exponential family.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 17 / 20

slide-77
SLIDE 77

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

fX(x|θ) belong to an exponential family

fX(x|θ) = θxθ−1I(0 < x < 1) = I(0 < x < 1)x−1θxθ I x x exp log x I x x exp log x h x c exp w t x where h x I x x c w t x log x Therefore, fX x belongs to an exponential family.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 17 / 20

slide-78
SLIDE 78

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

fX(x|θ) belong to an exponential family

fX(x|θ) = θxθ−1I(0 < x < 1) = I(0 < x < 1)x−1θxθ = I(0 < x < 1)x−1θ exp(log xθ) I x x exp log x h x c exp w t x where h x I x x c w t x log x Therefore, fX x belongs to an exponential family.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 17 / 20

slide-79
SLIDE 79

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

fX(x|θ) belong to an exponential family

fX(x|θ) = θxθ−1I(0 < x < 1) = I(0 < x < 1)x−1θxθ = I(0 < x < 1)x−1θ exp(log xθ) = I(0 < x < 1)x−1θ exp(θ log x) h x c exp w t x where h x I x x c w t x log x Therefore, fX x belongs to an exponential family.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 17 / 20

slide-80
SLIDE 80

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

fX(x|θ) belong to an exponential family

fX(x|θ) = θxθ−1I(0 < x < 1) = I(0 < x < 1)x−1θxθ = I(0 < x < 1)x−1θ exp(log xθ) = I(0 < x < 1)x−1θ exp(θ log x) = h(x)c(θ) exp(w(θ)t(x)) where        h(x) = I(0 < x < 1)x−1 c w t x log x Therefore, fX x belongs to an exponential family.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 17 / 20

slide-81
SLIDE 81

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

fX(x|θ) belong to an exponential family

fX(x|θ) = θxθ−1I(0 < x < 1) = I(0 < x < 1)x−1θxθ = I(0 < x < 1)x−1θ exp(log xθ) = I(0 < x < 1)x−1θ exp(θ log x) = h(x)c(θ) exp(w(θ)t(x)) where        h(x) = I(0 < x < 1)x−1 c(θ) = θ w t x log x Therefore, fX x belongs to an exponential family.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 17 / 20

slide-82
SLIDE 82

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

fX(x|θ) belong to an exponential family

fX(x|θ) = θxθ−1I(0 < x < 1) = I(0 < x < 1)x−1θxθ = I(0 < x < 1)x−1θ exp(log xθ) = I(0 < x < 1)x−1θ exp(θ log x) = h(x)c(θ) exp(w(θ)t(x)) where        h(x) = I(0 < x < 1)x−1 c(θ) = θ w(θ) = θ t x log x Therefore, fX x belongs to an exponential family.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 17 / 20

slide-83
SLIDE 83

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

fX(x|θ) belong to an exponential family

fX(x|θ) = θxθ−1I(0 < x < 1) = I(0 < x < 1)x−1θxθ = I(0 < x < 1)x−1θ exp(log xθ) = I(0 < x < 1)x−1θ exp(θ log x) = h(x)c(θ) exp(w(θ)t(x)) where        h(x) = I(0 < x < 1)x−1 c(θ) = θ w(θ) = θ t(x) = log x Therefore, fX(x|θ) belongs to an exponential family.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 17 / 20

slide-84
SLIDE 84

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.10

fX(x|θ) = h(x)c(θ) exp(w(θ)t(x)) h x I x x c w t x log x By Theorem 6.2.10, T X

n i

t Xi

n i

log Xi is a sufficient statistic for .

n i

Xi exp log

n i

Xi exp

n i

log Xi eT X Because

n i

Xi is an one-to-one function of T X , it is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 18 / 20

slide-85
SLIDE 85

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.10

fX(x|θ) = h(x)c(θ) exp(w(θ)t(x))        h(x) = I(0 < x < 1)x−1 c w t x log x By Theorem 6.2.10, T X

n i

t Xi

n i

log Xi is a sufficient statistic for .

n i

Xi exp log

n i

Xi exp

n i

log Xi eT X Because

n i

Xi is an one-to-one function of T X , it is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 18 / 20

slide-86
SLIDE 86

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.10

fX(x|θ) = h(x)c(θ) exp(w(θ)t(x))        h(x) = I(0 < x < 1)x−1 c(θ) = θ w t x log x By Theorem 6.2.10, T X

n i

t Xi

n i

log Xi is a sufficient statistic for .

n i

Xi exp log

n i

Xi exp

n i

log Xi eT X Because

n i

Xi is an one-to-one function of T X , it is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 18 / 20

slide-87
SLIDE 87

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.10

fX(x|θ) = h(x)c(θ) exp(w(θ)t(x))        h(x) = I(0 < x < 1)x−1 c(θ) = θ w(θ) = θ t x log x By Theorem 6.2.10, T X

n i

t Xi

n i

log Xi is a sufficient statistic for .

n i

Xi exp log

n i

Xi exp

n i

log Xi eT X Because

n i

Xi is an one-to-one function of T X , it is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 18 / 20

slide-88
SLIDE 88

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.10

fX(x|θ) = h(x)c(θ) exp(w(θ)t(x))        h(x) = I(0 < x < 1)x−1 c(θ) = θ w(θ) = θ t(x) = log x By Theorem 6.2.10, T X

n i

t Xi

n i

log Xi is a sufficient statistic for .

n i

Xi exp log

n i

Xi exp

n i

log Xi eT X Because

n i

Xi is an one-to-one function of T X , it is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 18 / 20

slide-89
SLIDE 89

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.10

fX(x|θ) = h(x)c(θ) exp(w(θ)t(x))        h(x) = I(0 < x < 1)x−1 c(θ) = θ w(θ) = θ t(x) = log x By Theorem 6.2.10, T(X) = ∑n

i=1 t(Xi) = ∑n i=1 log Xi is a sufficient

statistic for θ.

n i

Xi exp log

n i

Xi exp

n i

log Xi eT X Because

n i

Xi is an one-to-one function of T X , it is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 18 / 20

slide-90
SLIDE 90

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.10

fX(x|θ) = h(x)c(θ) exp(w(θ)t(x))        h(x) = I(0 < x < 1)x−1 c(θ) = θ w(θ) = θ t(x) = log x By Theorem 6.2.10, T(X) = ∑n

i=1 t(Xi) = ∑n i=1 log Xi is a sufficient

statistic for θ.

n

∏

i=1

Xi = exp ( log

n

∏

i=1

Xi ) = exp ( n ∑

i=1

log Xi ) = eT(X) Because

n i

Xi is an one-to-one function of T X , it is also a sufficient statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 18 / 20

slide-91
SLIDE 91

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.10

fX(x|θ) = h(x)c(θ) exp(w(θ)t(x))        h(x) = I(0 < x < 1)x−1 c(θ) = θ w(θ) = θ t(x) = log x By Theorem 6.2.10, T(X) = ∑n

i=1 t(Xi) = ∑n i=1 log Xi is a sufficient

statistic for θ.

n

∏

i=1

Xi = exp ( log

n

∏

i=1

Xi ) = exp ( n ∑

i=1

log Xi ) = eT(X) Because ∏n

i=1 Xi is an one-to-one function of T(X), it is also a sufficient

statistic.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 18 / 20

slide-92
SLIDE 92

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.25 and 6.2.28

.

T(X) is a complete statistic

. . Let A = {w(θ) : θ ∈ Ω} = {θ : θ > 0}. A contains an open subset in R. By Theorem 6.2.25, T X

n i

log Xi is a complete statistic for . .

T X is a minimal sufficient statistic

. . . . . . . . By Theorem 6.2.28, because T X is both sufficient and complete, it is also minimal sufficient. .

n i

Xi eT X is also minimal sufficient and complete

. . . . . . . . Because

n i

Xi eT X is an one-to-one function of T X ,

n i

Xi is sufficient, complete, and minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 19 / 20

slide-93
SLIDE 93

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.25 and 6.2.28

.

T(X) is a complete statistic

. . Let A = {w(θ) : θ ∈ Ω} = {θ : θ > 0}. A contains an open subset in R. By Theorem 6.2.25, T(X) = ∑n

i=1 log Xi is a complete statistic for θ.

.

T X is a minimal sufficient statistic

. . . . . . . . By Theorem 6.2.28, because T X is both sufficient and complete, it is also minimal sufficient. .

n i

Xi eT X is also minimal sufficient and complete

. . . . . . . . Because

n i

Xi eT X is an one-to-one function of T X ,

n i

Xi is sufficient, complete, and minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 19 / 20

slide-94
SLIDE 94

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.25 and 6.2.28

.

T(X) is a complete statistic

. . Let A = {w(θ) : θ ∈ Ω} = {θ : θ > 0}. A contains an open subset in R. By Theorem 6.2.25, T(X) = ∑n

i=1 log Xi is a complete statistic for θ.

.

T(X) is a minimal sufficient statistic

. . By Theorem 6.2.28, because T(X) is both sufficient and complete, it is also minimal sufficient. .

n i

Xi eT X is also minimal sufficient and complete

. . . . . . . . Because

n i

Xi eT X is an one-to-one function of T X ,

n i

Xi is sufficient, complete, and minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 19 / 20

slide-95
SLIDE 95

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Apply Theorem 6.2.25 and 6.2.28

.

T(X) is a complete statistic

. . Let A = {w(θ) : θ ∈ Ω} = {θ : θ > 0}. A contains an open subset in R. By Theorem 6.2.25, T(X) = ∑n

i=1 log Xi is a complete statistic for θ.

.

T(X) is a minimal sufficient statistic

. . By Theorem 6.2.28, because T(X) is both sufficient and complete, it is also minimal sufficient. .

∏n

i=1 Xi = eT(X) is also minimal sufficient and complete

. . Because ∏n

i=1 Xi = eT(X) is an one-to-one function of T(X), ∏n i=1 Xi is

sufficient, complete, and minimal sufficient.

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 19 / 20

slide-96
SLIDE 96

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family

.

Today

. .

  • Curved and full exponential families
  • Sufficient statistics for exponential families
  • Complete statistics for exponential families

.

Next Lecture

. . . . . . . .

  • Review of Chapter 6

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 20 / 20

slide-97
SLIDE 97

. . . . . .

. . . . . . . . . . . . . . . . . . Exponential Family . Summary

Exponential Family

.

Today

. .

  • Curved and full exponential families
  • Sufficient statistics for exponential families
  • Complete statistics for exponential families

.

Next Lecture

. .

  • Review of Chapter 6

Hyun Min Kang Biostatistics 602 - Lecture 07 January 31st, 2013 20 / 20