IWSM 2013, the University of Georgia in Athens, Georgia, USA 18-21 - - PowerPoint PPT Presentation

IWSM 2013, the University of Georgia in Athens, Georgia, USA

18-21 July, 2013

Presentation structure

• 1. Introduction
• 2. Constrained Bayesian problems of the multiple

comparisons

• 3. Properties of the hypotheses acceptance regions
• 4. The method of sequential analysis of Bayesian type
• 5. Investigation of the method of sequential analysis of

Bayesian type

• 6. Experimental research

• 1. Introduction

We will consider the following methods

• f

sequential analysis:

Sequential methods of Wald (middle of forties of 20th

century); century);

Bayesian sequential procedures (end of forties of 20th

century);

Berger sequential tests (beginning of nineties of 20th

century);

The sequential analysis method of Bayesian type

(2010).

• 2. Constrained Bayesian problems of

the multiple comparisons

To minimize the averaged value of probabilities of incorrectly rejected hypotheses (1)

∑ ∫ ∑

≠ = Γ =

=

S i j j i S i i

j

d H p H p r

, 1 1

) | ( ) ( x x

δ

subject to the averaged value of incorrectly accepted hypotheses . (2)

∑ ∫ ∑

≠ = Γ = i j j i

j

, 1 1

α ≤ −

∫ ∑

Γ =

i

d H p H p

i S i i

x x ) | ( ) ( 1

1

Here is the number of tested hypotheses , supposed that the sample ( , is

• bservation

space) is generated by distribution , ;

S

i i

H θ θ = : ) ,..., ( 1

n T

x x = x

n

R ∈ x

n

R

≡ = ) ,..., ; ,..., ( ) , (

1 1 i k i n i

x x p p θ θ θ x ) | (

i

H p x

S i ,..., 1 =

, ; is the vector of parameters of distribution, ; ; is

• dimensional parametrical

space; in general, ; is the a priori probability of hypothesis ; is the region of acceptance of hypothesis ; is the maximum allowed level of the averaged value

• f

the probabilities

• f

incorrectly accepted hypotheses.

≡ = ) ,..., ; ,..., ( ) , (

1 1 k n

x x p p θ θ θ x ) | (

i

H p x

S i ,..., 1 =

) ,..., ( 1

k T

θ θ = θ

k i

Θ ∈ θ

k

Θ

k

k n ≠

) (

i

H p

i

H

i

Γ

i

H

α

S i i ,..., 1 : = ∀

Task (1), (2) is one of possible formulations of the constrained Bayesian problem. In a similar manner, we can introduce and solve different constrained Bayesian

• tasks. Solutions of these tasks differ to each other but,

generally, they can be written as follows: , , (3) where , .

} ) | ( ) | ( : {

, 1

∑

≠ =

> = Γ

S j j j j j j

H p k H p k

• x

x x

S j ,..., 1 =

+∞ < ≤

j

k

S ,..., 1 =

Investigation of these regions shows that they have identical properties, namely, in general case they can intersect and their union could not coincide with observation space. This fact allows us to use each of them for construction of the appropriate sequential method of testing of hypotheses. But, for simplicity, further we shall consider only the task (1), (2), though all results

• btained below, after appropriate modifications, are true for all
• ther

tasks also.

• ther

tasks also. For concreteness let us rewrite the regions of acceptance of hypotheses (3) for the task (1), (2): , (4) where ( ) the same scalar value for all regions, is determined so that in (2) the equality takes place.

{ }

S j H p H p H p H p

j j S j i i i i j

,..., 1 , ) | ( ) ( ) | ( ) ( :

, 1

= < = Γ

∑

≠ =

x x x λ

λ

+∞ < < λ

• 3. Properties of the hypotheses acceptance

regions

It is known that, in traditional statements of the problem

• f

statistical hypotheses testing, their acceptance regions are not intersected, i.e. , , and the union of all regions of acceptance of

∅ = Γ Γ

j i

j i ≠

, and the union of all regions of acceptance of hypotheses coincides with the observation space, i.e. . These conditions break down at consideration of above-formulated constrained Bayesian task

• f

hypotheses testing.

j i ≠

n S i i

R = Γ

=

1

Namely, for any value of ( ) in restriction (2) (that is the same, for any value of ( )), in the observation space exist: the regions of unambiguous acceptance of the tested hypotheses, the regions of the suspicion on the validity of several (more

α

1 < < α

λ

+∞ < < λ

n

R

regions of the suspicion on the validity of several (more than one) tested hypotheses (corresponding to sub- regions of the intersection of the regions of acceptance of corresponding hypotheses (4)) and the region

• f

impossibility of acceptance of the tested hypotheses (corresponding to the region of the space , which do not belong to any of the regions of acceptance of hypotheses (4)).

n

R

Accordingly, for any concrete observation result , on the basis of which the decision is made, in the interval there are such values that for the observation result belongs to only one

• f the regions of acceptance of hypotheses (4) and the

corresponding hypothesis is accepted, respectively. At , the observation result appears in a sub-region of intersection of two or several regions of acceptance of

x

) 1 ; (

x

) ( ) (

* *

x x α α ≤

)] ( ); ( [

* *

x x α α α ∈

) (

* x

α α <

x

intersection of two or several regions of acceptance of hypotheses (4), and it is impossible to make a unique decision. In that case, the appropriate hypotheses are suspected on the validity. At , the observation result appears in the region of the space which does not belong to any of the regions of acceptance of hypotheses (4). In this case, it is impossible to make the decision on the basis

• f the set observation result

. ) (

* x

α α >

x

n

R

x

Thus, the situation is similar to the sequential analysis when, on the basis of present observation results, it could be impossible to make a decision (with the given probabilities of errors) about the validity of one of the hypotheses from the considered set. Therefore, in the considered tasks, if there is the situation of impossibility considered tasks, if there is the situation of impossibility

• f making an unambiguous decision for the given

significance level, we shall continue the observations until such an opportunity appears. Thus, on the basis of above-considered constrained Bayesian task, let us determine the method of sequential analysis for multiple testing problems. For clarity let us call this method the sequential analysis method of Bayesian type.

• 4. The method of sequential analysis of

Bayesian type

Let us suppose that there is an opportunity of

• btaining repeated observations. Let

be the sampling space of all possible samples of indepen- dent -dimensional observation vectors . Let us split into disjoint sub-regions ,

n m

R

m

n

) ,..., ( 1

n T

x x = x

n

R

1 + S

n

R

Let us split into disjoint sub-regions , ,..., , such that .

n m

R

1 + S

n m

R

1 ,

n m

R

2 ,

n S m

R ,

n S m

R

1 , +

• 1

1 , + =

=

S i n i m n m

R R

Let us determine the following decision rule. If the matrix of observation results belongs to the sub-region , , then hypothesis is accepted, and, if belongs to the sub-region , the decision is not made, and the observations go

• n until one of the tested hypotheses is accepted.

) ,..., (

1 m

x x x =

n i m

R ,

S i ,..., 1 =

i

H

) ,..., (

1 m

x x x =

n S m

R

1 , +

Regions , are determined in the following way: , is such a part of acceptance region

• f hypothesis

which does not belong to any

• ther region

; is such a part

• f sampling space

which belongs simultaneously to more than one region , or it does not belong

1 ,..., 1 ,

,

+ = S i Rn

i m

S i Rn

i m

,..., 1 ,

,

=

m i

Γ

i

H S i i j

m j

,..., 1 , 1 ,..., 1 , + − = Γ

n S m

R

1 , + n m

R

S i

m i

,..., 1 , = Γ

to any

• f

these regions. Here the index ( ) points to the fact that the regions are determined on the basis of sequential observation results. On the basis of the properties described in Item 3, hypotheses acceptance regions , could be determined as follows.

i

m

,... 2 , 1 = m

m

1 ,..., 1 ,

,

+ = S i Rn

i m

Let us designate the population of sub-regions of intersections of acceptance regions

• f hypotheses

in constrained Bayesian task of hypotheses testing with the regions of acceptance of other hypotheses , , by . By , we designate the population of regions

m i

Γ

i

H ) ,..., 1 ( S i =

j

H

i j S j ≠ = ; ,..., 1

m i

I

• S

i m i n m n m

R E

1 = Γ

− =

• f space

which do not belong to any of hypotheses acceptance regions. Then the hypotheses acceptance regions in the method of sequential analysis of Bayesian type are determined in the following way: ; . (6) Here regions , are defined on the basis of hypotheses acceptance regions (4) from Item 2.

i

i m m

R E

1 = Γ

− =

n m

R

S i I R

m i m i n i m

,..., 1 , /

,

= Γ =

( )

• n

m S i m i n S m

E I R

1 1 , = + =

,

m i

Γ

,

m i

I ,

n m

E

S i ,..., 1 =

• 5. Investigation of the method of sequential

analysis of Bayesian type

For clarity, by and , we shall designate the probabilities of errors of the first and the second kinds for sequential method of Bayesian type, and, by and , the same quantities for constrained Bayesian

1

α

1

β

α

β

and , the same quantities for constrained Bayesian task.

β

Theorem 5.1. If the probability distribution , , is such that an increase in the sample size entails a decrease in the entropy concerning

) | (

i

H p x

S i ,..., 1 = entails a decrease in the entropy concerning distribution parameters about which the hypotheses are formulated, then infinitely increasing number of repeated observations, i.e. in the sequential analysis method

• f

Bayesian type, entails infinite decreasing probabilities of errors of the first and the second kinds, i.e. and .

m

θ

∞ → m

1 →

α

1 →

β

Lemma 5.1. In the conditions of Theorem 5.1, at increasing divergence between tested hypotheses and , , Lagrange coefficient in solution (4) decreases, and, in the limit, at , takes place for the given .

) , (

j i H

H J

i

H

j

H j i S j i ≠ = ; ,..., 1 ,

λ

{ }

∞ → ) , ( min

, j i j i

H H J

→ λ

α

Lemma 5.2. In the conditions of Theorem 5.1, at infinitely decreasing divergence between tested hypotheses and , , i.e. at Lagrange coefficient in solution (4) tends to a certain value from the interval: depending on the value of .

) , (

j i H

H J

i

H

j

H

j i S j i ≠ = ; ,..., 1 ,

{ }

) , ( max

,

→

j i j i

H H J

λ

{ } { }

     

∑ ∑

≠ = ≠ = S j i i j i j S j i i j i j

H p H p H p H p

, 1 , 1

) ( / ) ( max , ) ( / ) ( min

α

Hereinafter we shall suppose that probability distributions , , are such that increasing information causes a decrease in the entropy relative to parameter which the hypotheses are formulated about.

) | (

i

H p x

S i ,..., 1 =

θ

parameter which the hypotheses are formulated about. Theorem 5.2. For any given sample size and as small errors of the first and the second kinds and as one likes, there always exists such a positive value that, if the divergence between tested hypotheses is more than that value, i.e. , and hold true.

θ

m

α′

β′

*

J

{ }

* ,

) , ( min J H H J

j i j i

>

α α ′ < ) (

1 J

β β ′ < ) (

1 J

Theorem 5.3. For any value of in constrained Bayesian task there always exists such an integer that if the number of repeated observations , in the method of sequential analysis of Bayesian type, is more than this value, i.e. , there will be accepted one of the tested hypotheses with the probability equal to unity.

α

*

m

m

*

m m >

• 6. Experimental research

Suppose that and that it is desired to test versus . The computation results

• f

the sequentially processed sample generated by with 17

• bservations were used (Table 1, where the arithmetic

) 1 , 1 ( N

) 1 , ( ~ θ ξ N

1 : − = θ H 1 : = θ

A

H

• bservations were used (Table 1, where the arithmetic

mean of the observations is denoted by ). From here it is seen that the Wald and Berger’s tests yield absolutely the same results, though the reported error probabilities in the Berger’s and the Wald’s test differ to each other because Berger computed the error probabilities for the given value of the statistics.

m k

x x ,...,

m k

x ,

Out of 17 observations, correct decisions were taken 7 times on the basis of 3, 3, 5, 1, 1, 3 and 1 observations in times on the basis of 3, 3, 5, 1, 1, 3 and 1 observations in both tests (the Wald and Berger’s). The average value of

• bservations for making the decision is equal to 2.43. In

the sequential test of Bayesian type for the same sample correct decisions were taken 10 times on the basis of 1, 2, 2, 1, 3, 2, 1, 1, 3 and 1 observations. The average value of

• bservations for making the decision is equal to 1.7.

The reported error probabilities in the sequential test of Bayesian type and the Wald’s test decrease depending on the number of observations used for making the decision (see Table 2). By the Type II error probability it strongly surpasses the Wald’s test. While these characteristics for surpasses the Wald’s test. While these characteristics for the Berger’s test have no monotonous dependence on the number of observations (for the reason mentioned above). They basically are defined by the value of the likelihood ratio. For example, the value of the Type I error probability for 5 observations ( ) surpasses the analogous value for 3 observations and both

• f them surpass the same value for 1 observation

.

11 7,..., x

x

16 15 14

, , x x x

17

x