2 2 (4-1) = = k n k P k X k P ( k ) p - - PowerPoint PPT Presentation

1

• 4. Binomial Random Variable Approximations,

Conditional Probability Density Functions and Stirling’s Formula

Let X represent a Binomial r.v as in (3-42). Then from (2-30) Since the binomial coefficient grows quite rapidly with n, it is difficult to compute (4-1) for large n. In this context, two approximations are extremely useful. 4.1 The Normal Approximation (Demoivre-Laplace Theorem) Suppose with p held fixed. Then for k in the neighborhood of np, we can approximate

( )

∑ ∑

= − =

        = = ≤ ≤

2 1 2 1

. ) (

2 1 k k k k n k k k k n

q p k n k P k X k P

(4-1)

! )! ( ! k k n n k n − =        

∞ → n npq

PILLAI

2

(4-2)

. 2 1

2 / ) (

2

npq np k k n k

e npq q p k n

− − −

≈         π

Thus if and in (4-1) are within or around the neighborhood of the interval we can approximate the summation in (4-1) by an integration. In that case (4-1) reduces to where We can express (4-3) in terms of the normalized integral that has been tabulated extensively (See Table 4.1).

1

k

2

k

( ),

, npq np npq np + −

( )

, 2 1 2 1

2 / 2 / ) ( 2 1

2 2 1 2 2 1

dy e dx e npq k X k P

y x x npq np x k k − − −

∫ ∫

= = ≤ ≤ π π

(4-3)

) ( 2 1 ) (

2 /

2

x erf dy e x erf

x y

− = =

∫

−

π

(4-4)

. ,

2 2 1 1

npq np k x npq np k x − = − =

PILLAI

3

For example, if and are both positive ,we obtain Example 4.1: A fair coin is tossed 5,000 times. Find the probability that the number of heads is between 2,475 to 2,525. Solution: We need Here n is large so that we can use the normal approximation. In this case so that and Since and the approximation is valid for and Thus Here

( )

). ( ) (

1 2 2 1

x erf x erf k X k P − = ≤ ≤

1

x

2

x ). 525 , 2 475 , 2 ( ≤ ≤ X P

(4-5)

, 2 1 = p

500 , 2 = np

. 35 ≈ npq , 465 , 2 = − npq np , 535 , 2 = + npq np 475 , 2

1 =

k . 525 , 2

2 =

k

( ) ∫

−

= ≤ ≤

2 1 2

. 2 1

2 / 2 1 x x y

dy e k X k P π

. 7 5 , 7 5

2 2 1 1

= − = − = − = npq np k x npq np k x

PILLAI

4

2 1 ) ( 2 1 ) ( erf

2 /

2

− = =

∫

−

x G dy e x

x y

π

x erf(x) x erf(x) x erf(x) x erf(x)

0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.01994 0.03983 0.05962 0.07926 0.09871 0.11791 0.13683 0.15542 0.17364 0.19146 0.20884 0.22575 0.24215 0.25804 0.27337 0.80 0.85 0.90 0.95 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.45 1.50 0.28814 0.30234 0.31594 0.32894 0.34134 0.35314 0.36433 0.37493 0.38493 0.39435 0.40320 0.41149 0.41924 0.42647 0.43319 1.55 1.60 1.65 1.70 1.75 1.80 1.85 1.90 1.95 2.00 2.05 2.10 2.15 2.20 2.25 0.43943 0.44520 0.45053 0.45543 0.45994 0.46407 0.46784 0.47128 0.47441 0.47726 0.47982 0.48214 0.48422 0.48610 0.48778 2.30 2.35 2.40 2.45 2.50 2.55 2.60 2.65 2.70 2.75 2.80 2.85 2.90 2.95 3.00 0.48928 0.49061 0.49180 0.49286 0.49379 0.49461 0.49534 0.49597 0.49653 0.49702 0.49744 0.49781 0.49813 0.49841 0.49865

Table 4.1 PILLAI

5

Since from Fig. 4.1(b), the above probability is given by where we have used Table 4.1 4.2. The Poisson Approximation As we have mentioned earlier, for large n, the Gaussian approximation of a binomial r.v is valid only if p is fixed, i.e., only if and what if np is small, or if it does not increase with n?

,

1 <

x

( )

, 516 . 7 5 erf 2 |) (| erf ) ( erf ) ( erf ) ( erf 525 , 2 475 , 2

1 2 1 2

=       = + = − = ≤ ≤ x x x x X P 1 >> np . 1 >> npq

( ).

258 . ) 7 . ( erf =

• Fig. 4.1

x

(a)

1

x

2

x

2 /

2

2 1

x

e − π

,

2 1

> > x x

x

(b)

1

x

2

x

2 /

2

2 1

x

e − π

,

2 1

> < x x

PILLAI

6

Obviously that is the case if, for example, as such that is a fixed number. Many random phenomena in nature in fact follow this

• pattern. Total number of calls on a telephone line, claims in

an insurance company etc. tend to follow this type of

• behavior. Consider random arrivals such as telephone calls
• ver a line. Let n represent the total number of calls in the

interval From our experience, as we have so that we may assume Consider a small interval of duration ∆ as in Fig. 4.2. If there is only a single call coming in, the probability p of that single call occurring in that interval must depend on its relative size with respect to T.

→ p , ∞ → n λ = np

∞ → T

). , ( T ∞ → n . T n µ =

• 1

2 n

∆

T

• Fig. 4.2

PILLAI

7

Hence we may assume Note that as However in this case is a constant, and the normal approximation is invalid here. Suppose the interval ∆ in Fig. 4.2 is of interest to us. A call inside that interval is a “success” (H), whereas one outside is a “failure” (T ). This is equivalent to the coin tossing situation, and hence the probability of obtaining k calls (in any order) in an interval of duration ∆ is given by the binomial p.m.f. Thus and here as such that It is easy to

• btain an excellent approximation to (4-6) in that situation.

To see this, rewrite (4-6) as

. T p ∆ =

→ p . ∞ → T

λ µ µ = ∆ = ∆ ⋅ = T T np

) ( k Pn

, ) 1 ( ! )! ( ! ) (

k n k n

p p k k n n k P

−

− − =

(4-6)

, → ∞ → p n . λ = np

PILLAI

8

. ) / 1 ( ) / 1 ( ! 1 1 2 1 1 1 ) / 1 ( ! ) ( ) 1 ( ) 1 ( ) (

k n k k n k k n

n n k n k n n n np k np n k n n n k P λ λ λ − −       − −       −       − = − + − − =

−

• (4-7)

Thus

, ! ) ( lim

, , λ λ

λ

− = → ∞ →

= e k k P

k n np p n

(4-8)

since the finite products as well as tend to unity as and The right side of (4-8) represents the Poisson p.m.f and the Poisson approximation to the binomial r.v is valid in situations where the binomial r.v parameters n and p diverge to two extremes such that their product np is a constant.

      − −       −       − n k n n 1 1 2 1 1 1

• k

n       − λ 1

, ∞ → n

. 1 lim

λ

λ

− ∞ →

=       − e n

n n

) , ( → ∞ → p n

PILLAI

9

Example 4.2: Winning a Lottery: Suppose two million lottery tickets are issued with 100 winning tickets among

• them. (a) If a person purchases 100 tickets, what is the

probability of winning? (b) How many tickets should one buy to be 95% confident of having a winning ticket? Solution: The probability of buying a winning ticket Here and the number of winning tickets X in the n purchased tickets has an approximate Poisson distribution with parameter Thus and (a) Probability of winning

. 10 5 10 2 100 tickets

• f

no. Total tickets winning

• f

No.

5 6 −

× = × = = p

, 100 = n . 005 . 10 5 100

5 =

× × = =

−

np λ , ! ) ( k e k X P

k

λ

λ −

= =

. 005 . 1 ) ( 1 ) 1 ( ≈ − = = − = ≥ =

−λ

e X P X P

PILLAI

10

(b) In this case we need But or Thus one needs to buy about 60,000 tickets to be 95% confident of having a winning ticket! Example 4.3: A space craft has 100,000 components The probability of any one component being defective is The mission will be in danger if five or more components become defective. Find the probability of such an event. Solution: Here n is large and p is small, and hence Poisson approximation is valid. Thus and the desired probability is given by

. 95 . ) 1 ( ≥ ≥ X P

. 3 20 ln implies 95 . 1 ) 1 ( = ≥ ≥ − = ≥

−

λ

λ

e X P

3 10 5

5 ≥

× × = =

−

n np λ . 000 , 60 ≥ n

( )

∞ → n

). ( 10 2

5

→ ×

−

p , 2 10 2 000 , 100

5 =

× × = =

−

λ np

PILLAI

11

. 052 . 3 2 3 4 2 2 1 1 ! 1 ! 1 ) 4 ( 1 ) 5 (

2 4 2 4

=       + + + + − = − = − = ≤ − = ≥

− = − = −

∑ ∑

e k e k e X P X P

k k k k

λ λ

λ

Conditional Probability Density Function For any two events A and B, we have defined the conditional probability of A given B as Noting that the probability distribution function is given by we may define the conditional distribution of the r.v X given the event B as

. ) ( , ) ( ) ( ) | ( ≠ ∩ = B P B P B A P B A P

(4-9)

) ( x FX

{ },

) ( ) ( x X P x F X ≤ = ξ

(4-10)

PILLAI

12

{ } ( ) { }.

) ( ) ( | ) ( ) | ( B P B x X P B x X P B x F X ∩ ≤ = ≤ = ξ ξ

Thus the definition of the conditional distribution depends

• n conditional probability, and since it obeys all probability

axioms, it follows that the conditional distribution has the same properties as any distribution function. In particular Further

( ) { } ( ) { }

. ) ( ) ( ) ( ) ( ) | ( , 1 ) ( ) ( ) ( ) ( ) | ( = = ∩ −∞ ≤ = −∞ = = ∩ +∞ ≤ = +∞ B P P B P B X P B F B P B P B P B X P B F

X X

φ ξ ξ

(4-12) (4-11)

( ) { }

), | ( ) | ( ) ( ) ( ) | ) ( (

1 2 2 1 2 1

B x F B x F B P B x X x P B x X x P

X X

− = ∩ ≤ < = ≤ < ξ ξ

(4-13)

PILLAI

13

Since for The conditional density function is the derivative of the conditional distribution function. Thus and proceeding as in (3-26) we obtain Using (4-16), we can also rewrite (4-13) as

,

1 2

x x ≥

( ) ( ) ( ).

) ( ) ( ) (

2 1 1 2

x X x x X x X ≤ < ∪ ≤ = ≤ ξ ξ ξ

(4-14)

, ) | ( ) | ( dx B x dF B x f

X X

=

(4-15)

∫

∞ −

=

x X X

du B u f B x F . ) | ( ) | (

(4-16)

( ) ∫

= ≤ <

2 1

2 1

. ) | ( | ) (

x x X

dx B x f B x X x P ξ

(4-17)

PILLAI

14

Example 4.4: Refer to example 3.2. Toss a coin and X(T)=0, X(H)=1. Suppose Determine Solution: From Example 3.2, has the following form. We need for all x. For so that and

). | ( B x FX ) (x FX ) | ( B x FX

{ }

, ) ( , φ ξ = ≤ < x X x

( ) { }

, ) ( φ ξ = ∩ ≤ B x X . ) | ( = B x FX

}. {H B =

) (x F

X

x

(a)

q

1 1

( | )

X

F x B

• Fig. 4.3

1

x

(b) 1 PILLAI

15

For so that For and (see Fig. 4.3(b)). Example 4.5: Given suppose Find Solution: We will first determine From (4-11) and B as given above, we have

{ } { }

, ) ( , 1 T x X x = ≤ < ≤ ξ

( ) { } { } { }

φ ξ = ∩ = ∩ ≤ H T B x X ) (

. ) | ( and = B x FX { }

, ) ( , 1 Ω = ≤ ≥ x X x ξ

( ) { } { }

} { ) ( B B B x X = ∩ Ω = ∩ ≤ ξ

1 ) ( ) ( ) | ( and = = B P B P B x FX

), (x FX

{ }

. ) ( a X B ≤ = ξ ). | ( B x f X ). | ( B x FX

( ) ( ) { } ( )

. ) | ( a X P a X x X P B x F X ≤ ≤ ∩ ≤ =

(4-18)

PILLAI

16

( ) ( ) ( )

x X a X x X a x ≤ = ≤ ∩ ≤ < ,

( ) ( )

. ) ( ) ( ) | ( a F x F a X P x X P B x F

X X X

= ≤ ≤ =

( ) ( )

) ( , a X a X x X a x ≤ = ≤ ∩ ≤ ≥

. 1 ) | ( = B x F X      ≥ < = , , 1 , , ) ( ) ( ) | ( a x a x a F x F B x F

X X X

For so that For so that Thus and hence

(4-19) (4-20)

     < = =

• therwise.

, , , ) ( ) ( ) | ( ) | ( a x a F x f B x F dx d B x f

X X X X

(4-21)

PILLAI

17

) | ( B x FX ) (x FX

x a 1

(a)

• Fig. 4.4

) | ( B x f X ) (x f X

x a

(b)

Example 4.6: Let B represent the event with For a given determine and Solution: For we have and hence

{ }

b X a ≤ < ) ( ξ . a b > ), (x FX ) | ( B x FX ). | ( B x f X

{ } ( ) ( ) { } ( ) ( ) ( ) { }.

) ( ) ( ) ( ) ( ) ( ) ( ) ( | ) ( ) | ( a F b F b X a x X P b X a P b X a x X P B x X P B x F

X X X

− ≤ < ∩ ≤ = ≤ < ≤ < ∩ ≤ = ≤ = ξ ξ ξ ξ ξ ξ

(4-22)

, a x <

{ } { }

, ) ( ) ( φ ξ ξ = ≤ < ∩ ≤ b X a x X . ) | ( = B x FX

(4-23)

PILLAI

18

For we have and hence For we have so that Using (4-23)-(4-25), we get (see Fig. 4.5)

{ } { }

} ) ( { ) ( ) ( x X a b X a x X ≤ < = ≤ < ∩ ≤ ξ ξ ξ

( )

. ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) | ( a F b F a F x F a F b F x X a P B x F

X X X X X X X

− − = − ≤ < = ξ

, b x a < ≤ , b x ≥

{ } { } { }

b X a b X a x X ≤ < = ≤ < ∩ ≤ ) ( ) ( ) ( ξ ξ ξ . 1 ) | ( = B x F X

(4-24) (4-25)

     ≤ < − =

• therwise.

, , , ) ( ) ( ) ( ) | ( b x a a F b F x f B x f

X X X X

(4-26)

) | ( B x f X ) (x f X

x

• Fig. 4.5

a b

PILLAI

19

We can use the conditional p.d.f together with the Bayes’ theorem to update our a-priori knowledge about the probability of events in presence of new observations. Ideally, any new information should be used to update our

• knowledge. As we see in the next example, conditional p.d.f

together with Bayes’ theorem allow systematic updating. For any two events A and B, Bayes’ theorem gives Let so that (4-27) becomes (see (4-13) and

(4-17))

. ) ( ) ( ) | ( ) | ( B P A P A B P B A P =

(4-27)

{ }

2 1

) ( x X x B ≤ < = ξ

{ } ( ) ( )

). ( ) ( ) | ( ) ( ) ( ) ( ) | ( ) | ( ) ( ) ( | ) ) ( ( ) ) ( ( |

2 1 2 1

1 2 1 2 2 1 2 1 2 1

A P dx x f dx A x f A P x F x F A x F A x F x X x P A P A x X x P x X x A P

x x X x x X X X X X

∫ ∫

= − − = ≤ < ≤ < = ≤ < ξ ξ ξ

(4-28)

PILLAI

20

Further, let so that in the limit as

• r

From (4-30), we also get

• r

and using this in (4-30), we get the desired result

, , ,

2 1

> + = = ε ε x x x x , → ε

{ } ( )

). ( ) ( ) | ( ) ( | ) ) ( ( | lim A P x f A x f x X A P x X x A P

X X

= = = + ≤ <

→

ξ ε ξ

ε

(4-29)

. ) ( ) ( ) | ( ) | (

|

A P x f x X A P A x f

X A X

= =

(4-30) (4-31)

, ) ( ) | ( ) | ( ) (

1

dx x f x X A P dx A x f A P

X X

∫ ∫

+∞ ∞ − +∞ ∞ −

= =

• dx

x f x X A P A P

X

) ( ) | ( ) (

∫

+∞ ∞ −

= =

(4-32)

. ) ( ) | ( ) ( ) | ( ) | (

|

∫

∞ + ∞ −

= = = dx x f x X A P x f x X A P A x f

X X A X

(4-33)

PILLAI

21

To illustrate the usefulness of this formulation, let us reexamine the coin tossing problem. Example 4.7: Let represent the probability of

• btaining a head in a toss. For a given coin, a-priori p can

possess any value in the interval (0,1). In the absence of any additional information, we may assume the a-priori p.d.f to be a uniform distribution in that interval. Now suppose we actually perform an experiment of tossing the coin n times, and k heads are observed. This is new information. How can we update Solution: Let A= “k heads in n specific tosses”. Since these tosses result in a specific sequence,

) (H P p = ) ( p fP ? ) ( p fP

) ( p fP p 1

Fig.4.6

, ) | (

k n kq

p p P A P

−

= =

(4-34)

PILLAI

22

and using (4-32) we get The a-posteriori p.d.f represents the updated information given the event A, and from (4-30) Notice that the a-posteriori p.d.f of p in (4-36) is not a uniform distribution, but a beta distribution. We can use this a-posteriori p.d.f to make further predictions, For example, in the light of the above experiment, what can we say about the probability of a head occurring in the next (n+1)th toss?

. )! 1 ( ! )! ( ) 1 ( ) ( ) | ( ) (

1 1

+ − = − = = =

∫ ∫

−

n k k n dp p p dp p f p P A P A P

k n k P

(4-35)

| (

| )

P A

f p A

). , ( 1 , ! )! ( )! 1 ( ) ( ) ( ) | ( ) | (

|

k n p q p k k n n A P p f p P A P A p f

k n k P A P

β < < − + = = =

−

(4-36)

) | (

|

A p f

A P

p

• Fig. 4.7

1

∼

PILLAI

23

Let B= “head occurring in the (n+1)th toss, given that k heads have occurred in n previous tosses”. Clearly and from (4-32) Notice that unlike (4-32), we have used the a-posteriori p.d.f in (4-37) to reflect our knowledge about the experiment already performed. Using (4-36) in (4-37), we get Thus, if n =10, and k = 6, then which is more realistic compare to p = 0.5.

, ) | ( p p P B P = =

∫

= =

1

. ) | ( ) | ( ) ( dp A p f p P B P B P

P

(4-37)

∫

+ + = − + ⋅ =

− 1

. 2 1 ! )! ( )! 1 ( ) ( n k dp q p k k n n p B P

k n k

(4-38)

, 58 . 12 7 ) ( = = B P

PILLAI

24

To summarize, if the probability of an event X is unknown,

• ne should make noncommittal judgement about its a-priori

probability density function Usually the uniform distribution is a reasonable assumption in the absence of any

• ther information. Then experimental results (A) are
• btained, and out knowledge about X must be updated

reflecting this new information. Bayes’ rule helps to obtain the a-posteriori p.d.f of X given A. From that point on, this a-posteriori p.d.f should be used to make further predictions and calculations.

). (x f X ) | (

|

A x f

A X

PILLAI

25

Stirling’s Formula : What is it? Stirling’s formula gives an accurate approximation for n! as follows: in the sense that the ratio of the two sides in (4-39) is near to one; i.e., their relative error is small, or the percentage error decreases steadily as n increases. The approximation is remarkably accurate even for small n. Thus 1! = 1 is approximated as and is approximated as 5.836. Prior to Stirling’s work, DeMoivre had established the same formula in (4-39) in connection with binomial distributions in probability theory. However DeMoivre did not establish the constant

1 2

! ~ 2

n n

n n e π

+ −

(4-39)

2 / 0.9221, e π

• PILLAI

3! 6 =

26

term in (4-39); that was done by James Stirling ( 1730). How to prove it? We start with a simple observation: The function log x is a monotone increasing function, and hence we have Summing over we get

• r

2π

• 1

1log

log log .

k k k k

x dx k x dx

+ −

< <

∫ ∫

1, 2, , k n =

• 1

1

log log ! log

n n

x dx n x dx

+

< <

∫ ∫

log log ! ( 1)log( 1) . n n n n n n n − < < + + −

(4-40)

PILLAI

log x logk 1 k+ 1 k− k x

1

( )

log log xdx x x x = −

∫

27

The double inequality in (4-40) clearly suggests that log n! is close to the arithmetic mean of the two extreme numbers

• there. However the actual arithmetic mean is complicated

and it involves several terms. Since (n + )log n – n is quite close to the above arithmetic mean, we consider the difference1 This gives

1According to W. Feller this clever idea to use the approximate mean (n + )log n – n is due

to H.E. Robbins, and it leads to an elementary proof.

(4-41)

1 1 2 2 1 1 2 2

1 2 1 3 2 3 1 2 2 1 1 1 2 2

log ! ( )log log( 1)! ( )log( 1) ( 1) log( 1) ( )log ( )log( 1) 1 ( )log 1 ( )log 1.

n n n n n n

a a n n n n n n n n n n n n n n n

+ + + + + −

− = − + + − + + + + − + = − + − + + + + − = + − = + −

PILLAI

1 2 1 2

log ! ( )log .

n

a n n n n − + +

• 1

2

28

PILLAI

( )

1 2 1 1 2 1

1 1 2 1 1 1 2 3 5 2 4 6

1 1 1 2 1 3(2 1) 5(2 1) 1 1 1 3(2 1) 5(2 1) 7(2 1)

( )log( ) 1 2( ) 1

n n

n n

n n n n n n

a a n n

+ +

+ + −

+ + + + + +

− = + − = + + + + − = + + + >

• (4-42)

Thus {an} is a monotone decreasing sequence and let c represent its limit, i.e., From (4-41), as this is equivalent to To find the constant term c in (4-44), we can make use of a formula due to Wallis ( 1655).

1By Taylor series expansion

lim

n n

a c

→∞

= n → ∞

1 2

! ~ .

n c n

n e n e

+ −

(4-44) (4-43)

• Hence1

2 3 4 2 3 4 1 (1 )

1 1 1 1 1 1 log(1 ) ; log 2 3 4 2 3 4

x

x x x x x x x x x

−

+ = − + − + = + + + +

29

The well known function goes to zero at moreover these are the only zeros of this function. Also has no finite poles. (All poles are at infinity). As a result we can write

• r [for a proof of this formula, see chapter 4 of Dienes,

The Taylor Series] which for gives the Wallis’ formula

sinc

sin

x

x x

=

; x nπ = ±

sin x x

/ 2 x π =

2 2 2 2 2 2 2

4

sin (1 )(1 ) (1 ) ,

x x x n

x x

π π π

= − − −

• 2

2 2 2 2 2 2

4

sin (1 )(1 ) (1 )

x x x n

x x

π π π

= − − −

• 2

2 2 2 2 2 2

(2 1) (2 1) 13 35 5 7 1 1 1 2 4 (2 ) 2 4 6 (2 )

2 2

1 )(1 ) (1 ) ( )( )( ) ( )

n n n n

π π

− ⋅ + ⋅ ⋅ ⋅

= − − − =

• sin x

x

x π π − nπ nπ −

1 (

PILLAI

30

Thus as this gives Thus as But from (4-41) and (4-43) and hence letting in (4-45) and making use

2 2 2 2 2

(2 ) 4 6 2 4 (2 1)(2 1) 1 3 3 5 5 7 (2 1) (2 1) 1 2 2 4 6 2 1 1 3 5 (2 1) 2 1

( )( )( ) ( ) 2 ( ) .

n n n n n n n n n n

π

∞ − + ⋅ ⋅ ⋅ − ⋅ + = ⋅ ⋅ ⋅ ⋅ − +

= = =

∏

• 2

2 2

1 1 2 1 1 . 1 1 2 2

2 4 6 2 1 3 5 (2 1) (2 4 6 2 ) ( !) (2 )! (2 )!

2 n

n n n

n n n n n n

π

+ + +

⋅ ⋅ ⋅ ⋅ − ⋅ ⋅

= = =

• 1

2

1 2

log 2 log2 2log ! log(2 )! log( ) n n n n π = + − − +

1 2

lim log ! lim {( )log }

n n

n c n n n

→∞ →∞

= + + −

(4-45) (4-46)

n → ∞

PILLAI

, n → ∞

• r

n → ∞

31

PILLAI

• f (4-46) we get

which gives With (4-47) in (4-44) we obtain (4-39), and this proves the Stirling’s formula. Upper and Lower Bounds It is possible to obtain reasonably good upper and lower bounds for n! by elementary reasoning as well. To see this, note that from (4-42) we get so that {an – 1/12n} is a monotonically increasing

1 2

1 1 1 2 2 2

log log2 lim log(1 ) log2

n n

c c π

→∞

= − − + = −

(4-47)

2 .

c

e π =

1 2

1 1 2 4 (2 1) (2 1)

1 3 1 1 1 1 12 12 ( 1) 12( 1) 3[(2 1) 1]

n n

n n

n n n n n

a a +

    +   + +  

+ + + −

− < = = = −

32

PILLAI

sequence whose limit is also c. Hence for any finite n and together with (4-41) and (4-47) this gives Similarly from (4-42) we also have so that {an – 1/(12n+1)} is a monotone decreasing sequence whose limit also equals c. Hence

• r

Together with (4-48)-(4-49) we obtain

1 1 12 12

n n

n n

a c a c − < ⇒ < +

( )

2

1 1 12 2

1

! 2

n

n n

n n e π

− − +

<

(4-48)

2 1

1 1 1 12 1 12( 1) 1 3(2 1)

n n

n n n

a a +

>

− + + + +

− > >

1 1 12 1 12 1

n n

n n

a c a c

> ⇒

+ +

− > +

(4-49)

1 1 12 ( 1) 2

(1 )

! 2 .

n n

n n

n n e π

+

− − +

>

33

PILLAI

Stirling’s formula also follows from the asymptotic expansion for the Gamma function given by Together with the above expansion can be used to compute numerical values for real x. For a derivation of (4-51), one may look into Chapter 2

• f the classic text by Whittaker and Watson (Modern

Analysis). We can use Stirling’s formula to obtain yet another approximation to the binomial probability mass ( 1) ( ), x x x Γ + = Γ

1 1 1 1 12 1 12 2 2

2 ! 2 .

n n

n n n n

n e e n n e e π π

+

+ + − −

< <

(4-50) (4-51)

{ }

1 2

2 3 4

139 1 1 1 12 286 51840

( ) 2 1 ( )

x x

x x x x

x e x

• π

− −

Γ = + + − +

34

• function. Since

using (4-50) on the right side of (4-52) we obtain and where and Notice that the constants c1 and c2 are quite close to each other.

( ) ( )

1

2 ( )

k n k k n k

np nq n n k k k n k

n p q c k

π

− −

− −

  >     ! , ( )! !

k n k k n k

n n p q p q n k k k

− −

  =   −  

(4-52)

{ }

1 1 1 12 1 12( ) 12

1

n n k k

c e

+ −

− −

=

{ }

1 1 1 12 12( ) 1 12 1

2

.

n n k k

c e

− + +

− −

=

( ) ( )

2

2 ( )

k n k k n k

np nq n n k k k n k

n p q c k

π

− −

− −

  <    

PILLAI