[PPT] - Estimation of moment-based models with latent variables work in PowerPoint Presentation

SLIDE 1

Estimation of moment-based models with latent variables

work in progress Ra¤aella Giacomini and Giuseppe Ragusa

UCL/Cemmap and UCI/Luiss

UPenn, 5/4/2010

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 1 / 35

SLIDE 2

Dynamic latent variables in macroeconomic models

E.g., time-varying parameters, structural shocks, stochastic volatility etc. Typical parametric setting: X T = (X1, ..., XT ) = (Y T , ZT ), Y T observable, ZT latent Joint density p(X, θ0) = p(Y T jZT , θ0)p(ZT , θ0) =

)

estimation of θ0 based on integrated likelihood b θ = arg max

θ

Z

p(Y T jZT , θ)p(ZT , θ)dZT Integrated likelihood computed by state-space methods

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 2 / 35

SLIDE 3

Existing state-space methods

State equation ! p(ZT , θ)

known in closed form

Observation equation ! p(Y T jZT , θ) "…ltering" density

known in closed form (e.g. Kalman …lter) or easy to simulate

Integral can be computed by MCMC methods

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 3 / 35

SLIDE 4

State-space methods for limited information models?

We consider the following scenario: p(ZT , θ) known ! state equation same as before p(Y T jZT , θ) unknown. Only information about θ is in the form

f (non-linear) moment conditions

Et1 [g(Yt, Zt, θ)] = 0

! substitute observation equation with moment conditions

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 4 / 35

SLIDE 5

Applications. GMM with time-varying parameters

Example #1. Time-varying "structural" parameters: E [g(Yt, βt)] = 0 βt = Φβt1 + εt, εt iidN(0, Σ) E [] de…ned with respect to joint distribution of Yt and βt Want to estimate θ = (Φ, Σ) and sequence of "smoothed" βt Application: Cogley and Sbordone’s (2005) analysis of stability

f structural parameters in a Calvo model of in‡ation

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 5 / 35

SLIDE 6

Applications. "Robust" stochastic volatility

estimation

Example #2. Yt = σtεt log

σ2

t

= α + β log

σ2

t1

+ vt, vt iidN(0, 1) Existing estimation methods require distributional assumption on εt (typically N(0, 1)) Problem: does not capture "fat tails" of …nancial data =

)

include jumps or use fat-tailed distribution for εt (not as straightforward as in GARCH case) Our method is robust to misspeci…cation in distribution of εt

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 6 / 35

SLIDE 7

Applications. Nonlinear DSGE models

Example #3. Prototypical DSGE model. Optimality conditions: Et1 [m(Yt, St, Zt, β)]

=

St

=

f (St1, Yt, Zt, β) Zt

=

ΦZt1 + εt, εt iidN(0, Σ) Want to estimate θ = (β, Φ, Σ) Yt = observable variables St = endogenous latent variables Zt = exogenous latent variables m () and f () known

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 7 / 35

SLIDE 8

An and Schorfheide (2007) DSGE model

In AS model, the endogenous latent variable equation has a simple form: St = f (Yt, Zt, β) (1) Can substitute St and rewrite the equilibrium conditions as Et1 [g(Yt, Zt, β)] = 0 Zt = ΦZt1 + εt, εt iid N(0, Σ) Warning: not all DSGEs …t this framework

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 8 / 35

SLIDE 9

Existing approaches to estimation of DSGE models

1

Theory does not provide likelihood

! must use approximation

methods

2

Linearize around steady state (Smets and Wouters, 2003; Woodford, 2003)

Solve the model to …nd policy functions Yt = h(St, Zt) Construct likelihood by Kalman …lter

3

Nonlinear approximations (Fernandez-Villaverde and Rubio-Ramirez, 2005)

Solve the model (numerically or analytically in the case of second order approximations around steady state) to …nd policy functions Construct likelihood by nonlinear state-space methods (e.g., particle …lter)

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 9 / 35

SLIDE 10

Drawbacks of existing likelihood-based approaches

1

Linearization = possible loss of information (Fernandez-Villaverde and Rubio-Ramirez, 2005)

2

Must impose structure to solve the model

1

Add "shocks"/measurement error to avoid stochastic singularity

2

Restrict parameters to rule out indeterminacy (multiple rational expectations solutions)

3

Nonlinear state-space methods computationally intensive (must solve the model for each parameter draw) =

) so far mostly

applied to simple models

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 10 / 35

SLIDE 11

Relationship with simulation-based method of moments

GMM, SMM, EMM, Indirect inference (eg, Ruge-Murcia, 2010) Di¤erence: requires knowledge of p(Y T jZT ) or focuses on moments of the type EY [g (Y , β)] = 0, (2) where g (Y , β) can be computed by simulation In our case, the model gives EY,Z [m (Y , Z, β)] = 0

= ) can be written as (2) only if p(ZjY ) known

Unlike these methods, we directly obtain estimates of the smoothed latent variables

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 11 / 35

SLIDE 12

The idea

Propose methods for estimating non-linear moment-based models that "exploit" the information contained in the moment conditions Methods are:

1

Computationally convenient

2

Classical or Bayesian

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 12 / 35

SLIDE 13

Key elements of methodology

Recall problem we want to solve (e.g., classical framework) max

θ

Z

p(Y T jZT , θ) p(ZT , θ)dZT

" unknown " known

Two steps:

1

Approximate the unknown likelihood p(Y T jZ T , θ)

2

Integrate out the latent variables using classical or Bayesian methods

3

For DSGEs: from an exact likelihood of the approximate model.... to an approximate likelihood of the exact model

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 13 / 35

SLIDE 14

Approximate likelihoods

We consider two di¤erent approximation strategies Both use projection theory (for no latent variables, Kim (2002), Chernozhukov and Hong (2003), Ragusa (2009)): out of all probability measures satisfying the moment conditions, choose the one that minimizes the Kullback-Leibler information distance Method 1 does not require solving the model (but not applicable to models with dynamic latent endogenous variables) Method 2 applicable to all models but requires solution of (approximate) model

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 14 / 35

SLIDE 15

Approximate likelihoods - Method 1

Find density that satis…es moment conditions and minimizes distance from the true density: gives approximate likelihood e p(Y T jZT , θ) ∝ exp

1

2g0

T

Y T , ZT , θ
V 1

T

Y T , ZT , θ
gT
Y T , ZT , θ
gT
ZT , θ
=

1

p

T

∑

t=1

g(Yt, Zt, θ) wt1 VT

Y T , ZT , θ
=

Var(gT

Y T , ZT , θ
), wt1 instruments

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 15 / 35

SLIDE 16

Approximate likelihoods - Method 1

e p(Y T jZT , θ) ∝ exp

1

2g0

T

Y T , ZT , θ
V 1

T

Y T , ZT , θ
gT
Y T , ZT , θ
e

p(Y T jZT , θ) is a simple transformation of the GMM objective function. Intuition:

When (Z T , θ) is consistent with the model gT

Y T , Z T , θ

= ) e p(Y T jZ T , θ) close to max value of 1. When (Z T , θ) is inconsistent with the moment conditions = ) large values of g 0

T

Y T , Z T , θ
V 1

T

Y T , Z T , θ
gT
Y T , Z T , θ

= ) e p(Y T jZ T , θ) 0.

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 16 / 35

SLIDE 17

Approximate likelihoods - Method 2

Write p(Y T jZT ) = ΠT

t=1p(YtjZt, Y t1)

Choose approximate density b p(YtjZt, Y t1, θ) (does not need to satisfy moment condition but easy to calculate) - For DSGEs, e.g., linearize model around steady state and apply Kalman …lter =

) b

p(YtjZt, Y t1, θ) are the …ltered densities

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 17 / 35

SLIDE 18

Approximate likelihoods - Method 2

"Tilt" b p(YtjZt, Y t1, θ) towards moment condition Et1 [g(Yt, Zt, θ)] = 0 , new density e p() satis…es moment condition and minimizes Kullback Leibler distance from b p() : Solve problem: min

h2H

Z Z

log h(YtjZt, Y t1) b p (YtjZt, Y t1, θ)

b

p

YtjZt, Y t1, θ
dYtdF
Zt,

s.t.

Z Z

g(Yt, Zt, θ)h(YtjZt, Y t1)dYtdF (Zt) = 0

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 18 / 35

SLIDE 19

Approximate likelihoods - Method 2

Under regularity conditions the solution is e p(YtjZt, Y t1, θ)

=

exp fηt + λtg (Yt, Zt, θ)g b p(YtjZt, Y t1, θ) where

(ηt, λt) = arg min

η,λ

Z

exp fη + λg (Yt, Zt, θ)g b p(YtjZt, Y t1, θ)dY λt = "weights for each moment condition"; ηt = integration constant

(ηt, λt) are functions of Zt, Y t1, θ

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 19 / 35

SLIDE 20

Approximate likelihoods - Method 2

e p(YtjZt, Y t1, θ)

=

exp fηt + λtg (Yt, Zt, θ)g b p(YtjZt, Y t1, θ In practice, approximate integral and compute (ηt, λt) by simulating N times from b p(YtjZt, Y t1, θ) =

) (ηt, λt) = arg min

η,λ

1 N

N

∑

i=1

exp n η + λg

Y (i)

t

, Zt, θ

Well-behaved objective function =

) for DSGEs, small additional

computational cost relative to Kalman …lter (cf. particle …lter?)

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 20 / 35

SLIDE 21

The two methods in a simple case

No latent variables, Y T = (Y1, ..., YT ) mean µ0, variance σ2 Moment condition identifying parameters are g1(Yt, µ, σ2)

=

Yt µ g2(Yt, µ, σ2)

=

Y 2

t σ2

Method 1:

b

µ, b σ2

=

arg max

θ=(µ,σ2) exp

1

2g0

T

Y T , θ
V 1

T

Y T , θ
gT
Y T , θ
=

) our estimator is same as GMM (Chernozhukov and Hong

(2003))

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 21 / 35

SLIDE 22

The two methods in a simple case

Method 2: Start from pdf of N(µ, σ2) : b p (Yt) =

1 p 2πσ exp

n

1

2σ (Yt µ)2o

and "tilt it" towards moment conditions e p (Yt)

=

exp

η + λ1 (Yt µ) + λ2
Y 2

t σ2

1

p

2πσ e 1

2 (Yt

λ1

=

µ0 σ0

µ

σ; λ2

=

1 2σ 1 2σ0 No tilting if µ = µ0, σ2 = σ2 In this case e p (Yt) N(µ0, σ2

0) =

) our estimator is the same

as (Q)MLE Normality here is a special result - e p () no longer normal if e.g., g () non-linear

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 22 / 35

SLIDE 23

Step 2. Integrate out latent variables

Classical estimation approach: solve b θ = max

θ

Z

e p(Y T jZT , θ)p(ZT , θ)dZT

using Jacquier, Johannes and Polson (2007) to compute integral here works well in our limited experience

Bayesian estimation approach: assume prior for θ (and Z0), π(θ) and calculate the approximate posterior e p(θ, ZT jY T ) ∝ e p(Y T jZT , θ)p(ZT jθ)π(θ) Integration of latent variables step is the same as previous literature

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 23 / 35

SLIDE 24

Econometric properties

For method 2 (tilted density), can show that MLE based on approximate integrated likelihood e p(Y T , θ) is consistent for θ = arg min

θ

Z

log e p(Y T , θ) p(Y T )

p(Y T )dY T

θ = parameter that sets the approximate density that is consistent with the moment conditions as close as possible to true density In particular if moment condition uniquely identi…es parameter θ0, by construction θ = θ0

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 24 / 35

SLIDE 25

Econometric properties

Back to simple example: Yt iid(µ0, σ2

0),

g(Yt, θ) = (Yt µ, Y 2

t σ2), initial density b

p N(µ, σ2) If tilt towards both moments, approximate density e p N(µ0, σ2

0) =

) our estimator (=QMLE) consistent for true

parameters What if tilt towards only one moment condition?

E.g., only use g2(Yt, θ) = Y 2

t σ2 =

) e p N( µ

σσ0, σ2 0)

Variance estimated consistently; mean not estimated consistently Suggests that not using moments can cause distortions = ) need to understand tradeo¤s between too many/too few moments

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 25 / 35

SLIDE 26

Econometric properties

Hypothesis testing, model selection relatively straightforward for method 2 E.g., could test whether λ (or individual components) = 0 , understand importance of non-linearities in DSGE models Open issue: identi…cation (here assumed but challenging because of presence of latent variables + nonlinearity of moment conditions)

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 26 / 35

SLIDE 27

Method 1 in a simple example

Data-generating process Yt

=

.9Zt + vt iid N(0, 1) Zt

=

.9Zt1 + εt iid N(0, 1) Moment condition E[Zt(Yt βZt)]

=

Zt

=

ρZt1 + εt iid N(0, 1) g(Yt, Zt, β) = Zt(Yt βZt) Priors: β U(0, 2), ρ U(0, 1), Z0 N(0,

1 1ρ2 ), T = 100

Use Jacquier et al. (2007)

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 27 / 35

SLIDE 28

−3 −2 −1 1 2 3 0.0 0.5 1.0 1.5 2.0 2.5 3.0

Distribution of β

c(−2, −2) Density

SLIDE 29

−1.0 −0.5 0.0 0.5 1.0 2 4 6 8 10

Distribution of ρ

ρ Density

SLIDE 30

20

40 60 80 100 4 2 −2 −4 −6

Smoothed Probabilities

Time x

Smoothed p(z|x)

Actual z

SLIDE 31

Simulation: AS New Keynesian model

1 = βEt

eτˆ

ct+1+τˆ ct+ ˆ Rtˆ zt+1 ˆ πt+1

(3) 1 ν νφπ2 (eτˆ

ct 1)

= (e ˆ

πt 1)

1 1

2ν

e ˆ

πt + 1

2ν

(4)

βE

(e ˆ

πt+1 1)eτˆ ct+1+τˆ ct+ˆ yt+1 ˆ yt+ ˆ πt+1

e ˆ

ctˆ yt

=

e ˆ

gt φπ2

2 (e ˆ

πt 1)2

(5) ˆ Rt

=

ρr ˆ Rt1 + (1 ρr)ψ1 ˆ πt + (1 ρr)ψ2(ˆ yt ˆ gt) + σRεR,t (6) ˆ zt = ρz ˆ zt1 + σzεz,t ˆ gt = ρg ˆ gt1 + σgεg,t ε’s independent N(0, 1)

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 28 / 35

SLIDE 32

AS New Keynesian model

Observable variables: Yt = (Xt, πt, Rt)0 (output, in‡ation and interest rate), where Xt

=

γ(Q) + 100(ˆ yt ˆ yt1 + ˆ zt) πt

=

π(A) + 400 ˆ πt Rt

=

π(A) + r(A) + 4γ(Q) + 400 ˆ Rt. ˆ yt, ˆ Rt, ˆ πt = deviation from steady state Endogenous latent variable: St = b ct = deviation from steady state of consumption Exogenous latent variables: Zt = (b zt, b gt)0 = technology and government spending

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 29 / 35

SLIDE 33

AS model in compact form

(4) implies expression for St as a function of Yt and Zt =

)

substitute into moment conditions Write policy rule as moment conditions Choose instruments to transform Et [] into E [] Write model as E [g(Yt+1, Yt, Zt+1, Zt, θ)] = 0 Zt = ρz ρg

Zt1 + εt, εt iidN
,

σ2

z

σ2

g

g () is 11 1, θ =

(τ, ν, φ, 1/g, ψ1, ψ2, ρR, σR, π(A), γ(Q), r(A), ρz, ρg, σz, σg)

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 30 / 35

SLIDE 34

AS model posterior

Approximate posterior e p(θ, ZT jY T ) ∝ exp

1

2g0

T

Y T , ZT , θ
V 1

T

Y T , ZT , θ
gT
Y
T

∏

t=1

p(ztjzt1, θ)

T

∏

t=1

p(gtjgt1, γ)p(z0, g0jγ) z0 and g0 drawn from their stationary distributions

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 31 / 35

SLIDE 35

Simulation exercise

Same DGP as AS:

Generate a time series (T = 80) from a second order approximation to the model Parameters and priors as in AS

Compare posteriors for θ obtained by our method to those in AS (both linear and nonlinear solution methods)

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 32 / 35

SLIDE 36

AS estimation results

Draws from priors and posteriors for parameters π(A), γ(Q), r(A), ρz, ρg, σz, σg Red lines = true parameter values Estimation time: 100,000 MCMC draws 6 days

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 33 / 35

SLIDE 37

Figure 17: Posterior Draws: Linear versus Quadratic Approximation II

2 4 6 0.2 0.4 0.6 0.8 1 Prior

π(A) γ(Q)

2 4 6 0.2 0.4 0.6 0.8 1 Linear/Kalman Posterior

π(A) γ(Q)

2 4 6 0.2 0.4 0.6 0.8 1 Quadratic/Particle Posterior

π(A) γ(Q)

1 2 1 2 3 4 5 6 7

r(A) π(A)

1 2 1 2 3 4 5 6 7

r(A) π(A)

1 2 1 2 3 4 5 6 7

r(A) π(A)

SLIDE 38

0.5 1 1.5 0.2 0.4 0.6 0.8 1 1.2 1.4

ρg ρz

0.5 1 1.5 0.2 0.4 0.6 0.8 1 1.2 1.4

ρg ρz

0.5 1 1.5 0.2 0.4 0.6 0.8 1 1.2 1.4

ρg ρz

0.005 0.01 1 2 3 4 5 6 x 10

−3

σg σz

0.005 0.01 1 2 3 4 5 6 x 10

−3

σg σz

0.005 0.01 1 2 3 4 5 6 x 10

−3

σg σz

SLIDE 39

Our estimation results

Draws from priors and posteriors for parameters π(A), γ(Q), r(A), ρz, ρg, σz, σg Red lines = true parameter values Estimation time: 2 million MCMC draws 4-5 hours

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 34 / 35

SLIDE 40

1

2 3 4 5 6 7 0.0 0.2 0.4 0.6 0.8 1.0

Priors

π(A) γ(Q)

1.0

1.2 1.4 1.6 1.8 2.0 1 2 3 4 5 6 7

r(A) π(A)

●
●
●
●
●
1

2 3 4 5 6 7 0.0 0.2 0.4 0.6 0.8 1.0

Posteriors

π(A) γ(Q)

1.0

1.2 1.4 1.6 1.8 2.0 1 2 3 4 5 6 7

r(A) π(A)

SLIDE 41

● ●
0.6

0.8 1.0 1.2 1.4 0.2 0.4 0.6 0.8 1.0 1.2 1.4

Priors

ρg ρz

●
0.000

0.002 0.004 0.006 0.008 0.010 0.000 0.001 0.002 0.003 0.004 0.005 0.006

σz σz

●
●
●
●
●
0.6

0.8 1.0 1.2 1.4 0.2 0.4 0.6 0.8 1.0 1.2 1.4

Posteriors

rho_g rho_z

●
●
●
●
●
●
●
0.000

0.002 0.004 0.006 0.008 0.010 0.000 0.001 0.002 0.003 0.004 0.005 0.006

sigma_g sigma_z

SLIDE 42

Conclusion

Two new methods for estimating structural parameters in moment-based models that depend on dynamic latent variables Projection-based approximate likelihoods that satisfy the moment conditions Marries the computational convenience of MCMC in high-dimensional problems with the ability of GMM to handle nonlinear moment conditions Directly delivers "smoothed" latent variables Potential for estimating realistic models and understanding importance of non-linearities

Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss) Moments and latent variables UPenn, 5/4/2010 35 / 35