SLIDE 1
Problem
- How do we define long-range dependence?
A Statistical Investigation of Long Memory in Language and Music - - PowerPoint PPT Presentation
A Statistical Investigation of Long Memory in Language and Music - - PowerPoint PPT Presentation
A Statistical Investigation of Long Memory in Language and Music Alexander Greaves-Tunnell and Zaid Harchaoui ICML 2019 Problem How do we define long-range dependence ? Problem How do we define long-range dependence ? How can it be
SLIDE 2
SLIDE 3
Problem
- How do we define long-range dependence?
- How can it be estimated in modern sources of sequence data?
SLIDE 4
Problem
- How do we define long-range dependence?
- How can it be estimated in modern sources of sequence data?
- How can we evaluate if a model has captured this property?
SLIDE 5
Contributions
- Introduce a framework for evaluation of long-range dependence
anchored in the literature of long memory stochastic processes.
SLIDE 6
Contributions
- Introduce a framework for evaluation of long-range dependence
anchored in the literature of long memory stochastic processes.
50 100 150 200 250
k
0.0 0.2 0.4 0.6 0.8 1.0
γ(k) (solid)
AR(1) FI(d)-AR(1)
SLIDE 7
Contributions
- Introduce a framework for evaluation of long-range dependence
anchored in the literature of long memory stochastic processes.
50 100 150 200 250
k
0.0 0.2 0.4 0.6 0.8 1.0
γ(k) (solid)
AR(1) FI(d)-AR(1)
10 20 30 40 50
|γ(k)| (dashed)
SLIDE 8
Contributions
- Introduce a framework for evaluation of long-range dependence
anchored in the literature of long memory stochastic processes.
50 100 150 200 250
k
0.0 0.2 0.4 0.6 0.8 1.0
γ(k) (solid)
AR(1) FI(d)-AR(1)
10 20 30 40 50
|γ(k)| (dashed)
- Adapt semiparametric statistical methods to define estimation
and testing procedure for long memory in high dimensions.
SLIDE 9
Results: Language and Music
Do language and music data have long memory?
SLIDE 10
Results: Language and Music
Do language and music data have long memory? Visual heuristic from differing behavior of partial sums:
K
- k=1
|γ(k)| →
- ∞
long memory c < ∞ short memory , as K → ∞
SLIDE 11
Results: Language and Music
Do language and music data have long memory? Visual heuristic from differing behavior of partial sums:
K
- k=1
|γ(k)| →
- ∞
long memory c < ∞ short memory , as K → ∞
50 100 150 200 250
k
1.0 1.5 2.0 2.5 3.0 3.5 4.0
|γ(k)| Natural language
Penn TreeBank Bible Facebook bAbI CBT
50 100 150 200 250
k
10 20 30 40 50 60
|γ(k)| Music
Miles Davis Oum Kalthoum J.S. Bach
SLIDE 12
Results: RNN Models
Hypothesis test for long memory: H0 : d = 0 vs.
anticipated result
- HA : d > 0 .
SLIDE 13
Results: RNN Models
Hypothesis test for long memory:
experimental result
- H0 : d = 0
- vs. HA : d > 0.
Model memory d p-value Reject H0? LSTM (trained) −8.59 × 10−4 0.583 X LSTM (untrained) −4.17 × 10−4 0.572 X Memory cell −5.96 × 10−4 0.552 X SCRN 2.37 × 10−3 0.324 X
SLIDE 14
Results: RNN Models
Hypothesis test for long memory:
experimental result
- H0 : d = 0
- vs. HA : d > 0.