A Recurrent Neural Cascade-based Model for Continuous-Time Diffusion - - PowerPoint PPT Presentation

A Recurrent Neural Cascade-based Model for Continuous-Time Diffusion

Sylvain Lamprier LIP6 - Sorbonne Universit´ es

1 / 5

Cascade-based models for diffusion

Information spreads from users to users in the network, following independent transmission probabilities

0.4 0.6 1 0.2 0.6 0.1 0.7 0.3

Observed Diffusion Episode = {(A;1);(B;2);(C;2);(D;3);(F;4)} A B C E D F

The Continuous-Time Independent Cascade Model (CTIC) defines two parameters ku,v and ru,v per pair (u, v) of nodes in the network (Saito et al., 2011) : ku,v : probability that u succeeds in infecting v ; ru,v : time-delay parameter from u to v Likelihood of a set of episodes D : P(D) =

• D∈D
• v∈UD

P(v infected at time tD

v )

• v∈UD

P(v not infected in D)

2 / 5

RNN models for diffusion

Markovian assumption does not hold in many situations :

C A B E D C A B E D

T ype 1 T ype 2 High proba for D if A is infected High proba for E if B is infected

⇒ Episode D as a sequence ((UD

1 , tD 1 ), (UD 2 , tD 2 ), ..., (UD |D|, tD |D|))

Recurrent Marked Temporal Point Processes (Du et al, 2016) :

hidden h1 hidden h2 hidden h|D|

time

hidden h0 (U1,t1 )

D

(U2,t2 ) (U|D|,t|D| )

D D D D D

P(U1|h0) P(t1|h0)

D D

P(U2|h1) P(t2-t1|h1)

D D D

P(U3|h2) P(t3-t2|h2)

D D D

P(stop|h|D|)

... But diffusion is not a sequence !

3 / 5

RNN models for diffusion

time

C A F D B E

t1

D

t2

D t3 D

t4

D

t5

D

t6

D

F does not depend on E

Tree Dependencies ⇒ Cyan (Wang et al., 2017b) : RNN with attention to select previous states ⇒ DAN (Wang et al., 2018) : Similar to Cyan, but with a pooling mechanism rather than RNN

4 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion

⇒ Idea : Assign a continuous state zD

v ∈ Rd to each infected

node v, which depends on its infection path

zD

v then conditions distributions of subsequent infections from v

P(u infects v)= σ

• < zD

u , ω(k) v

>

• , with ω(k)

v

∈ Rd a continuous representation of v If u is the first node to infect v : zD

v = fφ(zD u , ω(f ) v )

with :

fφ a GRU cell zD

u the state of u for D (the memory)

ω(f )

v

∈ Rd a static representation for v (the input)

5 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion

⇒ Idea : Assign a continuous state zD

v ∈ Rd to each infected

node v, which depends on its infection path

zD

v then conditions distributions of subsequent infections from v

A C B D E F G I J H

1 2 3 4 5 7

L M O

6 9 8 10

K

K L M O

P(infection from J) 5 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion

⇒ Idea : Assign a continuous state zD

v ∈ Rd to each infected

node v, which depends on its infection path

zD

v then conditions distributions of subsequent infections from v

A C B D E F G I J H

1 2 3 4 5 7

L M O

6 9 8 10

K

K L M O

P(infection from J) 5 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion

⇒ Idea : Assign a continuous state zD

v ∈ Rd to each infected

node v, which depends on its infection path

zD

v then conditions distributions of subsequent infections from v

A C B D E F G I J H

1 2 3 4 5 7

L M O

6 9 8 10

K

K L M O

P(infection from J) 5 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion

⇒ Idea : Assign a continuous state zD

v ∈ Rd to each infected

node v, which depends on its infection path

zD

v then conditions distributions of subsequent infections from v

A C B D E F G I J H

1 2 3 4 5 7

L M O

6 9 8 10

K

K L M O

P(infection from J) 5 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion

⇒ Idea : Assign a continuous state zD

v ∈ Rd to each infected

node v, which depends on its infection path

zD

v then conditions distributions of subsequent infections from v

Inference on ancestors sequences I is required : log p(D) = log

• I∈ID

p(D, I) Inference distribution : qD(I) =

|D|−1

• i=1

p(Ii|D≤i, I<i) ≈ p(I|D) Score function estimator : ∇ΘL(D; Θ) =

• D∈D

EI∼qD     

• log pI(D) − b
• ∇Θ log qD(I)
• favors good paths

+ ∇Θ log pI(D)

• increases likelihood

given the path

    

5 / 5