2020-1 UROP 2020.03.12 2016-12146 Seri Lee Goal Multi-Behavior - - PowerPoint PPT Presentation

2020-1 UROP 2020.03.12

2016-12146 Seri Lee

Multi-Behavior Recommendation

“Given user behavior data of multiple types, predict users’ next behaviors of target type.”

Goal

Approach

• Implement an RNN-based recommendation algorithm for

a single behavior type.

• Extend the algorithm to further utilize other types of

behaviors by using attention mechanisms.

Previous work (1)

Learning recommender systems from multi- behavior data

Limitations

• NMTR cannot capture sequential patterns since it does

not consider the time sequence of behaviors.

• New algorithm should capture sequential patterns by

using Recurrent Neural Network.

Previous work (2)

ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation

Attention-Based Heterogeneous Behaviors Modeling Framework

Raw Feature Spaces

U = {(aj, oj, tj)|j = 1,2,…, m} G = {bg1, bg2, …, bgn} bgi ∩ bgj = ∅ U = ∪n

i=1 bgi

behavior

• bject

timestamp behavior groups according to target object types group-specific neural nets to build up behavior embedding

Attention-Based Heterogeneous Behaviors Modeling Framework

Behavior Embedding Spaces

uij = fi(aj, oj, tj) uij = embi(oj) + lookupt

i(bucketizei(tj)) + lookupa i (aj)

B = {ubg1, ubg2, …, ubgn}

embedding building block

• utput: list of vectors in all behavior groups

Attention-Based Heterogeneous Behaviors Modeling Framework

Latent Semantic Spaces

S = concat(0)(FM1(ubg1), FM2(ubg2), …, FMn(ubgn)) Sk = FPk(S)

to fix-length encoding vectors projection function (put them into same semantic space)

Sall

• verall space of dimension size

projected behavior embedding in each spaces projection function (single layer perceptron, ReLu activation function)

Attention-Based Heterogeneous Behaviors Modeling Framework

Self-Attention Layer

Ak = softmax(a(Sk, S; θ)) a(Sk, S; θk) = SkWkST Ck = AkFQk(S) C = 𝔊self(concat(1)(C1, C2, …, CK))

goal: capture the inner-relationships among each semantic space self-attention concatenated & reorganized feedforward network with one hidden layer score vector score function <bilinear scoring function> attention vectors of space k projection function: single layer perceptron+ReLU

Attention-Based Heterogeneous Behaviors Modeling Framework

Downstream Application Network

⃗ ht = FMg(t)( ⃗ qt) ⃗ sk = FPk( ⃗ ht) ⃗ ck = softmax(a( ⃗ sk, C; θk))FQk(C) ⃗ et

u = Fvanilla(concat(1)((

⃗ c1, ⃗ c2, …, ⃗ cK ))) −∑

t,u

ytlog(σ(f(ht, et

u))) + (1 − yt)log(1 − σ(f(ht, et u)))

: point-wise / pair-wise fully connected nn final context vector final loss function: sigmoid cross entropy loss vanilla attention ranking function

Future Work

• Implement ATRank with the given Dataset.

Dataset

• https://www.kaggle.com/mkechinov/ecommerce-

behavior-data-from-multi-category-store/data#

• eCommerce behavior data from multi category store
• behavior: view, cart, remove_from_cart, purchase
• object behavior: purchase