Backpropagation, Self-Attention, Text Representations Through - - PowerPoint PPT Presentation

▶
backpropagation self attention text representations
SMART_READER_LITE
LIVE PREVIEW

Backpropagation, Self-Attention, Text Representations Through - - PowerPoint PPT Presentation

Feb 10, 2023 442 likes •1.01k views

CS 533: Natural Language Processing Backpropagation, Self-Attention, Text Representations Through Language Modeling Karl Stratos Rutgers University Karl Stratos CS 533: Natural Language Processing 1/52 Dropout (Slide credit Danqi Chen

slide-1
SLIDE 1

CS 533: Natural Language Processing

Backpropagation, Self-Attention, Text Representations Through Language Modeling

Karl Stratos

Rutgers University

Karl Stratos CS 533: Natural Language Processing 1/52

slide-2
SLIDE 2

Dropout

(Slide credit Danqi Chen & Karthik Narasimhan)

Karl Stratos CS 533: Natural Language Processing 2/52

slide-3
SLIDE 3

Unidirectional vs Bidirectional RNN

h x h x

Karl Stratos CS 533: Natural Language Processing 3/52

slide-4
SLIDE 4

Agenda

  • 1. Backpropagation
  • 2. Self-attention in NLP
  • 3. Representation learning through language modeling

Karl Stratos CS 533: Natural Language Processing 4/52

slide-5
SLIDE 5

Backpropagation: Input and Output

◮ A technique to automatically calculate ∇J(θ) for any

definition of scalar-valued loss function J(θ) ∈ R.

Input: loss function J(θ) ∈ R, parameter value ˆ θ Output: ∇J(ˆ θ), the gradient of J(θ) at θ = ˆ θ

◮ Calculates the gradient of an arbitrary differentiable function

  • f parameter θ

Including neural networks

Karl Stratos CS 533: Natural Language Processing 5/52

slide-6
SLIDE 6

Notation

◮ For the most part, we will consider (differentiable) function

f : R → R with a single 1-dimensional parameter x ∈ R.

◮ The gradient of f with respect to x is a function of x

∂f(x) ∂x : R → R

◮ The gradient of f with respect to x evaluated at x = a is

written as ∂f(x) ∂x

  • x=a

∈ R

Karl Stratos CS 533: Natural Language Processing 6/52

slide-7
SLIDE 7

Chain Rule

◮ Given any differentiable functions f, g from R to R,

∂g(f(x)) ∂x = ∂g(f(x)) ∂f(x) × ∂f(x) ∂x easy to calculate

◮ “Proof”: Linearization of linearization of g(z) around f(x)

around a g(f(x)) ≈ g(f(a)) + g′(f(a))f′(a)

∂g(f(x)) ∂x

  • x=a

(x − a)

Karl Stratos CS 533: Natural Language Processing 7/52

slide-8
SLIDE 8

Exercises At x = 42,

◮ What is the value of the gradient of f(x) := 7? ◮ What is the value of the gradient of f(x) := 2x? ◮ What is the value of the gradient of f(x) := 2x + 99999? ◮ What is the value of the gradient of f(x) := x3? ◮ What is the value of the gradient of f(x) := exp(x)? ◮ What is the value of the gradient of f(x) := exp(2x3 + 10)? ◮ What is the value of the gradient of

f(x) := log(exp(2x3 + 10))

Karl Stratos CS 533: Natural Language Processing 8/52

slide-9
SLIDE 9

Chain Rule for a Function of Multiple Input Variables

◮ Let f1 . . . fm denote any differentiable functions from R to R. ◮ If g : Rm → R is a differentiable function from Rm to R,

∂g(f1(x), . . . , fm(x)) ∂x =

m

  • i=1

∂g(f1(x), . . . , fm(x)) ∂fi(x) × ∂fi(x) ∂x easy to calculate

◮ Calculate the gradient of x + x2 + yx with respect to x using

the chain rule.

Karl Stratos CS 533: Natural Language Processing 9/52

slide-10
SLIDE 10

DAG

A directed acylic graph (DAG) is a directed graph G = (V, A) with a topological ordering: a sequence π of V such that for every arc (i, j) ∈ A, i comes before j in π. 1 2 3 4 5 6 For backpropagation: usually assume have many roots and 1 leaf

Karl Stratos CS 533: Natural Language Processing 10/52

slide-11
SLIDE 11

Notation

1 2 3 4 5 6 V = {1, 2, 3, 4, 5, 6} VI = {1, 2} VN = {3, 4, 5, 6} A = {(1, 3), (1, 5), (2, 4), (3, 4), (4, 6), (5, 6)} pa(4) = {2, 3} ch(1) = {3, 5} ΠG = {(1, 2, 3, 4, 5, 6), (2, 1, 3, 4, 5, 6)}

Karl Stratos CS 533: Natural Language Processing 11/52

slide-12
SLIDE 12

Computation Graph

◮ DAG G = (V, E) with a single output node ω ∈ V . ◮ Every node i ∈ V is equipped with a value xi ∈ R:

  • 1. For input node i ∈ VI, we assume xi = ai is given.
  • 2. For non-input node i ∈ VN, we assume a differentiable

function f i : R|pa(i)| → R and compute xi = f i((xj)j∈pa(i))

◮ Thus G represents a function: it receives multiple values

xi = ai for i ∈ VI and calculates a scalar xω ∈ R.

◮ We can calculate xω by a forward pass. Karl Stratos CS 533: Natural Language Processing 12/52

slide-13
SLIDE 13

Forward Pass

Input: computation graph G = (V, A) with output node ω ∈ V Result: populates xi = ai for every i ∈ V

  • 1. Pick some topological ordering π of V .
  • 2. For i in order of π, if i ∈ VN is a non-input node, set

xi ← ai := fi((aj)j∈pa(i)) Why do we need a topological ordering?

Karl Stratos CS 533: Natural Language Processing 13/52

slide-14
SLIDE 14

Exercise

Construct the computation graph associated with the function f(x, y) := (x + y)xy2 Compute its output value at x = 1 and y = 2 by performing a forward pass.

Karl Stratos CS 533: Natural Language Processing 14/52

slide-15
SLIDE 15

For Notational Convenience. . .

◮ Collectively refer to all input slots by xI = (xi)i∈VI. ◮ Collectively refer to all input values by aI = (ai)i∈VI. ◮ At i ∈ V :

Refer to its parental slots by xi

I = (xj)j∈pa(i).

Refer to its parental values by ai

I = (aj)j∈pa(i).

Two equally valid ways of viewing any ai ∈ R as a function:

◮ A “global” function of xI evaluated at aI. ◮ A “local” function of xi I evaluated at ai I.

Karl Stratos CS 533: Natural Language Processing 15/52

slide-16
SLIDE 16

Computation Graph: Gradients

◮ Now for every node i ∈ V , we introduce an additional slot

zi ∈ R defined as

zi := ∂xω ∂xi

  • xI=aI

◮ The goal of backpropagation is to calculate zi for every

i ∈ V .

◮ Why are we done if we achieve this goal? Karl Stratos CS 533: Natural Language Processing 16/52

slide-17
SLIDE 17

Key Ideas of Backpropagation

◮ Chain rule on the DAG structure

zi := ∂xω ∂xi

  • xI=aI

Karl Stratos CS 533: Natural Language Processing 17/52

slide-18
SLIDE 18

Key Ideas of Backpropagation

◮ Chain rule on the DAG structure

zi := ∂xω ∂xi

  • xI=aI

=

  • j∈ch(i)

∂xω ∂xj

  • xI=aI

× ∂xj ∂xi

  • xj

I=aj I Karl Stratos CS 533: Natural Language Processing 17/52

slide-19
SLIDE 19

Key Ideas of Backpropagation

◮ Chain rule on the DAG structure

zi := ∂xω ∂xi

  • xI=aI

=

  • j∈ch(i)

∂xω ∂xj

  • xI=aI

× ∂xj ∂xi

  • xj

I=aj I

=

  • j∈ch(i)zj × ∂fj(xj

I)

∂xi

  • xj

I=aj I

  • easy to calculate

Karl Stratos CS 533: Natural Language Processing 17/52

slide-20
SLIDE 20

Key Ideas of Backpropagation

◮ Chain rule on the DAG structure

zi := ∂xω ∂xi

  • xI=aI

=

  • j∈ch(i)

∂xω ∂xj

  • xI=aI

× ∂xj ∂xi

  • xj

I=aj I

=

  • j∈ch(i)zj × ∂fj(xj

I)

∂xi

  • xj

I=aj I

  • easy to calculate

◮ If we compute zi in a reverse topological ordering, then we

will have already computed zj for all j ∈ ch(i).

Karl Stratos CS 533: Natural Language Processing 17/52

slide-21
SLIDE 21

Key Ideas of Backpropagation

◮ Chain rule on the DAG structure

zi := ∂xω ∂xi

  • xI=aI

=

  • j∈ch(i)

∂xω ∂xj

  • xI=aI

× ∂xj ∂xi

  • xj

I=aj I

=

  • j∈ch(i)zj × ∂fj(xj

I)

∂xi

  • xj

I=aj I

  • easy to calculate

◮ If we compute zi in a reverse topological ordering, then we

will have already computed zj for all j ∈ ch(i).

◮ What’s the base case zω? Karl Stratos CS 533: Natural Language Processing 17/52

slide-22
SLIDE 22

Backpropagation

Input: computation graph G = (V, A) with output node ω ∈ V whose value slots xi = ai are already populated for every i ∈ V Result: populates zi for every i ∈ V

  • 1. Set zω ← 1.
  • 2. Pick some topological ordering π of V .
  • 3. For i in reverse order of π, set

zi ←

  • j∈ch(i)

zj × ∂fj(xj

I)

∂xi

  • xj

I=aj I Karl Stratos CS 533: Natural Language Processing 18/52

slide-23
SLIDE 23

Exercise

Calculate the gradient of f(x, y) := (x + y)xy2 with respect to x at x = 1 and y = 2 by performing

  • backpropagation. That is, calculate the scalar

∂f(x, y) ∂x

  • (x,y)=(1,2)

Karl Stratos CS 533: Natural Language Processing 19/52

slide-24
SLIDE 24

Answer

x y + * * *

1 2 3 3 4 12 1 3 16 16 4 4

Karl Stratos CS 533: Natural Language Processing 20/52

slide-25
SLIDE 25

Implementation

◮ Each type of function f creates a child node from parent

nodes and initializes its gradient to zero.

◮ “Add” function creates a child node c with two parents (a, b)

and sets c.z ← 0.

◮ Each node has an associated forward function.

◮ Calling forward at c populates c.x = a.x + b.x (assumes

parents have their values).

◮ Each node also has an associated backward function.

◮ Calling backward at c “broadcasts” its gradient c.z (assumes

it’s already calculated) to its parents a.z ← a.z + c.z b.z ← b.z + c.z

Karl Stratos CS 533: Natural Language Processing 21/52

slide-26
SLIDE 26

Implementation (Cont.)

◮ Express your loss JB(θ) on minibatch B at θ = ˆ

θ as a computation graph.

◮ Forward pass. For each node a in a topological ordering,

a.forward()

◮ Backward pass. For each node a in a reverse topological

  • rdering,

a.backward()

◮ The gradient of JB(θ) at θ = ˆ

θ is stored in the input nodes of the computation graph.

Karl Stratos CS 533: Natural Language Processing 22/52

slide-27
SLIDE 27

General Backpropagation

◮ Computation graph in which input values that are vectors

xi ∈ Rdi ∀i ∈ V But the output value xω ∈ R is always a scalar!

◮ The corresponding gradients are also vectors of the same size

zi ∈ Rdi ∀i ∈ V

◮ Backpropagation has exactly the same structure using the

generalized chain rule zi =

  • j∈ch(i)

∂xω ∂xj

  • xI=aI

1×dj

× ∂xj ∂xi

  • xj

I=aj I

dj×di

where second term is Jacobian of fj wrt xi evaluated at aI

Karl Stratos CS 533: Natural Language Processing 23/52

slide-28
SLIDE 28

Agenda

  • 1. Backpropagation
  • 2. Self-attention in NLP
  • 3. Representation learning through language modeling

Karl Stratos CS 533: Natural Language Processing 24/52

slide-29
SLIDE 29

Recurrent vs Self-Attention

h x h x

∂h ∂x

Karl Stratos CS 533: Natural Language Processing 25/52

slide-30
SLIDE 30

Attention: General Form

Input

◮ Q ∈ Rd×T : T query vectors of the “asker” ◮ K ∈ Rd×T ′: T ′ key vectors of the “answerer” ◮ V ∈ Rd×T ′: T ′ value vectors of the “answerer”

Output

◮ A ∈ Rd×T : T answer vectors of the “asker” after asking

A = V softmax

  • K⊤Q
  • Karl Stratos

CS 533: Natural Language Processing 26/52

slide-31
SLIDE 31

Example: Attention-Based Seq2Seq

Input

◮ Q = Y : target LSTM encodings ◮ K = X: source LSTM encodings ◮ V = X: source LSTM encodings

Output

◮ A = Attention(Y, X, X): new target encodings

A = Xsoftmax

  • X⊤Y
  • Karl Stratos

CS 533: Natural Language Processing 27/52

slide-32
SLIDE 32

Scaled Attention

Useful when d is large

A = V softmax K⊤Q √ d

  • Exercise: k, q ∈ Rd elementwise independent, mean 0, variance 1

◮ var

  • k⊤q
  • ?

◮ var

  • k⊤q/

√ d

  • ?

Karl Stratos CS 533: Natural Language Processing 28/52

slide-33
SLIDE 33

Multi-Head Attention

Same input Parameters

◮ W Q i

∈ R(d/H)×d for i = 1 . . . H: query projectors

◮ W K i

∈ R(d/H)×d for i = 1 . . . H: key projectors

◮ W V i

∈ R(d/H)×d for i = 1 . . . H: value projectors

◮ W ∈ Rd×d

A = W     Attention

  • W Q

1 Q, W K 1 K, W V 1 V

  • .

. . Attention

  • W Q

HQ, W K H K, W V HV

  

Karl Stratos CS 533: Natural Language Processing 29/52

slide-34
SLIDE 34

Multi-Head Attention with Residual (or Skip) Connection

Plus regularization: dropout, layer normalization (Ba et al., 2016)

A = MultiHeadAttention(Q, K, V ) A′ = LayerNorm (Drop (A) + Q)

Henceforth ResMHA(Q, K, V )

Karl Stratos CS 533: Natural Language Processing 30/52

slide-35
SLIDE 35

Transformer Encoder (Vaswani et al., 2017)

Using H = 8 heads, l = 0 . . . 5

  • X(l) = ResMHA
  • X(l), X(l), X(l)

X(l+1) = ResFF

  • X(l)

X(0) = Drop0.1(E + Π)

Karl Stratos CS 533: Natural Language Processing 31/52

slide-36
SLIDE 36

Transformer Decoder (Vaswani et al., 2017)

Using H = 8 heads, l = 0 . . . 5

  • Y (l) = ResMHA
  • Y (l), Y (l), Y (l)

Y (l) = ResMHA

  • Y (l), X(6), X(6)

Y (l+1) = ResFF

  • Y (l)

Prediction: softmax

  • EY (6)

Karl Stratos CS 533: Natural Language Processing 32/52

slide-37
SLIDE 37

Translation Performance (Vaswani et al., 2017)

Karl Stratos CS 533: Natural Language Processing 33/52

slide-38
SLIDE 38

Self-Attention Visualization (Vaswani et al., 2017)

Layer 5 and 6, one of the “heads” Different heads learn different weights

Karl Stratos CS 533: Natural Language Processing 34/52

slide-39
SLIDE 39

Agenda

  • 1. Backpropagation
  • 2. Self-attention in NLP
  • 3. Representation learning through language modeling

Karl Stratos CS 533: Natural Language Processing 35/52

slide-40
SLIDE 40

Text Representations Through Neural Language Modeling

  • 1. Language models can be trained on a lot of text (e.g., the web)
  • 2. They yield text representations generally useful for downstream tasks

Karl Stratos CS 533: Natural Language Processing 36/52

slide-41
SLIDE 41

Example Downstream Tasks

Sentence classification

◮ Binary sentiment classification

This film doesn’t care about intelligent humor →

0.05 0.95

  • r multi-class (e.g., 5 stars)

◮ Example datasets: SST-2, IMBb, Yelp Review, SemEval,

CoLA

◮ Sentiment analysis results: http:

//nlpprogress.com/english/sentiment_analysis.html

◮ Other types of classification: grammatical vs ungrammatical

(CoLA)

Karl Stratos CS 533: Natural Language Processing 37/52

slide-42
SLIDE 42

Example Downstream Tasks

Sentence pair classification, or natural language inference (NLI) (I am a lacto-vegetarian, I enjoy eating cheese) →   0.05 0.03 0.92   Example dataset: MNLI (Williams et al., 2018)

Karl Stratos CS 533: Natural Language Processing 38/52

slide-43
SLIDE 43

Example Downstream Tasks

SQuAD-style question answering (Rajpurkar et al., 2016) Example dataset: SQuAD (Rajpurkar et al., 2016) Can be framed as predicting start/end index of the passage

Karl Stratos CS 533: Natural Language Processing 39/52

slide-44
SLIDE 44

Setting

◮ Each such downstream task provides only a limited amount of

labeled data

◮ Can we transfer a large-scale pretrained language model to

improve performance in all these tasks simultaneously?

◮ Popular benchmarks (Wang et al, 2018):

◮ GLUE: https://gluebenchmark.com/leaderboard ◮ SuperGLUE:

https://super.gluebenchmark.com/leaderboard

Karl Stratos CS 533: Natural Language Processing 40/52

slide-45
SLIDE 45

ELMo (Peters et al., 2018)

Trained 10 epochs on 1B Word Benchmark https://arxiv.org/pdf/1802.05365.pdf

Karl Stratos CS 533: Natural Language Processing 41/52

slide-46
SLIDE 46

ELMo (Peters et al., 2018)

Karl Stratos CS 533: Natural Language Processing 42/52

slide-47
SLIDE 47

ELMo in Practice

  • 1. ELMo layer: new representation of i-th token in a sequence

ELMoi(γ, s1 . . . sL) = γ

L

  • l=0

sl      eELMo

i

if l = 0 − → h ELMo,l

i

← − h ELMo,l

i

  • therwise
  • 2. In your downstream task, concatenate ELMoi(γ, s1 . . . sL) to

your i-th input embedding.

  • 3. Train your original model AND γ, s1 . . . sL while keeping

ELMo parameters fixed

Karl Stratos CS 533: Natural Language Processing 43/52

slide-48
SLIDE 48

Using ELMo

Karl Stratos CS 533: Natural Language Processing 44/52

slide-49
SLIDE 49

Recurrent vs Self-Attention Encoding

h x x h

not bidirectional until later deeply bidirectional

Karl Stratos CS 533: Natural Language Processing 45/52

slide-50
SLIDE 50

Masked Language Modeling

◮ For the purposes of representation learning, we don’t care

about defining a proper language model which only conditions

  • n previous history.

◮ We want a prediction problem which conditions on entire

context all the time, so that we can use deeply bidirectional encoders

◮ Solution: mask out words at random

the man went to the [MASK] to buy a [MASK] of milk

◮ Need to be careful

◮ Too little masking: too expensive to train ◮ Too much masking: not enough context ◮ Test time: no [MASK] input, so training should also handle no

[MASK] input sometimes

◮ Details: https://arxiv.org/pdf/1810.04805.pdf Karl Stratos CS 533: Natural Language Processing 46/52

slide-51
SLIDE 51

BERT (Devlin et al., 2019)

[CLS] the dog [MASK] [SEP] the cat [MASK] away [SEP]

IsNext barked ran

Transformer

(Vaswani et al., 2017)

Karl Stratos CS 533: Natural Language Processing 47/52

slide-52
SLIDE 52

BERT (Devlin et al., 2019)

Number of parameters

◮ ELMo: 94 million ◮ BERT Base: 110 million ◮ BERT Large: 340 million

Karl Stratos CS 533: Natural Language Processing 48/52

slide-53
SLIDE 53

RoBERTa (Liu et al., 2019)

RoBERTa = BERT + more careful training + more data https://github.com/pytorch/fairseq/tree/master/ examples/roberta

Karl Stratos CS 533: Natural Language Processing 49/52

slide-54
SLIDE 54

BERT Manual

Critical difference from ELMo: all BERT weights are fine-tuned for the target task (expensive but worth it)

Karl Stratos CS 533: Natural Language Processing 50/52

slide-55
SLIDE 55

BERT Applications

Karl Stratos CS 533: Natural Language Processing 51/52

slide-56
SLIDE 56

Currently in NLP

Explosion of pretrained contextualized word embedding models

◮ TagLM (Peters et, 2017) ◮ CoVe (McCann et al. 2017) ◮ ULMfit (Howard and Ruder, 2018) ◮ ELMo (Peters et al, 2018) ◮ OpenAI GPT (Radford et al, 2018) ◮ BERT (Devlin et al, 2018) ◮ OpenAI GPT-2 (Radford et al, 2019) ◮ XLNet (Yang et al, 2019) ◮ SpanBERT (Joshi et al, 2019) ◮ RoBERTa (Liu et al, 2019) ◮ AlBERT (Anonymous) ◮ T5 (Raffel et al., 2019) ◮ . . .

Karl Stratos CS 533: Natural Language Processing 52/52