Classical Copying versus Qantum Entanglement in Natural Language: - - PowerPoint PPT Presentation

▶
classical copying versus qantum entanglement in natural
SMART_READER_LITE
LIVE PREVIEW

Classical Copying versus Qantum Entanglement in Natural Language: - - PowerPoint PPT Presentation

Jan 07, 2024 12 likes •453 views

Classical Copying versus Qantum Entanglement in Natural Language: the Case of VP-ellipsis Gijs Jasper Wijnholds 1 Mehrnoosh Sadrzadeh 1 Qeen Mary University of London, United Kingdom g.j.wijnholds@qmul.ac.uk SYCO 2 December 17, 2018

slide-1
SLIDE 1

Classical Copying versus Qantum Entanglement in Natural Language: the Case of VP-ellipsis

Gijs Jasper Wijnholds1 Mehrnoosh Sadrzadeh1

Qeen Mary University of London, United Kingdom g.j.wijnholds@qmul.ac.uk

SYCO 2 December 17, 2018

slide-2
SLIDE 2

DISTRIBUTIONAL SEMANTICS: MEANING IN CONTEXT

  • G. J. Wijnholds

SYCO 2 2 / 41

slide-3
SLIDE 3

COMPOSING WORD EMBEDDINGS: A CHALLENGE

Coordination − − − − − − − − − − − − − − − → dancing and running = ?? Qantification − − − − − − − − − − − − − − − − − − − − − − − − → Every student likes some teacher = ?? Anaphora − − − − − − − − − − → shaves himself = ?? ⇒ Ellipsis − − − − − − − − − − − − − − − − − − − − − − − − − − − − → Mat went to Croatia and Max did too = ??

  • G. J. Wijnholds

SYCO 2 3 / 41

slide-4
SLIDE 4

VERB PHRASE ELLIPSIS

◮ Ellipsis is a natural language phenomenon in which part of a phrase is

missing and has to be recovered from context.

◮ In verb phrase ellipsis, the missing part is… a verb phrase. ◮ There is ofen a marker that indicates the type of the missing part.

  • G. J. Wijnholds

SYCO 2 4 / 41

slide-5
SLIDE 5

VERB PHRASE ELLIPSIS

◮ Ellipsis is a natural language phenomenon in which part of a phrase is

missing and has to be recovered from context.

◮ In verb phrase ellipsis, the missing part is… a verb phrase. ◮ There is ofen a marker that indicates the type of the missing part.

Bob drinks a beer

  • ant VP

and Alice does too

  • marker
  • G. J. Wijnholds

SYCO 2 4 / 41

slide-6
SLIDE 6

ELLIPSIS NEEDS COPYING AND MOVEMENT ant VP marker Bob drinks a beer drinks a beer drinks a beer and Alice does too

  • G. J. Wijnholds

SYCO 2 5 / 41

slide-7
SLIDE 7

ELLIPSIS NEEDS COPYING AND MOVEMENT ant VP marker Bob drinks a beer drinks a beer drinks a beer and Alice does too

  • G. J. Wijnholds

SYCO 2 6 / 41

slide-8
SLIDE 8

ELLIPSIS NEEDS COPYING AND MOVEMENT ant VP marker Bob drinks a beer drinks a beer drinks a beer and Alice

✘✘✘✘ ✘

does too

  • G. J. Wijnholds

SYCO 2 7 / 41

slide-9
SLIDE 9

THE CHALLENGE: COMPOSE WORD VECTORS TO GET A MEANING REPRESENTATION FOR VP ELLIPSIS

  • G. J. Wijnholds

SYCO 2 8 / 41

slide-10
SLIDE 10

THE BIG PICTURE Qantum Entanglement SOURCE L♦,F TARGET FVect Functor Classical SOURCE L♦,F INTER λNL TARGET λFVecFrob Hder Hlex

  • G. J. Wijnholds

SYCO 2 9 / 41

slide-11
SLIDE 11

QUANTUM ENTANGLEMENT

  • G. J. Wijnholds

SYCO 2 10 / 41

slide-12
SLIDE 12

LAMBEK VS. LAMBEK

The core of the Lambek calculus: application, co-application

B ⊗ B\A → A A → B\(B ⊗ A) A/B ⊗ B → A A → (A ⊗ B)/B (A ⊗ B) ⊗ C ↔ A ⊗ (B ⊗ C)

Interpretation: words have types, and type-respecting embeddings

Word Type embedding john np − − → john ∈ N sleeps np\s sleep ∈ N ⊗ S (← matrix) likes (np\s)/np like ∈ N ⊗ S ⊗ N (← cube) beer np − − → beer ∈ N

(Coecke et al., 2013)

  • G. J. Wijnholds

SYCO 2 11 / 41

slide-13
SLIDE 13

IN PICTURES A ⊗ A\B → B A A B B → A\(A ⊗ B) A A B B/A ⊗ A → B A A B B → (B ⊗ A)/A A A B

(Coecke et al., 2013)

  • G. J. Wijnholds

SYCO 2 12 / 41

slide-14
SLIDE 14

IN PICTURES A ⊗ A\B → B A A B B → A\(A ⊗ B) A A B B/A ⊗ A → B A A B B → (B ⊗ A)/A A A B LINEAR‼

(Coecke et al., 2013)

  • G. J. Wijnholds

SYCO 2 13 / 41

slide-15
SLIDE 15

LAMBEK WITH CONTROL OPERATORS: L♦,F

The core of the Lambek calculus: application, co-application

B ⊗ B\A → A A → B\(B ⊗ A) A/B ⊗ B → A A → (A ⊗ B)/B (A ⊗ B) ⊗ C ↔ A ⊗ (B ⊗ C)

Modalities: application, co-application

♦A → A A → ♦A

◮ Linear logic: controlled duplication/deletion of resources via

! = ♦. Here: controlled copying, reordering

Controlled contraction, commutativity

A → ♦A ⊗ A (♦A ⊗ B) ⊗ C → B ⊗ (♦A ⊗ C) ♦A ⊗ (♦B ⊗ C) → ♦B ⊗ (♦A ⊗ C)

  • G. J. Wijnholds

SYCO 2 14 / 41

slide-16
SLIDE 16

ILLUSTRATION Bob ant VP drinks a beer and Alice marker does too np np\s ♦(np\s) np\s (s\s)/s np ♦(np\s)\(np\s)

  • G. J. Wijnholds

SYCO 2 15 / 41

slide-17
SLIDE 17

IN PICTURES A ⊗ A\B → B A A B B → A\(A ⊗ B) A A B B/A ⊗ A → B A A B B → (B ⊗ A)/A A A B A → ♦A ⊗ A A

  • A

A

(♦A ⊗ B) ⊗ C → B ⊗ (♦A ⊗ C)

A B C B A C

  • G. J. Wijnholds

SYCO 2 16 / 41

slide-18
SLIDE 18

QUANTUM ENTANGLEMENT AND ELLIPSIS

Bob N · · · · · drinks a beer N ⊗ S

  • and

S ⊗ S ⊗ S · · · Alice N · does too N ⊗ S ⊗ S ⊗ N · · · ·

  • G. J. Wijnholds

SYCO 2 17 / 41

slide-19
SLIDE 19

QUANTUM ENTANGLEMENT AND ELLIPSIS

Bob N · · · · · drinks a beer N ⊗ S

  • and

S ⊗ S ⊗ S · · · Alice N · does too N ⊗ S ⊗ S ⊗ N · · · · ·

  • ·
  • G. J. Wijnholds

SYCO 2 18 / 41

slide-20
SLIDE 20

QUANTUM ENTANGLEMENT AND ELLIPSIS

Bob N · · · drinks a beer N ⊗ S (− → Bob ⊙ − − → Alice)⊤ × − − − − − − − − → drinks a beer

  • Alice

N ·

  • G. J. Wijnholds

SYCO 2 19 / 41

slide-21
SLIDE 21

A MORE COMPLICATED CASE: SLOPPY READING Bob loves his beer and Alice does too Bob loves Bob’s beer and Alice loves Bob’s beer

Bob ant VP loves (np\s)/np his ♦np\(np/n) beer n and Alice marker does too np ♦np np ♦(np\s) np\s np\s (s\s)/s np ♦(np\s)\(np\s)

  • G. J. Wijnholds

SYCO 2 20 / 41

slide-22
SLIDE 22

A MORE COMPLICATED CASE: SLOPPY READING Bob loves his beer and Alice does too Bob loves Bob’s beer and Alice loves Bob’s beer

Bob ant VP loves (np\s)/np his ♦np\(np/n) beer n and Alice marker does too np ♦np np ♦(np\s) np\s np\s (s\s)/s np ♦(np\s)\(np\s)

  • G. J. Wijnholds

SYCO 2 21 / 41

slide-23
SLIDE 23

A MORE COMPLICATED CASE: STRICT READING Bob loves his beer and Alice does too Bob loves Bob’s beer and Alice loves Alice’s beer

Bob ant VP loves (np\s)/np his ♦np\(np/n) beer n and Alice marker does too np ♦np np ♦(np\s) np\s np\s (s\s)/s np ♦(np\s)\(np\s)

  • G. J. Wijnholds

SYCO 2 22 / 41

slide-24
SLIDE 24

A MORE COMPLICATED CASE: STRICT READING Bob loves his beer and Alice does too Bob loves Bob’s beer and Alice loves Alice’s beer

Bob ant VP loves (np\s)/np his ♦np\(np/n) beer n and Alice marker does too np ♦np np ♦

  • (np\s)/s ⊗ ( ♦np\n ⊗ n)
  • (s\s)/s

np ♦(np\s)\(np\s)

  • G. J. Wijnholds

SYCO 2 23 / 41

slide-25
SLIDE 25

A More Complicated Case: Sloppy Reading

Bob N

  • ·

· · · · · loves N ⊗ S ⊗ N

  • ◦

his N ⊗ N ⊗ N · beer N and S ⊗ S ⊗ S · · · Alice N · does too N ⊗ S ⊗ S ⊗ N · · · · · ·

  • G. J. Wijnholds

SYCO 2 24 / 41

slide-26
SLIDE 26

A More Complicated Case: Sloppy Reading

Bob N

  • ·

· · · · · loves N ⊗ S ⊗ N

  • ◦

his N ⊗ N ⊗ N · beer N and S ⊗ S ⊗ S · · · Alice N · does too N ⊗ S ⊗ S ⊗ N · · · · · · ·

  • ·

·

  • ·
  • G. J. Wijnholds

SYCO 2 25 / 41

slide-27
SLIDE 27

A More Complicated Case: Sloppy Reading Bob loves Bob’s beer and Alice loves Bob’s beer

Bob N Alice N

  • beer

N · · · · loves N ⊗ S ⊗ N ∆(Bob ⊙ Alice ⊙ Beer)iklovesijk

  • G. J. Wijnholds

SYCO 2 26 / 41

slide-28
SLIDE 28

A More Complicated Case: Strict Reading Bob loves Bob’s beer and Alice loves Alice’s beer

  • G. J. Wijnholds

SYCO 2 27 / 41

slide-29
SLIDE 29

A More Complicated Case: Strict Reading Bob loves Bob’s beer and Alice loves Alice’s beer

Bob N Alice N

  • beer

N · · · · loves N ⊗ S ⊗ N ∆(Bob ⊙ Alice ⊙ Beer)iklovesijk

  • G. J. Wijnholds

SYCO 2 27 / 41

slide-30
SLIDE 30

WHAT NOW?

  • G. J. Wijnholds

SYCO 2 28 / 41

slide-31
SLIDE 31

WHAT NOW? Classical Semantics

  • G. J. Wijnholds

SYCO 2 28 / 41

slide-32
SLIDE 32

General Interpretation

The syntax-semantics homomorphism interprets types and proofs of L♦,F as objects types and maps terms in a compact closed category non-linear lambda calculus:

Type Level

⌊A ⊗ B⌋ = ⌊A⌋ × ⌊B⌋ ⌊A/B⌋ = ⌊A⌋ → ⌊B⌋ ⌊A\B⌋ = ⌊A⌋ → ⌊B⌋ ⌊♦A⌋ = ⌊A⌋ = ⌊A⌋

Application, co-application

B × (B → A) λxM.M x − − − − − − − → A λx.λy.y, x − − − − − − − − − → B → (B × A) (B → A) × B λMx.Mx − − − − − − → A λx.λy.x, y − − − − − − − − − → B → (A × B)

Modalities

♦, are semantically vacuous, so only the control rules get a non-trivial interpretation: A λx.x, x − − − − − − − → A × A (A × B) × C λx, y, z.y, x, z − − − − − − − − − − − − − → B × (A × C)

  • G. J. Wijnholds

SYCO 2 29 / 41

slide-33
SLIDE 33

Lambda term for simple ellipsis Bob drinks and Alice does too

A proof of (np ⊗ np\s) ⊗ ((s\s)/s ⊗ (np ⊗ (♦(np\s) ⊗ ♦(np\s)\(np\s)))) − → s gives term λsubj1, verb, coord, subj2, verb∗, aux.(coord ((aux verb∗) subj2))(verb subj1) The movement and contraction give λsubj1, verb, coord, subj2, aux.(coord ((aux verb) subj2))(verb subj1) Plugging in some constants, we get an abstract term (and ((dt drinks) bob))(drinks alice) : s

  • G. J. Wijnholds

SYCO 2 30 / 41

slide-34
SLIDE 34

Modelling Vectors with Lambdas Vector: λi.vi I → R (= V) Matrix: λij.Mij I → I → R ⊙ : λvui.vi · ui V → V → R Vector ⊙ Vector: λv.v ⊙ v V → V Matrix⊤ λmij.mji M → M Matrix ×1 Vector λmvi.

j

mij · vj M → V → V Cube ×2 Vector λcvij.

k

cijk · vk C → V → M (Muskens & Sadrzadeh, 2016)

  • G. J. Wijnholds

SYCO 2 31 / 41

slide-35
SLIDE 35

Classical Semantics for simple ellipsis Bob drinks and Alice does too

(λP.λQ.P ⊙ Q ((λx.x (λv.drinks ×1 v)) bob))((λv.(drinks ×1 v)) alice) →β (λP.λQ.P ⊙ Q ((λv.drinks ×1 v) bob))((λv.(drinks ×1 v)) alice) →β (λP.λQ.P ⊙ Q (drinks ×1 bob))(drinks ×1 alice) →β (drinks ×1 bob) ⊙ (drinks ×1 alice)

  • G. J. Wijnholds

SYCO 2 32 / 41

slide-36
SLIDE 36

Classical Semantics for Ambiguous Ellipsis Bob loves Bob’s beer and Alice loves Bob’s beer (sloppy) (bob×1 loves×2 (bob⊙beer))⊙(alice×1 loves×2 (bob⊙beer)) Bob loves Bob’s beer and Alice loves Alice’s beer (strict) (bob×1 loves×2 (bob⊙beer))⊙(alice×1 loves×2 (alice⊙beer))

  • G. J. Wijnholds

SYCO 2 33 / 41

slide-37
SLIDE 37

Conclusion 1: Classical vs. Qantum Entanglement Developing Frobenius Semantics fits easily in the DisCoCat framework, but fails to give a proper account for more complex examples of ellipsis. Classical Semantics are more involved and are non-linear, but give a beter account of derivational ambiguity.

  • G. J. Wijnholds

SYCO 2 34 / 41

slide-38
SLIDE 38

LET THE DATA SPEAK

  • G. J. Wijnholds

SYCO 2 35 / 41

slide-39
SLIDE 39

EXPERIMENTING WITH VP ELLIPSIS

◮ GS2011 verb disambiguation dataset (200 samples):

man draw photograph ∼ man atract photograph man draw photograph ∼ man depict photograph

◮ KS2013 similarity dataset (108 samples):

man bites dog ∼ student achieve result

◮ We extended the above datasets to elliptical phrases (now with 400/416 sentence pairs)

man bites dog and woman does too ∼ student achieve result and boy does too

◮ Run experiments with several models:

Linear − − → subj ⋆ − − → verb ⋆ − →

  • bj ⋆ −

→ and ⋆ − − → subj∗ ⋆ − − → does ⋆ − → too Non-Linear − − → subj ⋆ − − → verb ⋆ − →

  • bj ⋆ −

− → subj∗ ⋆ − − → verb ⋆ − →

  • bj

Lambda-Based T(− − → subj, verb, − →

  • bj) ⋆ T(−

− → subj∗, verb, − →

  • bj)

Picture-Based T(− − → subj ⋆ − − → subj∗, verb, − →

  • bj)

where ⋆ is addition or multiplication, and T is some atested model for a transitive sentence.

  • G. J. Wijnholds

SYCO 2 36 / 41

slide-40
SLIDE 40

EXPERIMENTING WITH VP ELLIPSIS: DISAMBIGUATION RESULTS

CB W2V GloVe FT D2V1 D2V2 ST IS1 IS2 USE Verb Only Vector .4150 .2260 .4281 .2261 Verb Only Tensor .3039 .4028 .3636 .3548

  • Add. Linear

.4081 .2619 .3025 .1292

  • Mult. Linear

.3205

  • .0098

.2047 .2834

  • Add. Non-Linear

.4125 .3130 .3195 .1350

  • Mult. Non-Linear

.4759 .1959 .2445 .0249 Best Lambda .5078 .4263 .3556 .4543 2nd Best Lambda .4949 .4156 .3338 .4278 Best Picture .5080 .4263 .3916 .4572 Sent Encoder .1425 .2369

  • .1764

.3382 .3477 .2564 Sent Encoder+Res .2269 .3021

  • .1607

.3437 .3129 .2576 Sent Encoder-Log .1840 .2500

  • .1252

.3484 .3241 .2252

Table: Spearman ρ scores for the ellipsis disambiguation experiment. CB: count-based, W2V: Word2Vec, FT: FastText, ST: Skip-Thoughts, IS1:

InferSent (GloVe), IS2: InferSent (FastText), USE: Universal Sentence Encoder.

  • G. J. Wijnholds

SYCO 2 37 / 41

slide-41
SLIDE 41

EXPERIMENTING WITH VP ELLIPSIS: SIMILARITY RESULTS

CB W2V GloVe FT D2V1 D2V2 ST IS1 IS2 USE Verb Only Vector .4562 .5833 .4348 .6513 Verb Only Tensor .3946 .5664 .4426 .5337

  • Add. Linear

.7000 .7258 .6964 .7408

  • Mult. Linear

.6330 .1302 .3666 .1995

  • Add. Non-Linear

.6808 .7617 .7103 .7387

  • Mult. Non-Linear

.7237 .3550 .2439 .4500 Best Lambda .7410 .7061 .4907 .6989 2nd Best Lambda .7370 .6713 .4819 .6871 Best Picture .7413 .7105 .4907 .7085 Sent Encoder .5901 .6188 .5851 .7785 .7009 .6463 Sent Encoder+Res .6878 .6875 .6039 .8022 .7486 .6791 Sent Encoder-Log .1840 .6599 .4715 .7815 .7301 .6397

Table: Spearman ρ scores for the ellipsis similarity experiment. CB: count-based, W2V: Word2Vec, FT: FastText, ST: Skip-Thoughts, IS1: InferSent

(GloVe), IS2: InferSent (FastText), USE: Universal Sentence Encoder.

  • G. J. Wijnholds

SYCO 2 38 / 41

slide-42
SLIDE 42

Conclusion 2: Classical vs. Qantum Entanglement Experimentally, the linear approximation that Frobenius Semantics gives is equally performant to the classical semantics!

  • G. J. Wijnholds

SYCO 2 39 / 41

slide-43
SLIDE 43

Future Work

  • 1. Entailment:

Dogs sleep and cats too ⇒ cats walk

  • 2. Guess the antecedent (ambiguity!):

Dogs run, cats walk, and foxes …

  • 3. Negation:

Dogs sleep but cats do not.

  • G. J. Wijnholds

SYCO 2 40 / 41

slide-44
SLIDE 44

Thank you! g.j.wijnholds@qmul.ac.uk

  • G. J. Wijnholds

SYCO 2 41 / 41