[PPT] - CSCI 562: EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING 25 Aug PowerPoint Presentation

SLIDE 1

CSCI 562: EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING

25 Aug 2009

SLIDE 2

WHAT WE WANT

A.L.I.C.E. What we’ve got?

SLIDE 3

WHAT WOULD YOU DO?

Where is USC located?

SLIDE 4

DATA

1980s: if you wanted a computer to “know” something, you

had to program it in.

Now: just about everything has been posted on the Web by

someone, somewhere. But, it is all in natural language.

SLIDE 5

BAG OF WORDS

Where is USC located?

USC’s two primary campuses, both located in the heart of Los Angeles, welcome thousands of guests and visitors each year. The 226-acre University Park campus, home to the College of Letters, Arts and Sciences, the Graduate School, and most of USC’s professional schools, is adjacent to Exposition Park with its world-class museums and recreational facilities. A few miles to the northeast is the 61-acre Health Sciences campus, home to the Keck School of Medicine of USC and the School of Pharmacy as well as three major teaching hospitals.

is 1 located 1 usc 1 where 1 is 2 located 1 two 1 usc 3 where …

SLIDE 6

AMBIGUITY

Where can a snow leopard be seen?

SLIDE 7

GRAMMAR

What animal does a frog eat?

SLIDE 8

GRAMMAR

SBARQ WHNP SINV VBZ NP What animal does a frog VP VB NP eat t

SLIDE 9

MULTILINGUALITY

Tell me about Tai Yen-Hui.

SLIDE 10

STRUCTURE

All language has various levels of structure, more than just a

bag of words

Virtually all Natural Language Processing tasks involve inferring

structure from text or transforming one kind of structure into another

SLIDE 11

CS 562 - Intro (part 2)

CS 562 - Lecture 1, part II

More about ambiguities
Key problems to address in this class
grammar formalisms
search algorithms
learning methods
Learning Examples

11

SLIDE 12

CS 562 - Intro (part 2)

More about Ambiguities

to middle school kids: what does this sentence mean?

12

Aravind Joshi

I saw her duck.

lexical ambiguity (word-sense)

SLIDE 13

CS 562 - Intro (part 2)

More about Ambiguities

13

Aravind Joshi

I eat sushi with tuna.

to middle school kids: what does this sentence mean?

structural ambiguity (PP-attachment)

SLIDE 14

CS 562 - Intro (part 2)

More about Ambiguities

14

Aravind Joshi

I eat sushi with tuna.

to middle school kids: what does this sentence mean?

lexical ambiguity (word-sense)

SLIDE 15

CS 562 - Intro (part 2)

More about Ambiguities

15

Aravind Joshi

Everybody loves somebody.

to middle school kids: what does this sentence mean?

structural ambiguity (quantifier scope)

???

SLIDE 16

CS 562 - Intro (part 2)

More about Ambiguities

16

Aravind Joshi

to middle school kids: what does this sentence mean?

Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo

Dogs dogs dog dog dogs.

Police police police police police

http://www.cse.buffalo.edu/~rapaport/BuffaloBuffalo/buffalobuffalo.html

SLIDE 17

CS 562 - Intro (part 2)

Ambiguities in Translation

17

zi zhu zhong duan 自助终端 self help terminal device

SLIDE 18

CS 562 - Intro (part 2)

Ambiguities in Translation

18

SLIDE 19

CS 562 - Intro (part 2)

r even...

19

clear evidence that NLP is used in real life!

SLIDE 20

CS 562 - Intro (part 2)

Ambiguity Explosion

how about...
I saw her duck with a telescope.
I saw her duck with a telescope in the garden...

20

... I saw her duck.

SLIDE 21

CS 562 - Intro (part 2)

Ambiguity Explosion

exponential explosion of the search space
Q1: how to represent ambiguities (compactly)?
Q2: how to search over this space (efficiently)?
Q3: how to rank different hypotheses?

21

..

S NP PRP I VP VBD saw NP PRP$ her NN duck PP IN with NP DT a NN telescope

SLIDE 22

CS 562 - Intro (part 2)

Answers...

Q1: how to represent ambiguities?
context-free grammar (unit 2)
finite-state automata (unit I)
Q2: how to search in this space?
dynamic programming (units 1&2)
Q3: how to rank these hypotheses?
weighted grammar (units 1-3)
weights learned from data
(saw, with, telescope) seen more often in texts

22

S NP PRP I VP VBD saw NP PRP$ her NN duck PP IN with NP DT a NN telescop

SLIDE 23

CS 562 - Intro (part 2)

Why Learning?

learning is better than hand-written rules, because:
less work; easily adapts to new languages/domains
Powerset (now bing.com): 15 years for English grammar!
now they are writing their Chinese grammar...
and languages constantly change!
learning can work, and often works better!
machine translation: used to be dominated by rule-based
now statistical methods are better: google vs. systran
google learns from the web, and translates 40+ langs

23

[also CS 567, Machine Learning, Fall 2009]

SLIDE 24

CS 562 - Intro (part 2)

Example - Rosetta Stone

the most famous (tri-)parallel text
machines can do the same job! (if given parallel text)
UN/EU/Ca proceedings, News, tech docs, ...

24

SLIDE 25

A sci-fi example

(Knight, 1997)

farok crrrok hihok yorok clok kantok ok-yurp Y

ur assignment: translate this Centauri

sentence into Arcturan

SLIDE 26

1c. ok-voon ororok sprok .
1a. at-voon bichat dat .
7c. lalok farok ororok lalok sprok izok enemok .
7a. wat jjat bichat wat dat vat eneat .
2c. ok-drubel ok-voon anok plok sprok .
2a. at-drubel at-voon pippat rrat dat .
8c. lalok brok anok plok nok .
8a. iat lat pippat rrat nnat .
3c. erok sprok izok hihok ghirok .
3a. totat dat arrat vat hilat .
9c. wiwok nok izok kantok ok-yurp .
9a. totat nnat quat oloat at-yurp .
4c. ok-voon anok drok brok jok .
4a. at-voon krat pippat sat lat .
10c. lalok mok nok yorok ghirok clok .
10a. wat nnat gat mat bat hilat .
5c. wiwok farok izok stok .
5a. totat jjat quat cat .
11c. lalok nok crrrok hihok yorok zanzanok .
11a. wat nnat arrat mat zanzanat .
6c. lalok sprok izok jok stok .
6a. wat dat krat quat cat .
12c. lalok rarok nok izok hihok mok .
12a. wat nnat forat arrat vat gat .