Boolean Matrix Multiplication (BMM) and CFG parsing by Franziska - PowerPoint PPT Presentation

Seminar “Formal Grammars” Boolean Matrix Multiplication (BMM) and CFG parsing by Franziska Ebert 23.3.2007 Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 1

Overview 1. Motivation 2. Reduction of CFG parsing to BMM (Valiant - 1975) 3. Reverse direction: Conversion of a CFG parser into an algorithm for BMM (Lee - 2002) 4. Conclusion Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 2

Motivation CFG parsing Little success in finding Lee CFG parsing in O ( n 3 ) Valiant a better algorithm Fast CFG parsing O ( M ( n )) O ( n 3 / log 2 n ) requires fast BMM CKY, Earley Improvement 1969 1975 2002 O ( m 2 . 376 ) Strassen but still Strassen’s algorithm O ( n 2 . 81 ) BMM is practically the best Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 3

Motivation CFG parsing Little success in finding Lee CFG parsing in O ( n 3 ) Valiant a better algorithm Fast CFG parsing O ( M ( n )) O ( n 3 / log 2 n ) requires fast BMM CKY, Earley Improvement 1969 1975 2002 O ( m 2 . 376 ) Strassen but still Strassen’s algorithm O ( n 2 . 81 ) BMM is practically the best Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 4

Motivation • General context-free recognition in less than cubic time (Valiant - 1975) – CKY, Earley: O ( n 3 ) → CFG parsing in less than cubic time → Reduction of CFG parsing to BMM → Today: asymptotically fastest known algorithm Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 5

Motivation • Fast CFG parsing requires fast BMM (Lee - 2002) – Little success in developing practical fast CFG parsers – Practically fastest BMM algorithm: Strassen - O ( n 2 . 81 ) → No practical fast BMM algorithm → Proof: CFG parsing relies on BMM → Conversion of a CFG parser ( O ( gn 3 − ǫ ) ) into an algorithm for BMM ( O ( m 3 − ǫ/ 3 ) ) Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 6

Reduction of CFG parsing to BMM • General idea for an MM algorithm to parse CFG → Build upper triangular matrix → Define new matrix multiplication → Matrix has the form of CKY recognition matrix → A k ∈ b ij ⇔ A k → ∗ w j − 1 i     ∅ { A } ∅ ∅ ∅ ∅ { A } { X } ∅ { S }      ∅ ∅ { A } ∅ ∅   ∅ ∅ { A } ∅ ∅          ⇒ ∗ ∅ ∅ ∅ { B } ∅ ∅ ∅ ∅ { B } { Y }             ∅ ∅ ∅ ∅ { B } ∅ ∅ ∅ ∅ { B }         ∅ ∅ ∅ ∅ ∅ ∅ ∅ ∅ ∅ ∅ Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 7

Preliminaries • CFG ( N, Σ , P, A 1 ) in Chomsky normal form: – N : set of nonterminals { A 1 , . . . , A h } – Σ : set of terminals – P : set of productions – A 1 : the starting symbol • A i → ∗ w denotes: w can be derived from A i Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 8

Preliminaries Definition: ∗ -operator on subsets of N N 1 ∗ N 2 = { A i | ∃ A j ∈ N 1 , A k ∈ N 2 . ( A i → A j A k ) ∈ P } Example: X → Y Z { A, Y } ∗ { Z } = { X } Definition: Matrix Multiplication over matrices with subsets of N as elements. n � a ∗ b = c : ⇔ c ik = a ij ∗ b jk j =1 Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 9

Preliminaries Definition: Transitive Closure of a square matrix a + = a (1) ∪ a (2) ∪ . . . where i − 1 a ( j ) ∗ a ( i − j ) a ( i ) = � a (1) = a and j =1 Definition: Union of two matrices a ∪ b = c : ⇔ c ij = a ij ∪ b ij Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 10

Reduction of CFG parsing to BMM • Input: CFG G = ( N, Σ , P, A 1 ) and w = x 1 . . . x n • Output: A 1 → ∗ w ? • Algorithm: Let b be a ( n + 1) × ( n + 1) -matrix 1. b i,j = ∅ ∀ 1 ≤ i, j ≤ ( n + 1) 2. b i,i +1 = { A k | ( A k → x i ) ∈ P } 3. Compute a = b + 4. Check whether A 1 ∈ a 1 ,n +1 Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 11

Reduction of CFG parsing to BMM - Example S → XY   ∅ { A } ∅ ∅ ∅ X → XA | AA    ∅ ∅ { A } ∅ ∅  Y → Y B | BB     A → a a := ∅ ∅ ∅ { B } ∅       B → b ∅ ∅ ∅ ∅ { B }     ∅ ∅ ∅ ∅ ∅ w = aabb b i,i +1 = { A k | ( A k → x i ) ∈ P } Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 12

Reduction of CFG parsing to BMM - Example S → XY   ∅ { A } { X } ∅ ∅ X → XA | AA    ∅ ∅ { A } ∅ ∅  Y → Y B | BB     A → a a := ∅ ∅ ∅ { B } { Y }       B → b ∅ ∅ ∅ ∅ { B }     ∅ ∅ ∅ ∅ ∅ w = aabb a := a (1) a (2) ∪ ���� ���� = a = a (1) ∗ a (1) Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 13

Reduction of CFG parsing to BMM - Example S → XY   ∅ { A } { X } ∅ { S } X → XA | AA    ∅ ∅ { A } ∅ ∅  Y → Y B | BB     A → a a := ∅ ∅ ∅ { B } { Y }       B → b ∅ ∅ ∅ ∅ { B }     ∅ ∅ ∅ ∅ ∅ w = aabb a := a (1) ∪ a (2) ∪ a (3) ���� = a (1) ∗ a (2) ∪ a (2) ∗ a (1) Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 14

Time Bounds • Time complexity: O ( n 2 . 81 ) • T ( n ) := time to compute transitive closure • BM ( n ) := complexity for BMM algorithm • M ( n ) := complexity for multiplying two matrices T ( n ) ≤ M ( n ) · const M ( n ) ≤ BM ( n ) · const O ( BM ( n )) Time complexity for CFG parsing: O ( n 2 . 81 ) Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 15

An Algorithm for BMM Fundamental Theorem: Fast context-free grammar parsing requires fast boolean matrix multiplication. → Any CFG-parser running in O ( gn 3 − ǫ ) can be converted with little computational overhead into an O ( m 3 − ǫ/ 3 ) BMM algorithm Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 16

An Algorithm for BMM BMM algorithm ( G, w ) F G,w C reduction c−parser querier ( A, B ) Conversion of a CFG parser into an algorithm for BMM Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 17

Preliminaries Definition: c-derivation A 1 A k ∈ N c-derives w j i iff • A k → ∗ w j i , and A k • A 1 → ∗ w i − 1 A k w n j +1 1 w 1 . . . w i − 1 w i . . . w j w j +1 . . . w n Definition: c-parser A c-parser is an algorithm that takes a CFG G and a string w and outputs F G,w with: • A k c-derives w j i ⇒ F G,w = “yes” • A k �→ ∗ w j i ⇒ F G,w = “no” • F G,w answers queries in constant time. Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 18

Reduction: ( A, B ) → ( G, w ) • A , B are m × m boolean matrices • All information about A and B is coded in the grammar → string w does not depend on A and B . • Recall: C = A ∗ B is defined as c ij = � m k =1 ( a ik ∧ b kj ) → c ij = 1 ↔ ∃ k.a ik = b kj = 1 Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 19

Reduction: ( A, B ) → ( G, w ) • General Idea: – For each a ij = 1 introduce production A i,j → w i Ww j – For each b ij = 1 introduce production B i,j → w i +1 Ww j – For each c ij introduce production C i,j → A i,k B k,j ∀ 1 ≤ k ≤ m – c ij = 1 iff C i,j c-derives w j i S C i,j W A i,k B k,j W w 1 w i − 1 w i w k w k +1 w j w j +1 w n Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 20

Reduction: ( A, B ) → ( G, w ) • To obtain smaller complexity: → Keep grammar size small → Split indices, e.g. i = ( i 1 , i 2 ) • i 1 = ⌊ i/d ⌋ and i 2 = ( i mod d ) + 2 , with d = ⌈ m 1 / 3 ⌉ → 0 ≤ i 1 ≤ d 2 and 2 ≤ i 2 ≤ d + 1 → i can be computed uniquely by i 1 and i 2 Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 21

Reduction: ( A, B ) → ( G, w ) • A i 1 ,j 1 → w i 2 Ww j 2 + δ , with δ := d + 2 • String w = w 1 . . . w δ w δ +1 . . . w 2 δ w 2 δ +1 . . . w 3 δ → Σ = { w l | 1 ≤ l ≤ 3 δ = 3 d + 6 } Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 22

Reduction: ( A, B ) → ( G, w ) • W-rules: W → w l W | w l • A-rules: a ij = 1 ⇒ A i 1 ,j 1 → w i 2 Ww j 2 + δ • B-rules: b ij = 1 ⇒ B i 1 ,j 1 → w i 2 +1+ δ Ww j 2 +2 δ • C-rules: C i 1 ,j 1 → A i 1 ,k 1 B k 1 ,j 1 • S-rules: S → WC i 1 ,j 1 W Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 23

Reduction: ( A, B ) → ( G, w ) Theorem: For 1 ≤ i, j ≤ m , the entry c ij is 1 iff C i 1 ,j 1 c-derives w j 2 +2 δ . i 2 Proof: ” ⇒ ” to show: C i 1 ,j 1 c-derives w j 2 +2 δ i 2 C i 1 ,j 1 → A i 1 ,k 1 B k 1 ,j 1 C-rule → ∗ w i 2 Ww k 2 + δ w k 2 + δ +1 Ww j 2 +2 δ ∃ k.a ik = b kj = 1 → ∗ w j 2 +2 δ W-rule i 2 Boolean Matrix Multiplication and CFG Parsing 23.3.2007 Franziska Ebert 24