= [ [ ] ] = ( ( ) ) A A A A m i j , + + + + < - - PDF document

SLIDE 1

1

Designing a Dynamic Designing a Dynamic Programming Algorithm for an Programming Algorithm for an Optimization Problem Optimization Problem

• Step 1.

Step 1. Characterize the structure of optimal Characterize the structure of optimal solution solution

• Step 2.

Step 2. Recursively define the value of an Recursively define the value of an

• ptimal solution
• ptimal solution
• Step 3.

Step 3. Compute the value of an optimal Compute the value of an optimal solution in a bottom solution in a bottom-

• up fashion

up fashion

• Step 4.

Step 4. Construct an optimal solution from Construct an optimal solution from computed information computed information Dynamic Programming: Matrix Chain Multiplication

Matrix Chain Multiplication Problem Given a chain of matrices, where (for ) is a matrix Fully parenthesize the product so that the number of scalar multiplications is minimum minimum

1 2

, , ,

n

A A A … n 1,2, , i n = …

i

A

i i

p p ×

1 2 n

A A A

• Time Complexity of Matrix Multiplication

Time Complexity of Matrix Multiplication Matrix-Multiply(A, B) if columns[A] rows[B] then error “incompatible dimensions” else for i1 to rows[A] = p do for j 1 to columns[B] = r do C[I, j] 0 for k 1 to columns[A] = q do C[I, j] C[I, j] + A[I, k]*B[k, j] return C

≠

p x q q x r

{{{

pqr rq q Example Two ways to parenthesize A1 A2 A3 10 100 5 50 ( A1 A2 ) A3 10 100 5 50

= (10)(100)(5) + (10)(5)(50) = 5000 + 2500 = 7500

A1 ( A2 A3 ) 10 100 5 50

= (100)(5)(50) + (10)(100)(50) = 25000 + 50000 = 75000

SLIDE 2

2

1 2 3 4

A A A A

Can be parenthesized in 5 ways

( ) ( ) ( ) ( )

( ) ( )

1 2 3 4

A A A A

( ) ( ) ( ) ( )

( ) ( )

1 2 3 4

A A A A

( ) ( )( ) ( )

( ) ( )

1 2 3 4

A A A A

( ) ( ) ( ) ( )

( ) ( )

1 2 3 4

A A A A

( ) ( ) ( ) ( )

( ) ( )

1 2 3 4

A A A A

Let number of ways to parenthesize a product of n matrices, then

( ) P n =

1

1 1 ( ) ( ) ( ) 2

n k

if n P n P k P n k if n

=

=   =  − ≥ − ≥  ∑

It can be shown that , where is the (n-1)-th Catalan number

( ) ( 1) P n C n = − = − ( 1) C n −

3/ 2

2 1 4 ( ) 1

n

n C n n n n     = = = = Ω     +    

Notation

1 i j i i j

A A A A

+

=

…

• [

] [ ]

, m i j = Min # of mult. to evaluate

i j

A … [ ] [ ]

( ) ( )

, 1, 1

, min

i k k j i k j i k j

if i j m i j m m p p p if i j

+ − + − ≤ ≤ ≤ ≤

=   =  + + + + <  

[ ] [ ]

, s i j = that k such that

[ ] [ ] [ ] [ ]

1

, , 1,

i k j

m i j m i k m k j p p p

−

= + = + + +

Objective

Find a way to fully parenthesize the product in such a way that the cost is minimum

1 1 2 n n

A A A A =

…

SLIDE 3

3

• Optimal Substruc. Property ( --- ) ( --- )
• Overlapping Subproblems Property

Recursive formula for m[I, j] leads to an exponential problem Observation: Relatively few problems Set of problems { i…j } So # of subprobs. is the # of ways of choosing I, j such that , which is

1 i j n ≤ ≤ ≤ ≤ ≤ 2 n n   +     ⇒

Overlap

( )( ) ( ) ( )

SLIDE 4

4

Analysis of Recursive Analysis of Recursive-

• Matrix

Matrix-

• Chain

Chain

( ) T n = time to compute optimal parenth. of

chain of n matrices

( ) ( )

1 1

(1) 1 ( ) 1 ( ) ( ) 1 1

n k

T T n T k T n k for n

− =

≥    ≥ + ≥ + + − + >  

∑

( ) ( )

1 1 1 1

( ) 1 ( ) ( ) 1 2 ( )

n k n k

T n T k T n k T k n

− = − =

≥ + ≥ + + − + = + = +

∑ ∑

We use math induction to prove that

( ) ( )

( ) 2n T n = Ω = Ω

The basis step is easy:

(1) 1 2 T ≥ = ≥ =

Assuming the Induction hypothesis for , we have:

1

( ) 2 T

−

≥

• 1

n ≤ − ≤ −

• (

) ( )

1 2 1 1 1 1

( ) 2 2 2 2 2 2 1 2 2 2

n n k k k k n n n

T n n n n n

− − − − − = = = = − −

≥ + ≥ + = + = − = − + = − + ≥

∑ ∑

SLIDE 5

5

Memoization Memoization

• Create a table for solns to subprobs so far

computed

• Initialize each table entry to a symbol

indicating subprob not yet computed

• Each time a subproblem is first encountered,

its soln is computed and placed in the table

• Each subsequent time it is encountered, its

previously computed soln is simply retrieved from the table Memoization Memoization How to Design a Dynamic How to Design a Dynamic Prog

• Prog. Algorithm

. Algorithm

• 1. Identify the subproblems –
• 2. Characterize the op. soln as a combination
• f op solns to subprobs –
• Subprobs must be of same type as original probs
• Subprobs must be smaller
• How to specify each subprob?
• What is the base case? What is the recursive case?
• Focus on value of soln rather than description of soln
• Introduce notation to rep. value of op soln to each subprob
• Consider trying all possible solns to prob

How to Design a Dynamic How to Design a Dynamic Prog

• Prog. Algorithm

. Algorithm

• 3. Apply “generic dynamic prog. algorithm –
• 4. Modify your algorithm to compute

description of soln to value of soln, if needed

• Use bottom-up version
• use top-down version (Memoization)
• Add additional table

SLIDE 6

6

Longest Common Subsequence Longest Common Subsequence (LCS) (LCS) If then common subsequences are, for example, and Both are common subsequences (CS). But

• nly the last is a LCS.

, , , , , , , , , , , X A B C B D A B Y B D C A B A = = BCA BCBA

{

Longest Common Subsequence (LCS) Longest Common Subsequence (LCS)

• Def. A seq

is a subseq of if there is a strictly increasing seq

• f the indices of

such that LCS Problem: Given two seqs, find an LCS Notation: Given , the i-th prefix is defined as the subseq

1 2,

, ,

k

Z z z z = …

1 2,

, ,

n

X x x x = …

1 2

, , , k i i i … X

j i j

x z =

1 2,

, ,

n

X x x x = …

i

X

1 2,

, ,

i

x x x …

Thm 16 (p351). (Optimal substructure of LCS) Let be seqs, and let , then 1. and 2. . 3. .

1 2 1 2

, , , , , ,

m n

X x x x Y y y y =     =   … …

( ) ( )

1 2

, , , ,

k

Z z z z LCS X Y = = = = …

m n k m n

x y z x y = ⇒ = = = = ( ) ( )

1 1 1

,

k m n

Z LC S X Y

− − − − −

= ( ) ( )

1

& ,

m n m k m

x y Z L C S X Y z x

−

≠   ⇒ =   ≠ 

( ) ( )

1

& ,

m n n k n

x y Z L C S X Y z y

−

≠   ⇒ =   ≠ 

So So … …

Yes NO ? m n

x y

=

? ( ) ( )

1 1 1

& ,

k m n k m n

z x y Z LC S X Y

− − − − −

= = = = =

( ) ( )

1

& ,

k m m

z x Z LCS X Y

−

≠ =

( ) ( )

1

& ,

k n n

z y Z LCS X Y − ≠ =

SLIDE 7

7

Notation Notation Length of points to the entry of corresponding to the optimal subprob soln chosen when computing

[ ] [ ]

, c i j =

( ) ( )

,

i j

LCS X Y [ ] [ ] [ ] [ ] [ ] [ ] [ ] ( ) ( ) , 1 , 1 1 , 0& , 1 , 1 , , 0&

i j i j

if i

• r j

c i j c i j if i j x y M ax c i j c i j if i j x y  = = = =  = − = − − + > =   − − − − > ≠ 

[ ] [ ]

, b i j

[ ] [ ]

,0 c i j … …

[ ] [ ]

, c i j