CS 401 Integer Multiplication / Matrix Multiplication Xiaorui Sun - - PowerPoint PPT Presentation

CS 401

Integer Multiplication / Matrix Multiplication

Xiaorui Sun

1

Integer Multiplication

Integer Arithmetic

Add: Given two !-bit integers " and #, compute " + #. Multiply: Given two !-bit integers " and #, compute "×#. The “grade school” method:

3 1 1 1 1 1 1 1 + 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Add

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 *

Multiply

&(!) bit operations. &(!)) bit operations.

Divide and Conquer

Let !, # be two $-bit integers Write ! = 2'/)!* + !, and # = 2'/)#* + #, where !,, !*, #,, #* are all $/2-bit integers. Therefore,

• $ = 4- $

2 + Θ($) So,

• $ = Θ $) .

! = 2'/) ⋅ !* + !, # = 2'/) ⋅ #* + #, !# = 2'/) ⋅ !* +!, 2'/) ⋅ #* + #, = 2' ⋅ !*#* + 2 ⁄

' ) ⋅ !*#, + !,#* + !,#,

We only need 3 values !*#*, !,#,, !*#, + !,#* Can we find all 3 by only 3 multiplication?

Key Trick: 4 multiplies at the price of 3

! = 2$/& ⋅ !( + !* + = 2$/& ⋅ +( + +* !+ = 2$/& ⋅ !( +!* 2$/& ⋅ +( + +* = 2$ ⋅ !(+( + 2 ⁄

$ & ⋅ !(+* + !*+( + !*+*

• = !( + !*

. = +( + +*

• . = !( + !*

+( + +* = !(+( + !(+* + !*+( + !*+* !(+* + !*+( = -. − !(+( − !*+*

Key Trick: 4 multiplies at the price of 3

Theorem [Karatsuba-Ofman, 1962] Can multiply two n-digit integers in O(n1.585…) bit operations. To multiply two n-bit integers:

Add two !/2 bit integers. Multiply three !/2-bit integers. Add, subtract, and shift !/2-bit integers to obtain result.

$ ! = 3$ ! 2 + ( !

) = 2*/+ ⋅ )- + ). ⇒ 0 = )- + ). 1 = 2*/+ ⋅ 1- + 1. ⇒ 2 = 1- + 1. )1 = 2*/+ ⋅ )- +). 2*/+ ⋅ 1- + 1. = 2* ⋅ )-1- + 2 ⁄

* + ⋅ )-1. + ).1- + ).1.

A B 02 − 5 − 6

Quiz

Consider the following recurrence.

• A. ! " = O "%&'( ) = *(",../.)
• B. ! " = O "log "
• C. ! " = O "
• D. None of above

! " = 4 O 1 if " = 1 3!("/2) + O(") if " > 1

Integer Multiplication (Summary)

• Amusing exercise: generalize Karatsuba to do 5 size

!/3 subproblems

This gives Θ !%.'(… time algorithm Still open problem.

Matrix Multiplication

10

Multiplying Matrices

Let ! be an "×$ matrix, % be an $×& matrix. Then, ' = !% is an "×& matrix such that

11

ú ú ú ú û ù ê ê ê ê ë é

• ú

ú ú ú û ù ê ê ê ê ë é

44 43 42 41 34 33 32 31 24 23 22 21 14 13 12 11 44 43 42 41 34 33 32 31 24 23 22 21 14 13 12 11

b b b b b b b b b b b b b b b b a a a a a a a a a a a a a a a a

ú ú ú ú û ù ê ê ê ê ë é + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + =

44 44 34 43 24 42 14 41 42 44 32 43 22 42 12 41 41 44 31 43 21 42 11 41 44 34 34 33 24 32 14 31 42 34 32 33 22 32 12 31 41 34 31 33 21 32 11 31 44 24 34 23 24 22 14 21 42 24 32 23 22 22 12 21 41 24 31 23 21 22 11 21 44 14 34 13 24 12 14 11 42 14 32 13 22 12 12 11 41 14 31 13 21 12 11 11

b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a ! ! ! !

Multiplying Matrices

12

ú ú ú ú û ù ê ê ê ê ë é

• ú

ú ú ú û ù ê ê ê ê ë é

44 43 42 41 34 33 32 31 24 23 22 21 14 13 12 11 44 43 42 41 34 33 32 31 24 23 22 21 14 13 12 11

b b b b b b b b b b b b b b b b a a a a a a a a a a a a a a a a

ú ú ú ú û ù ê ê ê ê ë é + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + =

44 44 34 43 24 42 14 41 42 44 32 43 22 42 12 41 41 44 31 43 21 42 11 41 44 34 34 33 24 32 14 31 42 34 32 33 22 32 12 31 41 34 31 33 21 32 11 31 44 24 34 23 24 22 14 21 42 24 32 23 22 22 12 21 41 24 31 23 21 22 11 21 44 14 34 13 24 12 14 11 42 14 32 13 22 12 12 11 41 14 31 13 21 12 11 11

b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a ! ! ! !

Multiplying Matrices

13

ú ú ú ú û ù ê ê ê ê ë é

• ú

ú ú ú û ù ê ê ê ê ë é

44 43 42 41 34 33 32 31 24 23 22 21 14 13 12 11 44 43 42 41 34 33 32 31 24 23 22 21 14 13 12 11

b b b b b b b b b b b b b b b b a a a a a a a a a a a a a a a a

ú ú ú ú û ù ê ê ê ê ë é + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + =

44 44 34 43 24 42 14 41 42 44 32 43 22 42 12 41 41 44 31 43 21 42 11 41 44 34 34 33 24 32 14 31 42 34 32 33 22 32 12 31 41 34 31 33 21 32 11 31 44 24 34 23 24 22 14 21 42 24 32 23 22 22 12 21 41 24 31 23 21 22 11 21 44 14 34 13 24 12 14 11 42 14 32 13 22 12 12 11 41 14 31 13 21 12 11 11

b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a b a ! ! ! !

A11 A12 A21 A11B12+A12B22 A22 A11B11+A12B21 B11 B12 B21 B22 A21B12+A22B22 A21B11+A22B21

Multiplying Matrices

14

A11 A12 A21 A11B12+A12B22 A22 A11B11+A12B21 B11 B12 B21 B22 A21B12+A22B22 A21B11+A22B21 =

Simple Divide and Conquer

Naive Strassen

15

Strassen’s Divide and Conquer Algorithm

16

• Strassen’s algorithm

Multiply !×! matrices using # instead of $ multiplications (and 18 additions) ' ( = 7' ( 2 + 18(- Hence, we have ' ( = . (/012 3 .

Strassen’s Divide and Conquer Algorithm

Useful when (~500. I am curious how large ( need? One of the most important open problem: Solve matrix multiplication in O((-log= > () time

Summary of Divide and Conquer

Divide: We reduce a problem to several subproblems.

each sub-problem is at most a constant fraction of the size of the original problem

Conquer: Recursively solve each subproblem Combine: Merge the solutions Examples:

• Mergesort, Binary Search, Closest Point Pair, Integer

Multiplication, Matrix Multiplication Log n levels

n n/2 n/2 n/4

How to solve problems by D&C

Define subproblems

• Sometimes, subproblems are same to the original

problem (just input size is smaller), e.g. Mergesort

• Sometimes, need to redefine the subproblems to allow

solving subproblem recursively, e.g. Counting Inversions Algorithm design: how to obtain solution of a subproblem from solutions of smaller subproblems?

18

Master Theorem

Write down recurrence relation ! " = $ ! " % + Θ(")) Master theorem: Suppose ! " = $ !

+ , + Θ(")) for all

" > %. Then,

• If $ < %) then ! " = Θ ")
• If $ = %) then ! " = Θ ")log "
• If $ > %) then ! " = Θ "34567