Linear algebra NEU 466M Instructor: Professor Ila R. Fiete Spring - - PowerPoint PPT Presentation

Linear algebra

NEU 466M Instructor: Professor Ila R. Fiete Spring 2016

NotaBon

• Matrices: upper-case
• Vector: bold, (usually) lower-case

(handwriBng: )

• Elements of matrix, vector: lower-case
• Scalar numbers: lower-case, no indices

A, B, U, W x, y, v, w aij, bi, vj, ukl

x → x

a, b, c, γ, α

Vectors and matrices

size (n x m) matrix

A =     a11 a12 · · · a1m a21 a22 · · · a2m · · · · · · · · · · · · an1 an2 · · · anm    

v =      v1 v2 . . . vm     

size (m x 1) column vector

vi ∈ R v ∈ Rm A ∈ Rn×m

What is a vector?

v1 v2 v3

geometric view

v =      v1 v2 . . . vm     

size (m x 1) column vector

plotv in matlab

Vector length

Length (norm):

||v||

v =      v1 v2 . . . vm     

||v|| = q v2

1 + v2 2 + · · · v2 m

Vector-scalar product

v αv

αv =      αv1 αv2 . . . αvm     

geometric view same direcBon, different length

Sum of vectors

v geometric view Adding vectors: stacking them end-to-end

v + u =      v1 + u1 v2 + u2 . . . vm + um     

u v + u

v, u ∈ Rm

v

u

v

u

Unit vector: any vector of length 1

ˆ e1 ˆ e2 ˆ e3

Every point on (m-1)-dimensional sphere of unit radius in m-dim space is a unit vector

||v|| = q v2

1 + v2 2 + · · · v2 m = 1

= v u u t

m

X

i=1

v2

i

2 m = 1

Vector, matrix transpose

size (n x m) matrix

A =     a11 a12 · · · a1m a21 a22 · · · a2m · · · · · · · · · · · · an1 an2 · · · anm    

v =      v1 v2 . . . vm     

size (m x 1) column vector

AT =     a11 · · · an1 a12 · · · a2m · · · · · · · · · a1m · · · anm    

size (m x n) matrix

vT = [v1v2 · · · vm]

size (1 x m) row vector

Vector norm as an inner product

vT v = [v1v2 · · · vm]

=      v1 v2 . . . vm     

= ||v||2

=

m

X

i=1

v2

i

Inner product (dot product)

=      v1 v2 . . . vm     

v, u ∈ Rm

uT v = [u1 u2 · · · um]

= X

i

uivi

Geometric view: projecBon of v on u, Bmes norm of u:

uT v = ||u||||v|| cos(θ)

v

u

||u||||v|| cos(θ)

Inner product (dot product)

Example: unit vector along x-axis, v

u

uT v = v1

u =  1

• v =

 v1 v2

• 1

u, v ∈ R2

Inner product (dot product)

v, u ∈ Rm

Example: v

u

u ⊥ v

uT v = ||u||||v|| cos(θ) = 0

System of equaBons

a11v1 + · · · + a1mvm = b1 a21v1 + · · · + a2mvm = b2 · · · · · · · · · an1v1 + · · · + anmvm = bn

n equaBons in m unknowns (v1,…vm):

System of equaBons

(n x m) (m x 1) (n x 1)

a11v1 + · · · + a1mvm = b1 a21v1 + · · · + a2mvm = b2 · · · · · · · · · an1v1 + · · · + anmvm = bn

n equaBons in m unknowns (v1,…vm):

Av = b

    a11 a12 · · · a1m a21 a22 · · · a2m · · · · · · · · · · · · an1 an2 · · · anm    

     v1 v2 . . . vm           b1 b2 . . . bn     

=

System of equaBons: when does unique soluBon exist?

(m x m) (m x 1) (n x 1)

n equaBons in m unknowns: generically, a unique soluBon exists when same number of constraints (n) as unknowns (m): Thus, n=m or A is square.

Av = b

     v1 v2 . . . vm      =

(m x m) (m x 1) (m x 1)

=

m m

    a11 · · · a1m a21 · · · a2m · · · · · · · · · am1 · · · amm    

     b1 b2 . . . bm     

this is an algebraic view. Bme for some geometric insight.

Geometric view: when does a unique soluBon exist?

unknowns x1, x2

a11x1 + a12x2 = b1 a21x1 + a22x2 = b2

x1 x2

Start with 2-dimensional problem: 2 unknowns, 2 equaBons equaBon of a line

a11x1 + a12x2 = b1

a21x1 + a22x2 = b2

soluBon: at intersecBon where both equaBons hold

Geometric view: when does a unique soluBon exist?

unknowns x1, x2

a11x1 + a12x2 = b1 a21x1 + a22x2 = b2

x1 x2

Start with 2-dimensional problem: 2 unknowns, 2 equaBons equaBon of a line

a11x1 + a12x2 = b1

a21x1 + a22x2 = b2

soluBon: at intersecBon where both equaBons hold

Generically two infinite lines in 2D space intersect at a (single) locaBon thus (unique) soluBon exists.

Geometric view: when does a unique soluBon not exist?

x2

• 1. Offset parallel lines: no soluBon exists

a11x1 + a12x2 = b1

x1

a21x1 + a22x2 = b2

b1/a11 b2/a21

Algebra: when does a unique soluBon not exist?

x2

• 1. Offset parallel lines: no soluBon exists

a11x1 + a12x2 = b1

x1

a21x1 + a22x2 = b2

b1/a11 b2/a21

a11a22 = a12a21

equal slopes

a11a22 − a12a21 = 0

a21/a22 = a11/a12

Algebra: when does a unique soluBon not exist?

x2

• 2. Aligned parallel lines: infinitely many soluBons

a11x1 + a12x2 = b1

x1

a21x1 + a22x2 = b2

b1/a11 b2/a21 equal slopes

a11a22 − a12a21 = 0

equal intercepts

b1/a11 = b2/a21

Algebraic view: existence of unique soluBon in terms of coefficient matrix A

a11a22 − a12a21 = 0

A =  a11 a12 a21 a22

• det(A) ≡

determinant:

2-dim system of equaBons with square coefficient matrix A has a unique soluBon when:

det(A) 6= 0

Same condiBon for m-dim system of equaBons with square coefficient matrix.

Linear system: possibiliBes

• 1 unique soluBon
• No soluBons
• Infinitely many soluBons