Linear Programming DPV Chapter 7, Part 2 Jim Royer Part 2 March - - PowerPoint PPT Presentation

Linear Programming

DPV Chapter 7, Part 2

Jim Royer

Part 2

March 25, 2019

Uncredited diagrams are from DPV or homemade. DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 1 / 18

Bipartite matching, 1

Definition

(a)

A graph G = (V, E) is bipartite when V can be partitioned into two nonempty sets L and R and each edge in E has one endpoint in L and the other in R.

(b)

Given a bipartite graph G, a bipartite matching is a set of edges which give an 1-1 correspondence between L and R. (E.g., {(Alice,pie), (Bob,tart), (Carol,cake), (Dan,cookie)})

Alice cake pie cookie tart Bob Carol Dan DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 2 / 18

Bipartite matching, 1

Definition

(a)

A graph G = (V, E) is bipartite when V can be partitioned into two nonempty sets L and R and each edge in E has one endpoint in L and the other in R.

(b)

Given a bipartite graph G, a bipartite matching is a set of edges which give an 1-1 correspondence between L and R. (E.g., {(Alice,pie), (Bob,tart), (Carol,cake), (Dan,cookie)})

Alice cake pie cookie tart Bob Carol Dan

We can turn the problem of finding a bipartite matching into a network flow problem by: Adding a source and target verts s and t. Adding edges from s to each u ∈ L. Adding edges from each v ∈ R to t. Setting the capacity of each edge 1.

Alice cake pie cookie tart Bob Carol Dan t s DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 2 / 18

Bipartite matching, 2

bipartite matching → network flow Add a source and target verts s and t. Add edges from s to each u ∈ L. Add edges from each v ∈ R to t. Set the capacity of each edge 1.

Problem

We do not want fractional flows. (E.g., Dan wants a whole cookie!)

Saving Fact

If all edge capacities are integers, then our algorithm finds an integral optimal flow.

Alice cake pie cookie tart Bob Carol Dan t s

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 3 / 18

Bipartite matching, 2

bipartite matching → network flow Add a source and target verts s and t. Add edges from s to each u ∈ L. Add edges from each v ∈ R to t. Set the capacity of each edge 1.

Problem

We do not want fractional flows. (E.g., Dan wants a whole cookie!)

Saving Fact

If all edge capacities are integers, then our algorithm finds an integral optimal flow.

Alice cake pie cookie tart Bob Carol Dan t s

BUT!

In general it is hard to find solutions to LP problems if you demand all variables have integer values. (Set up for Chapter 8.)

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 3 / 18

Simplex at a very high level, 1

v ← any vertex in the feasible region while there is a neighbor v′ of v with a better objective value do v ← v′

With n variables, we are working in Rn. The feasible region is defined by the constraints:

◮ a1x1 + · · · + anxn = b defines a hyperplane in Rn ◮ a1x1 + · · · + anxn ≤ b and a1x1 + · · · + anxn ≥ b define halfspaces in Rn. ◮ The intersection of halfspaces gives a convex polyhedron. x1 x3 x2

1

• 4
• 2
• 3
• 5
• 6
• 7
• A

B C

max x1 + 6x2 + 13x3 x1 ≤ 200

1

• x2 ≤ 300

2

• x1 + x2 + x3 ≤ 400

3

• x2 + 3x3 ≤ 600

4

• x1 ≥ 0

5

• x2 ≥ 0

6

• x3 ≥ 0

7

• DPV Chapter 7 (Part 2)

Linear Programming March 25, 2019 4 / 18

Simplex at a very high level, 2

v ← any vertex in the feasible region while there is a neighbor v′ of v with a better objective value do v ← v′

Q: What is a vertex? A: A “corner.”

◮ In 2-D, two lines intersect in a point∗. ◮ In 3-D, three planes intersect in a point∗.

E.g., ➀, ➁, and ➆ with equality meet at point A.

∗Watch out for

◮ In n-D, n hyperplanes intersect in a point∗.

degeneracy!

x1 x3 x2

1

• 4
• 2
• 3
• 5
• 6
• 7
• A

B C

max x1 + 6x2 + 13x3 x1 ≤ 200

1

• x2 ≤ 300

2

• x1 + x2 + x3 ≤ 400

3

• x2 + 3x3 ≤ 600

4

• x1 ≥ 0

5

• x2 ≥ 0

6

• x3 ≥ 0

7

• DPV Chapter 7 (Part 2)

Linear Programming March 25, 2019 5 / 18

Simplex at a very high level, 3

v ← any vertex in the feasible region while there is a neighbor v′ of v with a better objective value do v ← v′

Q: What is a neighbor of a vertex? A: They have n − 1 defining constraints in common. E.g., A and C share ➂ and ➆.

x1 x3 x2

1

• 4
• 2
• 3
• 5
• 6
• 7
• A

B C

max x1 + 6x2 + 13x3 x1 ≤ 200

1

• x2 ≤ 300

2

• x1 + x2 + x3 ≤ 400

3

• x2 + 3x3 ≤ 600

4

• x1 ≥ 0

5

• x2 ≥ 0

6

• x3 ≥ 0

7

• DPV Chapter 7 (Part 2)

Linear Programming March 25, 2019 6 / 18

Simplex: Towards the Algorithm

The algorithm has two basic jobs

Job 1. See if the current vertex is optimal. (If so, halt.) Job 2. Determine where to move next.

Both jobs are easy at the origin: 0 = (0, . . . , 0).

Why is the origin so special?

max : cT x A

• x ≤

b

• x ≥

Suppose 0 is feasible.

Job 1.

0 is optimal ⇐ ⇒ ci ≤ 0 for each i. (Why?)

Job 2. If

0 is not optimal, pick some i with ci > 0 and increase xi until we hit another constraint.

What do we do if we are not at the origin?

Transform coordinates to move the origin to where we are.

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 7 / 18

Simplex: Moving the origin

Suppose we want to move the origin to u. Suppose u is given by the constraints: ai · x ≤ bi where i = 1, . . . , n. For i = 1, . . . , n, let yi = bi − ai · x. Then: in x-space in y-space

• ai ·

x ≤ bi ❀ yi ≥ 0

• u

❀

• max:

cT · x ❀ max: (cu + c′T · y) where cu = the cost at u and

• c′ = the transformed cost vector

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 8 / 18

Example from DPV, 1

Initial LP: max 2x1 + 5x2 2x1 − x2 ≤ 4

1

• x1 + 2x2

≤ 9

2

• −x1 + x2

≤ 3

3

• x1

≥

4

• x2

≥

5

• Current vertex: { 4

, 5 } (origin). Objective value: 0. Move: increase x2.

5

is released, 3 becomes tight. Stop at x2 = 3. New vertex { 4 , 3 } has local coordinates (y1, y2): y1 = x1, y2 = 3 + x1 − x2

{ 1 , 2 } { 3 , 4 } { 2 , 3 } y1 x2 Increase Increase { 1 , 5 } { 4 , 5 }

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 9 / 18

Example from DPV, 2

Rewritten LP: max 15 + 7y1 − 5y2 y1 + y2 ≤ 7

1

• 3y1 − 2y2

≤ 3

2

• y2

≥

3

• y1

≥

4

• −y1 + y2

≤ 3

5

• Current vertex: { 4

, 3 }. Objective value: 15. Move: increase y1.

4

is released, 2 becomes tight. Stop at y1 = 1. New vertex { 2 , 3 } has local coordinates (z1, z2): z1 = 3 − 3y1 + 2y2, z2 = y2

{ 1 , 2 } { 3 , 4 } { 2 , 3 } y1 x2 Increase Increase { 1 , 5 } { 4 , 5 }

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 10 / 18

Example from DPV, 3

Rewritten LP: max 22 − 7

3z1 − 1 3z2

− 1

3z1 + 5 3z2

≤ 6

1

• z1

≥

2

• z2

≥

3

• 1

3z1 − 2 3z2

≤ 1

4

• 1

3z1 + 1 3z2

≤ 4

5

• Current vertex: { 2

, 3 }. Objective value: 22. Optimal: all ci < 0. Solve

2

, 3 (in original LP) to get optimal solution (x1, x2) = (1, 4).

{ 1 , 2 } { 3 , 4 } { 2 , 3 } y1 x2 Increase Increase { 1 , 5 } { 4 , 5 }

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 11 / 18

Example from DPV, 3

• z1

• z2

• 1

• 1

• Current vertex: { 2

2019-03-25

Linear Programming Example from DPV, 3 (x1, x2) = (1, 4)

Simplex Issues: Finding the starting vertex

The origin is not always in the feasible region. Q: So how to find a starting vertex? A: It turns out this is another LP problem: Suppose we have min : cT · x ∋ A

• x =

b and x ≥ 0. Then:

◮ Make sure each bi ≥ 0.

If bi < 0, multiply both sides of the i-th eqn. by −1.

◮ Create new variables z1, . . . , zm ≥ 0 where m = # of equations. ◮ Add zi to the left-hand side of equation i. ◮ min : z1 + · · · + zm is the objective function. ◮ z1 = b1, . . . , zm = bm and all other variables = 0 is a starting vertex.

Case: Simplex reports 0 is the optimal value of the objective function. Then ignore the zi’s and the value of the other vars gives a starting vertex. Case: Simplex reports 0 is not the opt. value of the objective function. Then the original LP problem must be infeasible. (Why?)

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 12 / 18

Simplex Issues: Degeneracy

x1 x3 x2

1

• 4
• 2
• 3
• 5
• 6
• 7
• A

B C

max x1 + 6x2 + 13x3 x1 ≤ 200

1

• x2 ≤ 300

2

• x1 + x2 + x3 ≤ 400

3

• x2 + 3x3 ≤ 600

4

• x1 ≥ 0

5

• x2 ≥ 0

6

• x3 ≥ 0

7

• Vertex B is degenerate in that it is given by any one of:

➁, ➂, ➃ ➁, ➂, ➄ ➁, ➃, ➄ ➂, ➃, ➄

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 13 / 18

Simplex Issues: Degeneracy & Unboundedness

Degeneracy can confuse poor Simplex — causing it to return suboptimal answers. One way out is to perturb the bi’s by a small amount, i.e., bi → bi ± ǫi. This jolts each plan a bit and splits degenerate vertices into multiple verts.

Unboundedness

Simplex can detect if the feasible region is unbounded in which case it halts and fusses.

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 14 / 18

Simplex Running Time

With some cleverness (see DPV) each iteration can be gotten down to O(m · n) time, where m = the number of inequalities n = number of variables There are (m+n

n ) many vertices, which is exponential size.

There are cases where Simplex takes exponential time!! In practice, Simplex usually runs much faster than any other LP algorithm — including Karmarkar’s poly-time LP algorithm.

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 15 / 18

Boolean Circuit Evaluation

Boolean Circuit

A directed, acyclic graph of gates of the following sorts: inputs gates: indegree 0, labeled True

• r False

AND gates, OR gates: indegree 2 NOT gates: indegree 1 One of the gates is chosen as the

• utput gate

true AND false OR true NOT NOT OR AND

• utput

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 16 / 18

Boolean Circuit Evaluation as an LP problem

For each gate, create a variable xg constrained: 0 ≤ xg ≤ 1. If g is a True-input gate, add: xg = 1. If g is a False-input gate, add: xg = 0. If g is an Or-gate with inputs from h and h′, add: xg ≥ xh, xg ≥ xh′, xg ≤ xh + xh′ If g is an And-gate with inputs from h and h′, add: xg ≤ xh, xg ≤ xh′, xg ≥ xh + xh′ − 1 If g is a Not-gate with input from h, add: xg = 1 − xh

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 17 / 18

Boolean Circuit Evaluation as an LP problem—So?

Boolean Circuit Evaluation is, in a certain sense (see Chapter 8), the most general problem solvable in polytime. I.e., every polytime problem reduces to a Boolean Circuit Evaluation. Boolean Circuit Evaluation reduces to LP.

∴ Every polytime problem reduces to LP!!

Similarly, every polytime problem reduces to a Dynamic Programming problem.

DPV Chapter 7 (Part 2) Linear Programming March 25, 2019 18 / 18