An asynchronous parallel derivative-free algorithm for handling - - PowerPoint PPT Presentation

▶
an asynchronous parallel derivative free algorithm for
SMART_READER_LITE
LIVE PREVIEW

An asynchronous parallel derivative-free algorithm for handling - - PowerPoint PPT Presentation

Feb 13, 2024 131 likes •808 views

An asynchronous parallel derivative-free algorithm for handling general constraints Josh Griffin Computational Sciences and Mathematics Research Sandia National Laboratories Livermore, California USA Second International Congress on

slide-1
SLIDE 1

An asynchronous parallel derivative-free algorithm for handling general constraints

Josh Griffin

Computational Sciences and Mathematics Research Sandia National Laboratories Livermore, California USA Second International Congress on Mathematical Software Castro Urdiales, SPAIN September 1–3, 2006 Joint work with Tammy Kolda, Robert Michael Lewis, and Virginia Torczon

Computational Sciences and Mathematics Research Slide 1 September 2, 2006

slide-2
SLIDE 2

Talk outline

  • 1. Problems of interest
  • 2. Generating set search background
  • 3. Linear constraints
  • 4. Nonlinear equality constraints
  • 5. Numerical results

Computational Sciences and Mathematics Research Slide 2 September 2, 2006

slide-3
SLIDE 3

Why use derivative-free?

Punchline Derivative-free methods more reliable, less restrictive Should I take the

  • r the

?

Derivative-based if ...

  • Function evaluations quick
  • All points finite/defined
  • Continous and smooth
  • Little to no noise

Derivative-free if ...

  • Function evaluations slow
  • Points may be undefined
  • Discontinous, nonsmooth, okay
  • Noise okay

Computational Sciences and Mathematics Research Slide 3 September 2, 2006

slide-4
SLIDE 4

Problems we are interested in

  • Function evaluations are CPU-intensive
  • Simulation-based objective function can periodically crash
  • Noise limits ability to estimate derivatives
  • No analytic formula for objective function

Computational Sciences and Mathematics Research Slide 4 September 2, 2006

slide-5
SLIDE 5

Optimization in nuclear safety studies

  • 1. Question: Could accidental drop jeopardize

integrity of internal components?

  • 2. Plan: Drop model from different angles
  • 3. Goal: Find angle that maximizes damage

Resulting Problem: maximize

x∈R2

D(x) subject to 0 ≤ xi ≤ π

  • D(x) is measure of damage
  • Time per evaluation: 1-15 hrs
  • Software entities: 3

Computational Sciences and Mathematics Research Slide 5 September 2, 2006

slide-6
SLIDE 6

Generating Set Search and APPSPACK

Computational Sciences and Mathematics Research Slide 6 September 2, 2006

slide-7
SLIDE 7

Problem types for APPSPACK

minimize

x∈Rn

f(x) subject to c(x) = 0 Ax ≤ b Here f : Rn → R, c : Rn → Rp, and A is an m × n matrix.

  • linear equalities permitted
  • derivatives for f(x) and c(x) unavailable
  • number of variables relatively small (≤ 100)

Computational Sciences and Mathematics Research Slide 7 September 2, 2006

slide-8
SLIDE 8

Generating set search algorithms

General idea: Use set of positively spanning search directions Two examples: Guaranteed: Search direction within 90◦ of steepest descent direction Punchline: Always have a descent direction if one exists

Computational Sciences and Mathematics Research Slide 8 September 2, 2006

slide-9
SLIDE 9

Generating set search algorithms

General idea: Use set of positively spanning search directions Two examples: Guaranteed: Search direction within 90◦ of steepest descent direction Punchline: Always have a descent direction if one exists

Computational Sciences and Mathematics Research Slide 8 September 2, 2006

slide-10
SLIDE 10

Generating set search algorithms

General idea: Use set of positively spanning search directions Two examples: Guaranteed: Search direction within 90◦ of steepest descent direction Punchline: Always have a descent direction if one exists

Computational Sciences and Mathematics Research Slide 8 September 2, 2006

slide-11
SLIDE 11

Synchronous framework (unconstrained)

while (∆ > ∆tol)

  • 1. Generate trial-points and send to evaluation queue:

X = {x + ∆d(i) : d(i) ∈ search pattern}

  • 2. Collect evaluated trial points: Y = X
  • 3. Update: Is there a point y ∈ Y better than x?

Yes: x ← y (successful) No: ∆ ← .5∆ (unsuccessful) end

Computational Sciences and Mathematics Research Slide 9 September 2, 2006

slide-12
SLIDE 12

Synchronous framework (unconstrained)

while (∆ > ∆tol)

  • 1. Generate trial-points and send to evaluation queue:

X = {x + ∆d(i) : d(i) ∈ search pattern}

  • 2. Collect evaluated trial points: Y = X
  • 3. Update: Is there a point y ∈ Y better than x?

Yes: x ← y (successful) No: ∆ ← .5∆ (unsuccessful) end ❅ ❅ ❅ ■

We enforce a sufficient decrease conditions based on step size ∆ f(y) ≤ f(x) − α∆2

Computational Sciences and Mathematics Research Slide 9 September 2, 2006

slide-13
SLIDE 13

Synchronous framework (unconstrained)

while (∆ > ∆tol)

  • 1. Generate trial-points and send to evaluation queue:

X = {x + ∆d(i) : d(i) ∈ search pattern}

  • 2. Collect evaluated trial points: Y = X
  • 3. Update: Is there a point y ∈ Y better than x?

Yes: x ← y (successful) No: ∆ ← .5∆ (unsuccessful) end

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

✛

Step where asynchronous algorithms wins in parallel Y = X

Computational Sciences and Mathematics Research Slide 9 September 2, 2006

slide-14
SLIDE 14

Asynchronous framework (unconstrained)

while (∆(i) > ∆tol)

  • 1. Generate trial-points and send to evaluation queue:

X = {x + ∆(i)d(i) : d(i) ∈ search pattern and inactive}

  • 2. Collect nonempty set of evaluated trial points: Y = X
  • 3. Update: Is there a point y ∈ Y better than x?

Yes: x ← y, ∆(i) = max(∆y, ∆min) Successful: may prune queue No: ∆(i) ← .5∆(i) for ”evaluated” indices Unsuccessful: may not prune queue

Computational Sciences and Mathematics Research Slide 10 September 2, 2006

slide-15
SLIDE 15

Asynchronous framework (unconstrained)

while (∆(i) > ∆tol)

  • 1. Generate trial-points and send to evaluation queue:

X = {x + ∆(i)d(i) : d(i) ∈ search pattern and inactive}

  • 2. Collect nonempty set of evaluated trial points: Y = X
  • 3. Update: Is there a point y ∈ Y better than x?

Yes: x ← y, ∆(i) = max(∆y, ∆min) Successful: may prune queue No: ∆(i) ← .5∆(i) for ”evaluated” indices Unsuccessful: may not prune queue

Computational Sciences and Mathematics Research Slide 11 September 2, 2006

slide-16
SLIDE 16

Asynchronous framework (unconstrained)

while (∆(i) > ∆tol)

  • 1. Generate trial-points and send to evaluation queue:

X = {x + ∆(i)d(i) : d(i) ∈ search pattern and inactive}

  • 2. Collect nonempty set of evaluated trial points: Y = X
  • 3. Update: Is there a point y ∈ Y better than x?

Yes: x ← y, ∆(i) = max(∆y, ∆min) Successful: may prune queue No: ∆(i) ← .5∆(i) for ”evaluated” indices Unsuccessful: may not prune queue

Computational Sciences and Mathematics Research Slide 12 September 2, 2006

slide-17
SLIDE 17

Unconstrained optimization demo

best: a pending: b c d e evaluated: pruned: Trial points ✄ ✄ ✄ ✄ ✄ ✄ ✄ ✄ ✎ ❆ ❆ ❆ ❆ ❯ Current best point

  • ✠
  • ∆

Step size ❅ ❅ ■

Computational Sciences and Mathematics Research Slide 13 September 2, 2006

slide-18
SLIDE 18

Unconstrained optimization demo

best: a pending: b c d e evaluated: pruned:

Computational Sciences and Mathematics Research Slide 14 September 2, 2006

slide-19
SLIDE 19

Unconstrained optimization demo

best: a pending: c d evaluated: b e pruned:

Computational Sciences and Mathematics Research Slide 15 September 2, 2006

slide-20
SLIDE 20

Unconstrained optimization demo

best: a pending: f g c d evaluated: pruned:

Computational Sciences and Mathematics Research Slide 16 September 2, 2006

slide-21
SLIDE 21

Unconstrained optimization demo

best: a pending: c d evaluated: f g pruned:

Computational Sciences and Mathematics Research Slide 17 September 2, 2006

slide-22
SLIDE 22

Unconstrained optimization demo

best: f pending: h i j k c d evaluated: pruned:

Computational Sciences and Mathematics Research Slide 18 September 2, 2006

slide-23
SLIDE 23

Unconstrained optimization demo

best: f pending: i k evaluated: c j h pruned: d ①

Computational Sciences and Mathematics Research Slide 19 September 2, 2006

slide-24
SLIDE 24

Unconstrained optimization demo

best: c pending: l m n o i k evaluated: pruned:

Computational Sciences and Mathematics Research Slide 20 September 2, 2006

slide-25
SLIDE 25

Unconstrained optimization demo

best: c pending: n k evaluated: l m o i pruned:

Computational Sciences and Mathematics Research Slide 21 September 2, 2006

slide-26
SLIDE 26

Unconstrained optimization demo

best: l pending: p q r s n k evaluated: pruned:

Computational Sciences and Mathematics Research Slide 22 September 2, 2006

slide-27
SLIDE 27

Unconstrained optimization demo

best: l pending: p q r s evaluated: n k pruned:

Computational Sciences and Mathematics Research Slide 23 September 2, 2006

slide-28
SLIDE 28

Unconstrained optimization demo

best: l pending: p q r s evaluated: pruned:

Computational Sciences and Mathematics Research Slide 24 September 2, 2006

slide-29
SLIDE 29

Handling linear constraints:

Same algorithm, different directions

Computational Sciences and Mathematics Research Slide 25 September 2, 2006

slide-30
SLIDE 30

Computing conforming search directions

  • ❅

❅ ❅ ❅ ❅ ❅

  • ❅

❅ ❅ ❅ ❅ ❅

  • ❅

❅ ❅ ❅ ❅ ❅

  • ❅

❅ ❅ ❅ ❅ ❅

∗ ∗

✻ ✲ ❄ ✛

  • ✒

❅ ❅ ❘ ❅ ❅ ■ ✏ ✏ ✏ ✏ ✏ ✮ ✏ ✏ ✏ ✏ ✏ ✮

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

−∇f(x)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

−∇f(x)

Computational Sciences and Mathematics Research Slide 26 September 2, 2006

slide-31
SLIDE 31

Locally conforming directions

We want the ability to move parallel to active constraints

Computational Sciences and Mathematics Research Slide 27 September 2, 2006

slide-32
SLIDE 32

Locally conforming directions

We want the ability to move parallel to active constraints We also want the ability to move parallel to “nearby” constraints

Computational Sciences and Mathematics Research Slide 27 September 2, 2006

slide-33
SLIDE 33

ǫ-active constraints

We place a ball of radius ǫ about current best point. Constraints passing through this ǫ-ball are considered ǫ-active constraints.

ǫ

Computational Sciences and Mathematics Research Slide 28 September 2, 2006

slide-34
SLIDE 34

ǫ-active constraints

We place a ball of radius ǫ about current best point. Constraints passing through this ǫ-ball are considered ǫ-active constraints.

ǫ

ǫ-active constraints

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

❘ ✒

Computational Sciences and Mathematics Research Slide 28 September 2, 2006

slide-35
SLIDE 35

Conforming directions

We then compute corresponding conforming search directions

Computational Sciences and Mathematics Research Slide 29 September 2, 2006

slide-36
SLIDE 36

ǫ-tangent cone, T (x, ǫ)

The positive-span of conforming directions forms ǫ-tangent cone We will denote ǫ-tangent cone by T (x, ǫ)

Computational Sciences and Mathematics Research Slide 30 September 2, 2006

slide-37
SLIDE 37

Summarizing

Punch-line: We can always travel a distance of at least ǫ along each conforming search direction and remain feasible. Thus it makes sense to set ǫ equal to the current step size: ǫ = ∆. In asynchronous mode we have multiple step size: ∆(i), i = 1, ..., p. Implies we must work with multiple tangent cones.

Computational Sciences and Mathematics Research Slide 31 September 2, 2006

slide-38
SLIDE 38

Computing conforming search directions

Two-step process:

  • 1. Determine ǫ-active constraints normals
  • Positive-span form ǫ-normal cone N(x, ǫ)
  • Always a subset of rows from constraint matrix
  • 2. Find positive-spanning set for T (x, ǫ) = N(x, ǫ)◦
  • nondegenerate case: formed using LAPACK .
  • degenerate case: formed using C-library cddlib :

– Double description method of Motzkin et al. written by Komei Fukuda. Punchline: Conforming search directions are given by generators of T (x, ǫ)

Computational Sciences and Mathematics Research Slide 32 September 2, 2006

slide-39
SLIDE 39

Synchronous framework for linear constraints

while (∆ > ∆tol)

  • 1. Use conforming search directions for ǫ = min(∆, ǫmax).
  • 2. Generate trial-points and send to evaluation queue:

X = {x + ˜ ∆d(i) : d(i) ∈ search pattern}, ˜ ∆ ∈ [0, ∆]

  • 3. Collect evaluated trial points: Y = X
  • 4. Update: Is there a point y ∈ Y better than x?

Yes: x ← y (successful) No: ∆ ← .5∆ (unsuccessful)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

❨

Asymptotically need ǫ = ∆ Hence require ǫmax > ∆tol

Computational Sciences and Mathematics Research Slide 33 September 2, 2006

slide-40
SLIDE 40

Asynchronous tricky

  • Synchronous case:

– one tangent cone per iteration – swap if tangent cone changes

  • Asynchronous case:

– multiple tangent cones per iteration – swap vs. append tangent cone Punchline: Must include conforming search directions for

  • {i: ∆(i)≤ǫmax}

T (x, ∆(i)) ∪ T (x, ǫmax)

Computational Sciences and Mathematics Research Slide 34 September 2, 2006

slide-41
SLIDE 41

Asynchronous framework for linear constraints

For simplicity assume ǫmax = ∞ while (∆(i) > ∆tol)

  • 1. Generate trial-points and send to evaluation queue:

X = {x + ˜ ∆(i)d(i) : d(i) ∈ search pattern and inactive}

  • 2. Collect nonempty set of evaluated trial points: Y = X
  • 3. Update: Is there a point y ∈ Y better than x?

Yes: x ← y, ∆(i) = max(∆y, ∆min) Use conforming directions for ǫ = current step-size No: ∆(i) ← .5∆(i) for ”evaluated” indices Append conforming directions for ǫ = mini(∆(i))

Computational Sciences and Mathematics Research Slide 35 September 2, 2006

slide-42
SLIDE 42

Linear constrained optimization demo

best: a pending: b c evaluated:

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-43
SLIDE 43

Linear constrained optimization demo

best: a pending: b evaluated: c ①

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-44
SLIDE 44

Linear constrained optimization demo

best: c pending: d e b evaluated:

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-45
SLIDE 45

Linear constrained optimization demo

best: c pending: e b evaluated: d ①

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-46
SLIDE 46

Linear constrained optimization demo

best: c pending: f e b evaluated:

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-47
SLIDE 47

Linear constrained optimization demo

best: c pending: e evaluated: f b ① ①

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-48
SLIDE 48

Linear constrained optimization demo

best: b pending: g h e evaluated:

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-49
SLIDE 49

Linear constrained optimization demo

best: b pending: h evaluated: g e ① ①

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-50
SLIDE 50

Linear constrained optimization demo

best: b pending: i j k h evaluated:

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-51
SLIDE 51

Linear constrained optimization demo

best: b pending: i j k evaluated: h ①

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-52
SLIDE 52

Linear constrained optimization demo

best: b pending: l i j k evaluated:

Computational Sciences and Mathematics Research Slide 36 September 2, 2006

slide-53
SLIDE 53

Asynchronous convergence theory

A useful measure of optimality χ(x) = max

x+ω∈Ω w≤1

−∇f(x)Tw. Can show that χ(x) ≥ 0, χ(x) is continuous, and χ(x) = 0 iff x is first-order optimal Conn, Gould, Sartenaer, and Toint. (1996) (a) Under assumptions always satisfied before APPSPACK terminates, we can show PT (x, ˆ

∆)(−∇f(x)) ≤ C1 ˆ

∆ χ(x) ≤ C2 ˆ ∆ where ˆ ∆ equals the current maximum step size (b) lim inf ˆ ∆ = 0 (a) and (b) together imply global convergence to a first-order optimal point PT (x, ˆ

∆)(−∇f(x)) denotes projection of −∇f(x) onto local tangent cone T (x, ˆ

∆) C1 and C2 depend on properties of f and A

Computational Sciences and Mathematics Research Slide 37 September 2, 2006

slide-54
SLIDE 54

APPSPACK numerical results

Motivation:

  • Stress test APPSPACK’s new linear constraint capabilities

– CUTEr problem known to be difficult even for derivative-based methods

  • Verify new asynchronous theory numerically

Details:

  • Tested on linearly constrained CUTEr (Constrained and Unconstrained Testing

Environment, revisited) (non-trivial) problems with n ≤ 1000 variables

  • All problems tested asynchronously in parallel on Sandia’s Institutional Computing

Cluster (ICC) – 20 processors for n ≤ 10 – 40 processors for 10 < n ≤ 100 – 60 processors for 100 < n ≤ 1000

Computational Sciences and Mathematics Research Slide 38 September 2, 2006

slide-55
SLIDE 55

Numerical results: accuracy

bogus failed to converge converged

  • rel. err. < 1e−6

10 20 30 40 50 60 70 80 0−10 11−100 101−1,000 Number of problems Number of variables

  • Linearly-constrained CUTEr

test problems

  • Compared accuracy of final

solution with SNOPT

Computational Sciences and Mathematics Research Slide 39 September 2, 2006

slide-56
SLIDE 56

Numerical results: accuracy

Largest problem solved: 505 variables, 1010 simple bounds, and 1008 linear constraints

Computational Sciences and Mathematics Research Slide 40 September 2, 2006

slide-57
SLIDE 57

Synchronous vs asynchronous

9 midrange problems selected 5-15 seconds added randomly 27 comparisons made

Computational Sciences and Mathematics Research Slide 41 September 2, 2006

slide-58
SLIDE 58

Handling nonlinear constraints

A sequence of linearly constrained problems

Computational Sciences and Mathematics Research Slide 42 September 2, 2006

slide-59
SLIDE 59

The subproblem

We solve a series of linearly constrained subproblems for λk, µk fixed: min

x∈Rn

Φk(x) subject to Ax ≤ b where Φk(x)

△

= f(x) + λT

kc(x) +

1 2µk c(x)2 Each subproblem is solved approximately using APPSPACK. Key feature: Algorithm can be shown to be globally convergent to first-order optimal points without accessing/estimating derivatives.

Computational Sciences and Mathematics Research Slide 43 September 2, 2006

slide-60
SLIDE 60

Basic frame work with derivatives

while not converged do Solve subproblem approximately until PTk(−∇xΦk(x)) ≤ Cωk PTk(·) denotes projection onto T (x, ωk). Update λk, µk. if c(xk) ≤ ηk, (infeasibility sufficiently reduced) λk+1 = λk + c(xk)/µk (Hestenes-Powell)

  • therwise µk+1 = τµk. (increase penalty)

end

Conn, Gould, Sartenaer, Toint (1996).

Computational Sciences and Mathematics Research Slide 44 September 2, 2006

slide-61
SLIDE 61

Basic frame work with derivatives

while not converged do Solve subproblem approximately until PTk(−∇xΦk(x)) ≤ Cωk PTk(·) denotes projection onto T (x, ωk). Update λk, µk. if c(xk) ≤ ηk, (infeasibility sufficiently reduced) λk+1 = λk + c(xk)/µk (Hestenes-Powell)

  • therwise µk+1 = τµk. (increase penalty)

end Main problem: no access to first derivatives.

Computational Sciences and Mathematics Research Slide 44 September 2, 2006

slide-62
SLIDE 62

Recall linearly constrained optimization theory

We know that at unsuccessful iterations PT (x, ˆ

∆)(−∇xΦk) ≤ C(Φk, A) ˆ

∆ Recall we need a bound of the form PT (x,ωk)(−∇xΦk) ≤ Cωk where C is independent of k. Dependence on k removed by normalizing wrt λk and 1/µk: choose step tolerance ≤ ωk 1 1 + λk + 1/µk .

Computational Sciences and Mathematics Research Slide 45 September 2, 2006

slide-63
SLIDE 63

Preliminary numerical results

  • Current test suite consists of 18 Hock and Schittkowski CUTEr

problems that have nonlinear equality constraints and ≤ 10 variables

  • Current implementation caches f(x) and c(x)

Stopping criteria: ∆(k,tol) ≤ 10−4 c(x) ≤ 10−4

Computational Sciences and Mathematics Research Slide 46 September 2, 2006

slide-64
SLIDE 64

Conclusions

Computational Sciences and Mathematics Research Slide 47 September 2, 2006

slide-65
SLIDE 65

Conclusions and Summary

  • APPSPACK with linear constraints:

– Globally convergent to a KKT point. – Works well in practice. – Corresponding paper “Asynchronous parallel generating set search for linearly-constrained optimization” submitted to SISC.

  • APPSPACK with general equality constraints:

– Globally convergent to a KKT point. – Software in place; currently fine tuning and debugging. – Stable release (soon) Can download latest stable and developmental version here (LGPL license):

http://software.sandia.gov/appspack

Computational Sciences and Mathematics Research Slide 48 September 2, 2006

slide-66
SLIDE 66

Future work

  • Categorical variables:

minimize

xc∈Ω,xd∈S

f(xc, xd) subject to Ω ⊂ Rn S = red, blue, green, etc.

  • Nonlinear inequality constraints solved with slacks:

minimize

x

f(x) subject to h(x) ≤ 0, c(x) = 0, Ax ≤ b

  • Globalization of APPSPACK
  • Support for oracle points

Computational Sciences and Mathematics Research Slide 49 September 2, 2006

slide-67
SLIDE 67

Future work

  • Categorical variables:

minimize

xc∈Ω,xd∈S

f(xc, xd) subject to Ω ⊂ Rn S = red, blue, green, etc.

  • Nonlinear inequality constraints solved with slacks:

minimize

x,z

f(x) subject to h(x)+z = 0, z ≤ 0 c(x) = 0, Ax ≤ b

  • Globalization of APPSPACK
  • Support for oracle points

Computational Sciences and Mathematics Research Slide 50 September 2, 2006