Certifying Non-negativity with Lasserres Hierarchy and Semidefinite - - PowerPoint PPT Presentation

▶
certifying non negativity with lasserre s hierarchy and
SMART_READER_LITE
LIVE PREVIEW

Certifying Non-negativity with Lasserres Hierarchy and Semidefinite - - PowerPoint PPT Presentation

Nov 29, 2022 857 likes •1.73k views

Certifying Non-negativity with Lasserres Hierarchy and Semidefinite Programming Victor Magron , LAASCNRS 5 March 2019 Faculty of Mechanical Engineering University of Ljubljana Victor Magron Certifying Non-negativity with Lasserres

slide-1
SLIDE 1

Certifying Non-negativity with Lasserre’s Hierarchy and Semidefinite Programming

Victor Magron, LAAS–CNRS

5 March 2019

Faculty of Mechanical Engineering University of Ljubljana

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 0 / 40

slide-2
SLIDE 2

Introduction

VERIFICATION OF NONLINEAR SYSTEMS . . . SAFETY of critical parts for computing physical devices Cars

x

Control Software/Hardware

xi xj

Smart Grids Space Systems . . . CAST AS CERTIFIED OPTIMIZATION SOLVE OFFLINE Input: linear semidefinite polynomial Output: value + numerical/symbolic/formal certificate

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 1 / 40

slide-3
SLIDE 3

SDP for Polynomial Optimization

NP-hard NON CONVEX Problem p⋆ = inf p(x) Theory (Primal) (Dual) inf

  • p dµ

sup λ with µ proba ⇒

INFINITE LP

⇐ with p − λ 0

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 2 / 40

slide-4
SLIDE 4

SDP for Polynomial Optimization

NP-hard NON CONVEX Problem p⋆ = inf p(x) Practice (Primal Relaxation) (Dual Strengthening) moments

  • xα dµ

p − λ = sum of squares finite number ⇒ SDP ⇐ fixed degree LASSERRE’S HIERARCHY of CONVEX PROBLEMS ↑ p∗ [Lasserre/Parrilo 01] degree d n vars Numeric Solvers = ⇒ (n+d

n ) SDP VARIABLES

= ⇒ Approx Certificate

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 2 / 40

slide-5
SLIDE 5

Success Stories: Lasserre’s Hierarchy

MODELING POWER: Cast as ∞-dimensional LP over measures STATIC Polynomial Optimization Optimal Powerflow n ≃ 103 [Josz et al 16] Roundoff Error n ≃ 102 [Magron et al 17] DYNAMICAL Polynomial Optimization Regions of attraction [Henrion et al 14] Reachable sets [Magron et al 17]

!

△

APPROXIMATE OPTIMIZATION BOUNDS!

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 3 / 40

slide-6
SLIDE 6

Success Stories: Certified Optimization

Kepler’s Conjecture(1611)

The max density of sphere packings is π/ √ 18

Flyspeck : Formalizing the proof of Kepler by T.Hales (1994) Verification of thousands of “tight” nonlinear inequalities Seminal Paper:

Hales, Adams, Bauer, Dang, Harrison, Hoang, Kaliszyk, M., Mclaughlin, Nguyen, Nguyen, Nipkow, Obua, Pleso, Rute, Solovyev, Ta, Tran, Trieu, Urban, Vu & Zumkeller, Forum of Mathematics, Pi, 5 2017 ∼ 120 citations

MY CONTRIBUTION: (Non)-Polynomial optimization to verify Flyspeck inequalities

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 4 / 40

slide-7
SLIDE 7

Certification Challenges

APPROXIMATE SOLUTIONS sum of squares of a2 − 2ab + b2? (1.00001a − 0.99998b)2! a2 − 2ab + b2 ≃ (1.00001a − 0.99998b)2 a2 − 2ab + b2 = 1.0000200001a2 − 1.9999799996ab + 0.9999600004b2 ≃ → = ?

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 5 / 40

slide-8
SLIDE 8

Certification Challenges

SCALABILITY [Joswig et al 16] 20000+ terms, d = 39, n = 6

2d2

1d2 2k12 1 k4 5x6 1x13 9 + 46d2 1d2 2k12 1 k4 5x5 1x14 9 + 2d2 1d2 2k11 1 k5 5x5 1x14 9 + 46d2 1d2 2k11 1 k5 5x4 1x15 9 + 11d1d2k13 1 k4 5x6 1x14 9 +

297d1d2k13

1 k4 5x5 1x15 9 + 11d1d2k12 1 k5 5x5 1x15 9 + 297d1d2k12 1 k5 5x4 1x16 9 + 242k14 1 k4 5x5 1x16 9 + 242k13 1 k5 5x4 1x17 9 + 2d3 1d3 2k11 1 k4 5x6 1x11 9 +

46d3

1d3 2k11 1 k4 5x5 1x12 9 + 2d3 1d3 2k10 1 k5 5x5 1x12 9 + 46d3 1d3 2k10 1 k5 5x4 1x13 9 + 6d3 1d2 2k11 1 k4 5x5 1x13 9 + 138d3 1d2 2k11 1 k4 5x4 1x14 9 +

4d3

1d2 2k10 1 k5 5x4 1x14 9 + 92d3 1d2 2k10 1 k5 5x3 1x15 9 + 8d2 1d3 2k11 1 k4 5x6 1x12 9 + 184d2 1d3 2k11 1 k4 5x5 1x13 9 + 6d2 1d3 2k10 1 k5 5x5 1x13 9 +

138d2

1d3 2k10 1 k5 5x4 1x14 9 + 2d2 1d2 2k12 1 k4 5x7 1x11 9 + 73d2 1d2 2k12 1 k4 5x6 1x12 9 + 617d2 1d2 2k12 1 k4 5x5 1x13 9 + 2d2 1d2 2k12 1 k3 5x6 1x13 9 +

46d2

1d2 2k12 1 k3 5x5 1x14 9 + 2d2 1d2 2k11 1 k5 5x6 1x12 9 + 73d2 1d2 2k11 1 k5 5x5 1x13 9 + 617d2 1d2 2k11 1 k5 5x4 1x14 9 + 4d2 1d2 2k11 1 k4 5x5 1x14 9 +

92d2

1d2 2k11 1 k4 5x4 1x15 9 + 2d2 1d2 2k10 1 k5 5x4 1x15 9 + 46d2 1d2 2k10 1 k5 5x3 1x16 9 + 45d2 1d2k12 1 k4 5x5 1x14 9 + 1215d2 1d2k12 1 k4 5x4 1x15 9 +

34d2

1d2k11 1 k5 5x4 1x15 9 + 918d2 1d2k11 1 k5 5x3 1x16 9 + d1d2 2k12 1 k4 5x7 1x12 9 + 91d1d2 2k12 1 k4 5x6 1x13 9 + 1760d1d2 2k12 1 k4 5x5 1x14 9 +

d1d2

2k11 1 k5 5x6 1x13 9 + 80d1d2 2k11 1 k5 5x5 1x14 9 + 1463d1d2 2k11 1 k5 5x4 1x15 9 + 12d1d2k13 1 k4 5x7 1x12 9 + 467d1d2k13 1 k4 5x6 1x13 9 +

3575d1d2k13

1 k4 5x5 1x14 9 + 11d1d2k13 1 k3 5x6 1x14 9 + 297d1d2k13 1 k3 5x5 1x15 9 + 12d1d2k12 1 k5 5x6 1x13 9 + 467d1d2k12 1 k5 5x5 1x14 9 +

3575d1d2k12

1 k5 5x4 1x15 9 + 22d1d2k12 1 k4 5x5 1x15 9 + 594d1d2k12 1 k4 5x4 1x16 9 + 11d1d2k11 1 k5 5x4 1x16 9 + 297d1d2k11 1 k5 5x3 1x17 9 +

1254d1k13

1 k4 5x4 1x16 9 + 1012d1k12 1 k5 5x3 1x17 9 + +43d2k13 1 k4 5x6 1x14 9 + 1834d2k13 1 k4 5x5 1x15 9 + 43d2k12 1 k5 5x5 1x15 9 +

1592d2k12

1 k5 5x4 1x16 9 + 286k14 1 k4 5x6 1x14 9 + 2904k14 1 k4 5x5 1x15 9 + 242k14 1 k3 5x5 1x16 9 + 286k13 1 k5 5x5 1x15 9 + . . .

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 6 / 40

slide-9
SLIDE 9

Certification Challenges

“In theory, theory and practice are the same. In practice, they are different.” - A. Einstein CONVERGENCE RATE THEORY

1

c

√

log STAIRS

c

[Nie-Schweighofer 07]

↑ PRACTICE ?

Scientific challenge: bridge THEORY & PRACTICE gap

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 7 / 40

slide-10
SLIDE 10

Modeling Challenges Cyber-Physical

CONTROL SYSTEMS Vehicules Collisions Fluid mechanics PARTIAL DIFFERENTIAL EQUATIONS MIXING DISCRETE/CONTINUOUS EQUATIONS Discrete xt+1 = f(xt) = ⇒ µT = µ0 + f # µ Continuous ˙ x = f(x) = ⇒ µT = µ0 + div f µ   

Liouville Transport Equation

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 8 / 40

slide-11
SLIDE 11

Modeling Challenges Cyber-Physical

FINITE-PRECISION SOFTWARE/HARDWARE a+ (b+ c) = (a+ b) + c Tuned Precision FPGAs Approx Math Functions Optimize Programs PERFORMANCE

VS

ACCURACY MIXED PRECISION Scalability Loops

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 9 / 40

slide-12
SLIDE 12

What is Semidefinite Optimization?

Linear Programming (LP): min

z

c

⊤z

s.t. A z d .

Linear cost c Linear inequalities “∑i Aij zj di”

Polyhedron

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 10 / 40

slide-13
SLIDE 13

What is Semidefinite Optimization?

Semidefinite Programming (SDP): min

z

c

⊤z

s.t.

∑

i

Fi zi F0 .

Linear cost c Symmetric matrices F0, Fi Linear matrix inequalities “F 0” (F has nonnegative eigenvalues)

Spectrahedron

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 10 / 40

slide-14
SLIDE 14

What is Semidefinite Optimization?

Semidefinite Programming (SDP): min

z

c

⊤z

s.t.

∑

i

Fi zi F0 , A z = d .

Linear cost c Symmetric matrices F0, Fi Linear matrix inequalities “F 0” (F has nonnegative eigenvalues)

Spectrahedron

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 10 / 40

slide-15
SLIDE 15

Applications of SDP

Combinatorial optimization Control theory Matrix completion Unique Games Conjecture (Khot ’02) : “A single concrete algorithm provides optimal guarantees among all efficient algorithms for a large class of computational problems.” (Barak and Steurer survey at ICM’14) Solving polynomial optimization (Lasserre ’01)

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 10 / 40

slide-16
SLIDE 16

Lasserre’s Hierarchy

Prove polynomial inequalities with SDP: f(a, b) := a2 − 2ab + b2 0 . Find z s.t. f(a, b) =

  • a

b z1 z2 z2 z3

  • a

b

  • .

Find z s.t. a2 − 2ab + b2 = z1a2 + 2z2ab + z3b2 (A z = d)

z1 z2 z2 z3

  • =

1

  • F1

z1 + 1 1

  • F2

z2 + 1

  • F3

z3

  • F0

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 11 / 40

slide-17
SLIDE 17

Lasserre’s Hierarchy

Choose a cost c e.g. (1, 0, 1) and solve: min

z

c

⊤z

s.t.

∑

i

Fi zi F0 , A z = d . Solution

z1 z2 z2 z3

  • =

1 −1 −1 1

  • (eigenvalues 0 and 2)

a2 − 2ab + b2 =

  • a

b 1 −1 −1 1

  • a

b

  • = (a − b)2 .

Solving SDP = ⇒ Finding SUMS OF SQUARES certificates

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 12 / 40

slide-18
SLIDE 18

Lasserre’s Hierarchy

NP hard General Problem: f ∗ := min

x∈K f(x)

Semialgebraic set K := {x ∈ Rn : g1(x) 0, . . . , gm(x) 0}

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 13 / 40

slide-19
SLIDE 19

Lasserre’s Hierarchy

NP hard General Problem: f ∗ := min

x∈K f(x)

Semialgebraic set K := {x ∈ Rn : g1(x) 0, . . . , gm(x) 0} := [0, 1]2 = {x ∈ R2 : x1(1 − x1) 0, x2(1 − x2) 0}

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 13 / 40

slide-20
SLIDE 20

Lasserre’s Hierarchy

NP hard General Problem: f ∗ := min

x∈K f(x)

Semialgebraic set K := {x ∈ Rn : g1(x) 0, . . . , gm(x) 0} := [0, 1]2 = {x ∈ R2 : x1(1 − x1) 0, x2(1 − x2) 0}

f

  • x1x2 +1

8 =

σ0

  • 1

2

  • x1 + x2 − 1

2 2 +

σ1

  • 1

2

g1

  • x1(1 − x1) +

σ2

  • 1

2

g2

  • x2(1 − x2)

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 13 / 40

slide-21
SLIDE 21

Lasserre’s Hierarchy

NP hard General Problem: f ∗ := min

x∈K f(x)

Semialgebraic set K := {x ∈ Rn : g1(x) 0, . . . , gm(x) 0} := [0, 1]2 = {x ∈ R2 : x1(1 − x1) 0, x2(1 − x2) 0}

f

  • x1x2 +1

8 =

σ0

  • 1

2

  • x1 + x2 − 1

2 2 +

σ1

  • 1

2

g1

  • x1(1 − x1) +

σ2

  • 1

2

g2

  • x2(1 − x2)

Sums of squares (SOS) σi

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 13 / 40

slide-22
SLIDE 22

Lasserre’s Hierarchy

NP hard General Problem: f ∗ := min

x∈K f(x)

Semialgebraic set K := {x ∈ Rn : g1(x) 0, . . . , gm(x) 0} := [0, 1]2 = {x ∈ R2 : x1(1 − x1) 0, x2(1 − x2) 0}

f

  • x1x2 +1

8 =

σ0

  • 1

2

  • x1 + x2 − 1

2 2 +

σ1

  • 1

2

g1

  • x1(1 − x1) +

σ2

  • 1

2

g2

  • x2(1 − x2)

Sums of squares (SOS) σi Bounded degree: Qd(K) :=

  • σ0 + ∑m

j=1 σjgj, with deg σj gj 2d

  • Victor Magron

Certifying Non-negativity with Lasserre’s Hierarchy and SDP 13 / 40

slide-23
SLIDE 23

Lasserre’s Hierarchy

Hierarchy of SDP relaxations: λd := sup

λ

  • λ : f − λ ∈ Qd(K)
  • Convergence guarantees λd ↑ f ∗ [Lasserre 01]

Can be computed with SDP solvers (CSDP, SDPA) “No Free Lunch” Rule: (n+2d

n ) SDP variables

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 14 / 40

slide-24
SLIDE 24

SDP for Polynomial Optimization Success Stories Challenges SDP for Nonlinear Optimization RealCertify: Certify Non-negativity

slide-25
SLIDE 25

Oranges Stack

Kepler Conjecture (1611): The maximal density of sphere packings in 3D-space is

π √ 18

Face-centered cubic Packing Hexagonal Compact Packing

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 15 / 40

slide-26
SLIDE 26

A “Simple” Example

In the computational part: Multivariate Polynomials:

∆x := x1x4(−x1 + x2 + x3 − x4 + x5 + x6) + x2x5(x1 − x2 + x3 + x4 − x5 + x6) + x3x6(x1 + x2 − x3 + x4 + x5 − x6) − x2(x3x4 + x1x6) − x5(x1x3 + x4x6)

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 16 / 40

slide-27
SLIDE 27

A “Simple” Example

In the computational part: Semialgebraic functions: composition of polynomials with | · |, √, +, −, ×, /, sup, inf, . . . p(x) := ∂4∆x q(x) := 4x1∆x r(x) := p(x)/

  • q(x)

l(x) := −π 2 + 1.6294 − 0.2213 (√x2 + √x3 + √x5 + √x6 − 8.0) + 0.913 (√x4 − 2.52) + 0.728 (√x1 − 2.0)

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 16 / 40

slide-28
SLIDE 28

A “Simple” Example

In the computational part: Transcendental functions T : composition of semialgebraic functions with arctan, exp, sin, +, −, ×, . . .

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 16 / 40

slide-29
SLIDE 29

A “Simple” Example

In the computational part: Feasible set K := [4, 6.3504]3 × [6.3504, 8] × [4, 6.3504]2 Lemma9922699028 from Flyspeck: ∀x ∈ K, arctan p(x)

  • q(x)
  • + l(x) 0

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 16 / 40

slide-30
SLIDE 30

Existing Certified Frameworks

Lemma9922699028 from Flyspeck: ∀x ∈ K, arctan

  • ∂4∆x

√4x1∆x

  • + l(x) 0

Dependency issue using Interval Calculus:

One can bound ∂4∆x/√4x1∆x and l(x) separately Too coarse lower bound: −0.87 Subdivide K to prove the inequality

K = ⇒ K0 K1 K2 K3 K4

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 17 / 40

slide-31
SLIDE 31

Nonlinear Function Representation

Tree representations of multivariate functions: leaves are semialgebraic functions nodes are univariate functions or binary operations

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 18 / 40

slide-32
SLIDE 32

Nonlinear Function Representation

For the “Simple” Example from Flyspeck:

+ l(x) arctan r(x)

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 18 / 40

slide-33
SLIDE 33

Maxplus Optimization Algorithm

First iteration:

+ l(x) arctan r(x) a y par−

a1

arctan m M a1 1 control point {a1}: m1 = −4.7 × 10−3 < 0

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 19 / 40

slide-34
SLIDE 34

Maxplus Optimization Algorithm

Second iteration:

+ l(x) arctan r(x) a y par−

a1

par−

a2

arctan m M a1 a2 2 control points {a1, a2}: m2 = −6.1 × 10−5 < 0

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 19 / 40

slide-35
SLIDE 35

Maxplus Optimization Algorithm

Third iteration:

+ l(x) arctan r(x) a y par−

a1

par−

a2

par−

a3

arctan m M a1 a2 a3 3 control points {a1, a2, a3}: m3 = 4.1 × 10−6 > 0

OK!

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 19 / 40

slide-36
SLIDE 36

Contribution: Publications and Software

M., Allamigeon, Gaubert, Werner. Formal Proofs for Nonlinear Optimization, Journal of Formalized Reasoning 8(1):1–24, 2015. Hales, Adams, Bauer, Dang, Harrison, Hoang, Kaliszyk, M., Mclaughlin, Nguyen, Nguyen, Nipkow, Obua, Pleso, Rute, Solovyev, Ta, Tran, Trieu, Urban, Vu & Zumkeller, Forum of Mathematics, Pi, 5 2017 Software Implementation NLCertify: 15 000 lines of OCAML code 4000 lines of COQ code

  • M. NLCertify: A Tool for Formal Nonlinear Optimization, ICMS,

2014.

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 19 / 40

slide-37
SLIDE 37

Roundoff Error Bounds

Exact: f(x) := x1x2 + x3x4 Floating-point: ˆ f(x, e) := [x1x2(1 + e1) + x3x4(1 + e2)](1 + e3) x ∈ X , | ei | 2−p p = 24 (single) or 53 (double)

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 20 / 40

slide-38
SLIDE 38

Roundoff Error Bounds

Input: exact f(x), floating-point ˆ f(x, e) Output: Bounds for f − ˆ f

1: Error r(x, e) := f(x) − ˆ

f(x, e) = ∑

α

rα(e)xα

2: Decompose r(x, e) = l(x, e) + h(x, e), l linear in e 3: Bound h(x, e) with interval arithmetic 4: Bound l(x, e) with SPARSE SUMS OF SQUARES

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 20 / 40

slide-39
SLIDE 39

Roundoff Error Bounds

Sparse SDP Optimization [Waki, Lasserre 06] Correlative sparsity pattern (csp) of vars x2x5 + x3x6 − x2x3 − x5x6 + x1(−x1 + x2 + x3 − x4 + x5 + x6)

6 4 5 1 2 3

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 20 / 40

slide-40
SLIDE 40

Roundoff Error Bounds

Sparse SDP Optimization [Waki, Lasserre 06] Correlative sparsity pattern (csp) of vars x2x5 + x3x6 − x2x3 − x5x6 + x1(−x1 + x2 + x3 − x4 + x5 + x6)

6 4 5 1 2 3

1 Maximal cliques C1, . . . , Cl 2 Average size κ ❀ (κ+2k κ ) vars

C1 := {1, 4} C2 := {1, 2, 3, 5} C3 := {1, 3, 5, 6} Dense SDP: 210 vars Sparse SDP: 115 vars

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 20 / 40

slide-41
SLIDE 41

Contributions

l(x, e) = ∑m

i=1 si(x)ei

Maximal cliques correspond to {x, e1}, . . . , {x, em}

M., Constantinides, Donaldson. Certified Roundoff Error Bounds Using Semidefinite Programming, Trans. Math. Soft., 2016

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 20 / 40

slide-42
SLIDE 42

Reachable Sets of Polynomial Systems

Iterations xt+1 = f(xt) Uncertain xt+1 = f(xt, u) Converging SDP hierarchies Image measure Liouville equation (conservation) µt + µ = f # µ + µ0

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 21 / 40

slide-43
SLIDE 43

Reachable Sets of Polynomial Systems

Iterations xt+1 = f(xt) Uncertain xt+1 = f(xt, u) Converging SDP hierarchies Image measure Liouville equation (conservation) µt + µ = f # µ + µ0

M., Garoche, Henrion, Thirioux. Semidefinite Approximations of Reachable Sets for Discrete-time Polynomial Systems, 2017.

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 21 / 40

slide-44
SLIDE 44

Invariant Measures of Polynomial Systems

Discrete xt+1 = f(xt) = ⇒ f # µ − µ = 0 Continuous ˙ x = f(x) = ⇒ div f µ = 0 Converging SDP hierarchies measures with density in Lp singular measures = ⇒ chaotic attractors

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 22 / 40

slide-45
SLIDE 45

Invariant Measures of Polynomial Systems

Discrete xt+1 = f(xt) = ⇒ f # µ − µ = 0 Continuous ˙ x = f(x) = ⇒ div f µ = 0 Converging SDP hierarchies measures with density in Lp singular measures = ⇒ chaotic attractors

M., Forets, Henrion. Semidefinite Characterization of Invariant Measures for Polynomial Systems. 2018.

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 22 / 40

slide-46
SLIDE 46

SDP for Polynomial Optimization Success Stories Challenges SDP for Nonlinear Optimization RealCertify: Certify Non-negativity

slide-47
SLIDE 47

RealCertify: Certify Non-negativity

X = (X1, . . . , Xn)

co-NP hard problem: check f 0 on K

f ∈ Q[X]

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 23 / 40

slide-48
SLIDE 48

RealCertify: Certify Non-negativity

X = (X1, . . . , Xn)

co-NP hard problem: check f 0 on K

f ∈ Q[X]

1 Unconstrained K = Rn

n = 1 f = 1 + X + X2 + X3 + X4 n > 1 f = 4X4

1 + 4X3 1X2 − 7X2 1X2 2 − 2X1X3 2 + 10X4 2

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 23 / 40

slide-49
SLIDE 49

RealCertify: Certify Non-negativity

X = (X1, . . . , Xn)

co-NP hard problem: check f 0 on K

f ∈ Q[X]

1 Unconstrained K = Rn

n = 1 f = 1 + X + X2 + X3 + X4 n > 1 f = 4X4

1 + 4X3 1X2 − 7X2 1X2 2 − 2X1X3 2 + 10X4 2 2 Constrained K = {x ∈ Rn : g1(x) 0, . . . , gm(x) 0}

gj ∈ Q[X] f = −X2

1 − 2X1X2 − 2X2 2 + 6

K = {1 − X2

1 0, 1 − X2 2 0}

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 23 / 40

slide-50
SLIDE 50

RealCertify: Certify Non-negativity

X = (X1, . . . , Xn)

co-NP hard problem: check f 0 on K

f ∈ Q[X]

1 Unconstrained K = Rn

n = 1 f = 1 + X + X2 + X3 + X4 n > 1 f = 4X4

1 + 4X3 1X2 − 7X2 1X2 2 − 2X1X3 2 + 10X4 2 2 Constrained K = {x ∈ Rn : g1(x) 0, . . . , gm(x) 0}

gj ∈ Q[X] f = −X2

1 − 2X1X2 − 2X2 2 + 6

K = {1 − X2

1 0, 1 − X2 2 0} 1 f ∈ Σ = sums of squares (SOS)

f = σ = h12 + · · · + hp2 0

2 Weighted SOS

f = σ0 + σ1 g1 + · · · + σm gm 0 on K

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 23 / 40

slide-51
SLIDE 51

From Approximate to Exact Solutions

APPROXIMATE SOLUTIONS sum of squares of a2 − 2ab + b2? (1.00001a − 0.99998b)2! a2 − 2ab + b2 ≃ (1.00001a − 0.99998b)2 a2 − 2ab + b2 = 1.0000200001a2 − 1.9999799996ab + 0.9999600004b2 ≃ → = ?

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 24 / 40

slide-52
SLIDE 52

From Approximate to Exact Solutions

Win TWO-PLAYER GAME

Σ f

sum of squares of f? ≃ Output!

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 25 / 40

slide-53
SLIDE 53

From Approximate to Exact Solutions

Win TWO-PLAYER GAME

Σ f

Hybrid Symbolic/Numeric Algorithms sum of squares of f + ε? ≃ Output! Error Compensation ≃ → =

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 25 / 40

slide-54
SLIDE 54

Rational SOS Decompositions

Let f ∈ R[X] and f 0 on R (n = 1) Theorem There exist f1, f2 ∈ R[X] s.t. f = f12 + f22.

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 26 / 40

slide-55
SLIDE 55

Rational SOS Decompositions

Let f ∈ R[X] and f 0 on R (n = 1) Theorem There exist f1, f2 ∈ R[X] s.t. f = f12 + f22.

Proof.

f = h2(q + ir)(q − ir)

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 26 / 40

slide-56
SLIDE 56

Rational SOS Decompositions

Let f ∈ R[X] and f 0 on R (n = 1) Theorem There exist f1, f2 ∈ R[X] s.t. f = f12 + f22.

Proof.

f = h2(q + ir)(q − ir) Examples

1 + X + X2 =

  • X + 1

2 2 + √ 3 2 2 1 + X + X2 + X3 + X4 =

  • X2 + 1

2X + 1 + √ 5 4 2 + 10 + 2 √ 5 +

  • 10 − 2

√ 5 4 X +

  • 10 − 2

√ 5 4 2

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 26 / 40

slide-57
SLIDE 57

Rational SOS Decompositions

f ∈ Q[X] ∩ ˚ Σ[X] (interior of the SOS cone) Existence Question Does there exist fi ∈ Q[X], ci ∈ Q>0 s.t. f = ∑i ci fi2?

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 27 / 40

slide-58
SLIDE 58

Rational SOS Decompositions

f ∈ Q[X] ∩ ˚ Σ[X] (interior of the SOS cone) Existence Question Does there exist fi ∈ Q[X], ci ∈ Q>0 s.t. f = ∑i ci fi2? Examples

1 + X + X2 =

  • X + 1

2 2 + √ 3 2 2 = 1

  • X + 1

2 2 + 3 4 (1)2 1 + X + X2 + X3 + X4 =

  • X2 + 1

2 X + 1 + √ 5 4 2 + 10 + 2 √ 5 +

  • 10 − 2

√ 5 4 X +

  • 10 − 2

√ 5 4 2 = ???

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 27 / 40

slide-59
SLIDE 59

Existing Frameworks

project & round [Peyrl-Parrilo 08] [Kaltofen-Yang-Zhi 08] f ∈ ˚ Σ[X] with deg f = 2D f(X) ≃ vDT(X) ˜ Q vD(X) ˜ Q ≻ 0 vD(X): vector of monomials of deg D Find ˜ Q with semidefinite programming f(X) = vDT(X) ∏(Q) vD(X)

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 28 / 40

slide-60
SLIDE 60

Existing Frameworks

project & round [Peyrl-Parrilo 08] [Kaltofen-Yang-Zhi 08] f ∈ ˚ Σ[X] with deg f = 2D f(X) ≃ vDT(X) ˜ Q vD(X) ˜ Q ≻ 0 vD(X): vector of monomials of deg D Find ˜ Q with semidefinite programming f(X) = vDT(X) ∏(Q) vD(X) RAGLib (critical points) [Safey El Din] SamplePoints (CAD) [Moreno Maza-Alvandi et al.]

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 28 / 40

slide-61
SLIDE 61

One Answer when K = {x ∈ Rn : gj(x) 0}

Hybrid SYMBOLIC/NUMERIC methods Magron-Allamigeon-Gaubert-Werner 14 f ≃ ˜ σ0 + ˜ σ1 g1 + · · · + ˜ σm gm u = f − ˜ σ0 + ˜ σ1 g1 + · · · + ˜ σm gm

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 29 / 40

slide-62
SLIDE 62

One Answer when K = {x ∈ Rn : gj(x) 0}

Hybrid SYMBOLIC/NUMERIC methods Magron-Allamigeon-Gaubert-Werner 14 f ≃ ˜ σ0 + ˜ σ1 g1 + · · · + ˜ σm gm u = f − ˜ σ0 + ˜ σ1 g1 + · · · + ˜ σm gm ≃ → = ∀x ∈ [0, 1]n, u(x) −ε minK f ε when ε → 0 COMPLEXITY? Compact K ⊆ [0, 1]n

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 29 / 40

slide-63
SLIDE 63

Modules & Install

gricad-gitlab:RealCertify Depends on Maple & univsos n = 1 Square free decomposition with sqrfree PARI/GP for complex zero isolation multivsos n > 1 arbitrary precision SDP solver SDPA-GMP [Nakata 10] Newton Polytope with convex package [Franz 99] Cholesky’s decomposition with LUDecomposition

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 30 / 40

slide-64
SLIDE 64

intsos with n 1: Perturbation

Σ f

PERTURBATION idea Approximate SOS Decomposition f(X) - ε ∑α∈P/2 X2α = ˜ σ + u

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 31 / 40

slide-65
SLIDE 65

intsos with n = 1 [Chevillard et. al 11]

p ∈ Q[X], deg p = d = 2k, p > 0

x p p = 1 + X + X2 + X3 + X4

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 32 / 40

slide-66
SLIDE 66

intsos with n = 1 [Chevillard et. al 11]

p ∈ Q[X], deg p = d = 2k, p > 0 PERTURB: find ε ∈ Q s.t. pε := p − ε

k

∑

i=0

X2i > 0

x p

1 4(1 + x2 + x4)

pε p = 1 + X + X2 + X3 + X4 ε = 1 4 p > 1 4 (1 + X2 + X4)

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 32 / 40

slide-67
SLIDE 67

intsos with n = 1 [Chevillard et. al 11]

p ∈ Q[X], deg p = d = 2k, p > 0 PERTURB: find ε ∈ Q s.t. pε := p − ε

k

∑

i=0

X2i > 0 SDP Approximation: p − ε

k

∑

i=0

X2i = ˜ σ + u ABSORB: small enough ui = ⇒ ε ∑k

i=0 X2i + u SOS x p

1 4(1 + x2 + x4)

pε p = 1 + X + X2 + X3 + X4 ε = 1 4 p > 1 4 (1 + X2 + X4)

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 32 / 40

slide-68
SLIDE 68

intsos with n = 1 and SDP Approximation

Input: f 0 ∈ Q[X] of degree d 2, ε ∈ Q>0, δ ∈ N>0 Output: SOS decomposition with coefficients in Q

pε ←p − ε

k

∑

i=0

X2i ε ← ε 2 ˜ σ ←sdp(pε, δ) u ←pε − ˜ σ δ ←2δ f ˜ σ, ε, u while pε ≤ 0 while ε < |u2i+1| + |u2i−1| 2 − u2i

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 33 / 40

slide-69
SLIDE 69

intsos with n = 1: Absorbtion

X = 1

2

(X + 1)2 − 1 − X2 −X = 1

2

(X − 1)2 − 1 − X2

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 34 / 40

slide-70
SLIDE 70

intsos with n = 1: Absorbtion

X = 1

2

(X + 1)2 − 1 − X2 −X = 1

2

(X − 1)2 − 1 − X2 u2i+1X2i+1 = |u2i+1| 2 (Xi+1 + sgn (u2i+1)Xi)2 − X2i − X2i+2

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 34 / 40

slide-71
SLIDE 71

intsos with n = 1: Absorbtion

X = 1

2

(X + 1)2 − 1 − X2 −X = 1

2

(X − 1)2 − 1 − X2 u2i+1X2i+1 = |u2i+1| 2 (Xi+1 + sgn (u2i+1)Xi)2 − X2i − X2i+2

u ε ∑k

i=0 X2i

· · · 2i − 2 2i − 1 2i 2i + 1 2i + 2 · · · ε ε ε

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 34 / 40

slide-72
SLIDE 72

intsos with n = 1: Absorbtion

X = 1

2

(X + 1)2 − 1 − X2 −X = 1

2

(X − 1)2 − 1 − X2 u2i+1X2i+1 = |u2i+1| 2 (Xi+1 + sgn (u2i+1)Xi)2 − X2i − X2i+2

u ε ∑k

i=0 X2i

· · · 2i − 2 2i − 1 2i 2i + 1 2i + 2 · · · ε ε ε

ε |u2i+1| + |u2i−1| 2 − u2i = ⇒ ε

k

∑

i=0

X2i + u SOS

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 34 / 40

slide-73
SLIDE 73

intsos with n 1: Absorbtion f(X) - ε ∑α∈P/2 X2α = ˜ σ + u Choice of P?

x y 1 2 3 4 5 1 2 3 4 5 6 u1,3 ε ε xy3 = 1

2(x + y3)2 − x2+y6 2

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 35 / 40

slide-74
SLIDE 74

intsos with n 1: Absorbtion f(X) - ε ∑α∈P/2 X2α = ˜ σ + u Choice of P?

x y 1 2 3 4 5 1 2 3 4 5 6 u1,3 ε ε xy3 = 1

2(xy + y2)2 − x2y2+y4 2

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 35 / 40

slide-75
SLIDE 75

intsos with n 1: Absorbtion f(X) - ε ∑α∈P/2 X2α = ˜ σ + u Choice of P?

x y 1 2 3 4 5 1 2 3 4 5 6 u1,3 ε ε xy3 = 1

2(xy2 + y)2 − x2y4+y2 2

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 35 / 40

slide-76
SLIDE 76

intsos with n 1: Absorbtion f(X) - ε ∑α∈P/2 X2α = ˜ σ + u Choice of P?

f = 4x4y6 + x2 − xy2 + y2 spt(f) = {(4, 6), (2, 0), (1, 2), (0, 2)} Newton Polytope P = conv (spt(f)) Squares in SOS decomposition ⊆ P

2 ∩ Nn

[Reznick 78]

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 35 / 40

slide-77
SLIDE 77

Algorithm intsos

Input: f ∈ Q[X] ∩ ˚ Σ[X] of degree d, ε ∈ Q>0, δ ∈ N>0 Output: SOS decomposition with coefficients in Q

fε ← f − ε ∑

α∈P/2

X2α ε ← ε 2 ˜ σ ←sdp( fε, δ) u ← fε − ˜ σ δ ←2δ P ← conv (spt( f )) f ˜ σ, ε, u while fε ≤ 0 while u + ε ∑

α∈P/2

X2α / ∈ Σ

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 36 / 40

slide-78
SLIDE 78

Algorithm Putinarsos

Assumption: ∃i s.t. gi = 1 − X2

2

f > 0 on K := {x ∈ Rn : gj(x) 0} has Putinar’s representation: f = σ0 + ∑

j

σj gj with σj ∈ Σ[X] , deg σj 2D

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 37 / 40

slide-79
SLIDE 79

Algorithm Putinarsos

Assumption: ∃i s.t. gi = 1 − X2

2

f > 0 on K := {x ∈ Rn : gj(x) 0} has Putinar’s representation: f = σ0 + ∑

j

σj gj with σj ∈ Σ[X] , deg σj 2D Theorem [M.-Safey El Din 18] f = ˚ σ0 + ∑

j

˚ σj gj with ˚ σj ∈ ˚ Σ[X] , deg ˚ σj 2D

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 37 / 40

slide-80
SLIDE 80

Algorithm Putinarsos

Assumption: ∃i s.t. gi = 1 − X2

2

f > 0 on K := {x ∈ Rn : gj(x) 0} has Putinar’s representation: f = σ0 + ∑

j

σj gj with σj ∈ Σ[X] , deg σj 2D Theorem [M.-Safey El Din 18] f = ˚ σ0 + ∑

j

˚ σj gj with ˚ σj ∈ ˚ Σ[X] , deg ˚ σj 2D ABSORBTION as in Algorithm intsos: u = fε − ˜ σ0 − ∑j ˜ σj gj

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 37 / 40

slide-81
SLIDE 81

Unconstrained Benchmarks

Id n d multivsos RoundProject RAGLib CAD τ1 (bits) t1 (s) τ2 (bits) t2 (s) t3 (s) t4 (s) f20 2 20 745 419 110. 78 949 497 141. 0.16 0.03 M 3 8 17 232 0.35 18 831 0.29 0.15 0.03 f2 2 4 1 866 0.03 1 031 0.04 0.09 0.01 f6 6 4 56 890 0.34 475 359 0.54 598. − f10 10 4 344 347 2.45 8 374 082 4.59 − −

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 38 / 40

slide-82
SLIDE 82

Constrained Benchmarks

Id n d multivsos RAGLib CAD D τ1 (bits) t1 (s) t2 (s) t3 (s) f260 6 3 2 114 642 2.72 4.19 − f491 6 3 2 108 359 9.65 0.01 0.05 f752 6 2 2 10 204 0.26 0.07 − f859 6 7 4 6 355 724 303. 0.05 − f863 4 2 1 5 492 0.14 0.01 0.01 f884 4 4 3 300 784 25.1 113. − butcher 6 3 2 247 623 1.32 231. − heart 8 4 2 618 847 2.94 24.7 −

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 39 / 40

slide-83
SLIDE 83

Conclusion and Perspectives

Input f on K with deg f = d and bit size τ

Algo Input K OUTPUT BIT SIZE intsos ˚ Σ Rn τ dO (n) Putinarsos > 0 {x ∈ Rn : gj(x) 0} O (2τ dn CK ) compact

How to handle degenerate situations?

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 40 / 40

slide-84
SLIDE 84

Conclusion and Perspectives

Input f on K with deg f = d and bit size τ

Algo Input K OUTPUT BIT SIZE intsos ˚ Σ Rn τ dO (n) Putinarsos > 0 {x ∈ Rn : gj(x) 0} O (2τ dn CK ) compact

How to handle degenerate situations? Better arbitrary-precision SDP solvers Extension to other relaxations, sums of hermitian squares Crucial need for polynomial systems certification Available PhD/Postdoc Positions

Victor Magron Certifying Non-negativity with Lasserre’s Hierarchy and SDP 40 / 40

slide-85
SLIDE 85

End

Thank you for your attention! gricad-gitlab:RealCertify https://homepages.laas.fr/vmagron

Magron, Safey El Din & Schweighofer. Algorithms for Weighted Sums of Squares Decomposition of Non-negative Univariate Polynomials, JSC. arxiv:1706.03941 Magron & Safey El Din. On Exact Polya and Putinar’s Representations, ISSAC’18. arxiv:1802.10339 Magron & Safey El Din. RealCertify: a Maple package for certifying non-negativity, ISSAC’18. arxiv:1805.02201