Verified error b oun d s f or multipl e roots o f syst e ms o f - - PowerPoint PPT Presentation
Verified error b oun d s f or multipl e roots o f syst e ms o f nonlin ea r e qu a tions St ef Gr a ill a t LIP6/P E QU A N , Sor b onn e Univ e rsit s , UPM C Univ P a ris 06 , C NRS Joint work with Si e g f ri ed M. Rump Journ e s m tho de
Stef Graillat
LIP6/PEQUAN, Sorbonne Universités, UPMC Univ Paris 06, CNRS
Joint work with Siegfried M. Rump
Journées méthodes de subdivisions pour les systèmes singuliers IRCCyN, Nantes, December 15-16, 2014
Verified error bounds for multiple roots 1 / 42
Verify assumptions of mathematical theorems on the computer Making mathematical proofs with computers Getting verified results : → an interval enclosure of the true result → an approximate result with a rigorous error bound Possibly with proof of uniqueness Being fast and accurate Dealing with “ill-posed problems”
Verified error bounds for multiple roots 2 / 42
Proofs with computers: how to do that ? with computer algebra systems: exact results but sometimes not efficient with floating-numbers: fast but ofen wrong results due to rounding errors Possible solution: computing with floating-point but taking into account all the rounding errors !
Verified error bounds for multiple roots 3 / 42
1
Principle of self-validating methods
2
Multiple roots of polynomial systems
3
Numerical experiments
Verified error bounds for multiple roots 4 / 42
1
Principle of self-validating methods
2
Multiple roots of polynomial systems
3
Numerical experiments
Verified error bounds for multiple roots 5 / 42
Let A be a matrix and R another matrix such that ∥I − RA∥ < 1. Ten A is nonsingular
By contrapositive, if A is singular, there exists x ≠ 0 such that Ax = 0. Ten (I − RA)x = x and so ∥I − RA∥ ≥ 1. On a computer, choose for R ≈ A−1 and then compute ∥I − RA∥ with interval arithmetic.
Verified error bounds for multiple roots 6 / 42
Let A be a matrix of dimension n R = inv(A) C = eye(n) - R*intval(A) nonsingular = ( norm(C,1) < 1 ) If nonsingular = 1, then A is nonsingular. If nonsingular = 0, then we can say nothing
Verified error bounds for multiple roots 7 / 42
Let f ∶ Rn → Rn and ̂ x ∈ Rn unknown such that f (̂ x) = 0 Let ̃ x ≈ ̂ x such that f (̃ x) ≈ 0 Find a bound for ̃ x : an interval X such that ̂ x ∈ X We have f (x) = 0 ⇔ g(x) = x with g(x) ∶= x − R f (x) with det(R) ≠ 0.
Every continuous function from a closed ball of a Euclidean space to itself has a fixed point.
Verified error bounds for multiple roots 8 / 42
By Brouwer fixed point theorem, X ∈ IRn, g(X) ⊆ X ⇒ ∃̂ x ∈ X, g(̂ x) = ̂ x ⇒ f (̂ x) = 0 We just have to check g(X) ⊆ X and prove det(R) ≠ 0. But naive approach fails: g(X) ⊆ X − R f (X) ⊈ X
Verified error bounds for multiple roots 9 / 42
Mean Value Teorem : if f ∈ C1 then f (x) = f (̃ x) + M(x − ̃ x) with M = ( ∂ f
∂x(ξi))i
Let Y ∶= X − ̃ x and x ∈ X ⇒ g(x) − ̃ x = x − ̃ x − R f (x) = −R f (̃ x) + (I − RM)(x − ̃ x) ∈ −R f (̃ x) + (I − RM)Y As a consequence −R f (̃ x) + (I − RM)Y ⊆ Y ⇒ g(X) − ̃ x ⊆ Y ⇒ g(X) ⊆ X
Verified error bounds for multiple roots 10 / 42
Let f ∶ Rn → Rn with f = (f1, . . . , fn) ∈ C1, ˜ x ∈ Rn, X ∈ IRn with 0 ∈ X and R ∈ Rn×n be given. Let M ∈ IRn×n be given such that {∇fi(ζ) ∶ ζ ∈ ̃ x + X} ⊆ Mi,∶ . Denote by I the n × n identity matrix and assume −R f (̃ x) + (I − RM)X ⊆ int(X). Ten there is a unique ̂ x ∈ ̃ x + X with f (̂ x) = 0. Moreover, every matrix ̃ M ∈ M is nonsingular. In particular, the Jacobian J f (̂ x) = ∂ f
∂x(̂
x) is nonsingular.
Verified error bounds for multiple roots 11 / 42
Note that an inclusion of the range of the gradients ∇fi over the set ̃ x + X needs to be computed. A convenient way to do this in INTLAB is by interval arithmetic and the gradient toolbox. For a given (Matlab) function f, for xs = ̃ x and an interval vector X, the call M = f(gradientinit(xs + X)) computes an inclusion M.
Verified error bounds for multiple roots 12 / 42
1
Principle of self-validating methods
2
Multiple roots of polynomial systems
3
Numerical experiments
Verified error bounds for multiple roots 13 / 42
Verification method for computing guaranteed (real or complex) error bounds for double roots of systems of nonlinear equations. To circumvent the principle problem of ill-posedness we prove that a slightly perturbed system of nonlinear equations has a double root. For example, for a given univariate function f ∶ R → R we compute two intervals X, E ⊆ R with the property that there exists ̂ x ∈ X and ̂ e ∈ E such that ̂ x is a double root of f (x) ∶= f (x) − ̂ e. If the function f has a double root, typically the interval E is a very narrow interval around zero.
Verified error bounds for multiple roots 14 / 42
Te typical scenario in the univariate case is a function f ∶ R → R with a double root ̂ x, i.e. f (̂ x) = f ′(̂ x) = 0 and f ′′(̂ x) ≠ 0. Consider, for example,
f (x) = 18x7 − 183x6 + 764x5 − 1675x4 + 2040x3 − 1336x2 + 416x − 48 = (3x − 1)2(2x − 3)(x − 2)4
Verified error bounds for multiple roots 15 / 42
Verification methods for multiple roots of polynomials already exist (Rump,2003). A set containing k roots of a polynomial is computed, but no information on the true multiplicity can be given. A hybrid algorithm based on the methods of (Rump,2003) is implemented in algorithm verifypoly in INTLAB. Computing inclusions X1, X2 and X3 of the simple root x1 = 1.5, the double root x2 = 1/3 and the quadruple root x3 = 2 of f by algorithm verifypoly in INTLAB we obtain the following.
>> X1 = verifypoly(f,1.3), X2 = verifypoly(f,.3), X3 = verifypoly(f,2.1) intval X1 = [ 1.49999999999904, 1.50000000000078] intval X2 = [ 0.33333316656015, 0.33333343640539] intval X3 = [ 1.99741678159164, 2.00363593397305]
Verified error bounds for multiple roots 16 / 42
Te accuracy of the inclusion of the double root x2 = 1/3 is much less than that of the simple root x1 = 1.5, and this is typical. If we perturb f into ̃ f (x) ∶= f (x) − ε for some small real constant ε and look at a perturbed root ̃ f (̂ x + h) of ̃ f , then 0 = ̃ f (̂ x + h) = −ε + 1 2 f ′′(̂ x)h2 + O(h3) implies h ∼ √ 2ε/f ′′(̂ x). In general floating-point computations are afflicted with a relative error of size ε ≈ 10−16. Tis has the same effect as a perturbation of the given function f into ̃ f . But for double roots, we cannot expect this inclusion to be of better relative accuracy than √ε ≈ 10−8.
Verified error bounds for multiple roots 17 / 42
We consider for a double root the nonlinear system G ∶ R2 → R with G(x, e) = ( f (x) − e f ′(x) ) = 0 in the two unknowns x and e. Te Jacobian of this system is JG(x, e) = ( f ′(x) −1 f ′′(x) 0 ) , so that the nonlinear system is well-conditioned for the double root x2 = 1/3 of f .
Verified error bounds for multiple roots 18 / 42
Now we can apply a verification algorithm for solving general systems of nonlinear equation such as algorithm verifynlss in
>> Y2 = verifynlss(G,[.3;0]) intval Y2 = [ 3.333333333333328e-001, 3.333333333333337e-001] [ -2.131628207280424e-014, 2.131628207280420e-014] Tis proves that there is a constant ε with ∣ε∣ ≤ 2.14 ⋅ 10−14 such that the nonlinear equation f (x) − ε = 0 has a double root ̂ x with 0.3333333333333328 ≤ ̂ x ≤ 0.3333333333333337.
Verified error bounds for multiple roots 19 / 42
We presented the previous approach in preparation for the multivariate case; However, for univariate nonlinear functions we may proceed more directly. Suppose X ∈ IR is an inclusion of a root ̂ x of f ′, and use the interval evaluation of f at X to compute E ∈ IR with f (X) ⊆ E. In particular f (̂ x) ∈ E, so that there exists ̂ e ∈ E such that the function g(x) ∶= f (x) − ̂ e satisfies g(̂ x) = g′(̂ x) = 0. If, moreover, the inclusion X is computed by a verification method, then ̂ x is a unique root of f ′ in X, and ̂ x is proved to be a double root of g.
Verified error bounds for multiple roots 20 / 42
By this approach we obtain the inclusions for the double root ̂ x are of the same quality, but the inclusion for the shif is a little weaker than in Y2: intval X = [ 3.333333333333329e-001, 3.333333333333339e-001] intval E = [ -3.126388037344441e-013, 2.913225216616412e-013]
Verified error bounds for multiple roots 21 / 42
However, it is superior to expand f with respect to some point m ∈ X. For all x ∈ X we have f (x) ∈ f (m) + f ′(X)(X − m) =∶ E1, and in particular f (̂ x) ∈ E1. Here m should be close to the midpoint of X, but need not to be equal to the midpoint. In this case we obtain with intval E1 = [ -2.131628207280369e-014, 2.131628207280378e-014] an inclusion of the same quality as Y2 by solving G. Note that we use only a univariate verification method to include a root
Verified error bounds for multiple roots 22 / 42
Let a suitably smooth function f ∶ Rn → Rn and ̂ x ∈ Rn be given such that f (̂ x) = 0 and the Jacobian of f at ̂ x is singular. A standard verification method such as verifynlss must fail because with an inclusion of a root the nonsingularity of the Jacobian at the root is proved as well. Again it is an ill-posed problem and we need some regularization technique.
Verified error bounds for multiple roots 23 / 42
Consider the model problem
f (x, y) = ( f1(x, y) f2(x, y) ) = ( x2 + (x + 1)(y − 1)2 − asinh((x + 3)3 + y2)cos(x − xy) (x + 1.908718874061618)2 − sin(x)(y + 1)2 Figure : Contour lines of f1(x) = 0 (solid) and f2(x) = 0 (dashed)
Verified error bounds for multiple roots 24 / 42
As a regularization we add, similar to the univariate case, a smoothing parameter e and rewrite into F(x, y, e) = ⎛ ⎜ ⎝ f1(x, y) − e f2(x, y) detJ f (x, y) ⎞ ⎟ ⎠ = 0 . Te third equation forces the tangents of the zero contour lines to be parallel at the solution, whereas the first equation introduces a perturbation to f1 so that the root becomes a double root. Tis approach may work for two or three unknowns, however, an explicit formula for the determinant of the Jacobian is prohibitive for larger
singular.
Verified error bounds for multiple roots 25 / 42
Let a function f = (f1, . . . , fn) ∶ Rn → Rn be given and let ̂ x = (̂ x1, . . . , ̂ xn) be such that f (̂ x) = 0 and the Jacobian J f (̂ x) of f at ̂ x is
and g(x, e) = ⎛ ⎜ ⎜ ⎜ ⎝ f1(x) − e f2(x) ⋯ fn(x) ⎞ ⎟ ⎟ ⎟ ⎠ = 0 at n equations in n +1 unknowns. We force the Jacobian to be singular by J f (x)y = 0 for some vector y in the kernel of J f . In order to make y unique we normalize some component of y to 1.
Verified error bounds for multiple roots 26 / 42
Let f = (f1, . . . , fn) ∶ Rn → Rn with f ∈ C2 be given. Define F ∶ R2n → R2n by F(x, e, y) = ( g(x, e) J f (x)y ) = 0 , where x = (x1, . . . , xn), e ∈ R and y = (1, y2, . . . , yn). Suppose F suitable assumptions and yields inclusions for ̂ x ∈ Rn, ̂ e ∈ R and ̂ y ∈ Rn−1 such that F(̂ x, ̂ e, ̂ y) = 0. Ten g(̂ x, ̂ e) = f (̂ x) − (̂ e, 0, . . . , 0)T = 0, and the rank of the Jacobian J f (̂ x) of f at ̂ x is n − 1.
Verified error bounds for multiple roots 27 / 42
Te system f (x1, x2) = ( x2
1 − x2 2
x1 − x2
2
) = 0 yields JF(x, e, y) = ⎛ ⎜ ⎜ ⎜ ⎝ 2x1 −2x2 −1 1 −2x2 0 −2 1 2y −2 0 2x1 ⎞ ⎟ ⎟ ⎟ ⎠ , as the Jacobian of the augmented system, which is nonsingular for x1 = x2 = 0. Tus an inclusion is in principle possible.
Verified error bounds for multiple roots 28 / 42
>> f=inline(’[x(1)^2-x(2)^2;x(1)-x(2)^2]’), verifynlss2(f,[0.002;0.001]) f = Inline function: f(x) = [x(1)^2-x(2)^2;x(1)-x(2)^2] intval ans = 1.0e-323 * [
0.66666666666666] [
1.00000000000000] [
1.00000000000000]
Verified error bounds for multiple roots 29 / 42
Computing eigenvalues can be viewed as solving the nonlinear system: f (x, λ) = ( Ax − λx eT
k x − 1 ) = 0 ,
As before we regularize the system, but now not by shifing a whole partial function but by changing an individual component ai j of A: g(x, λ, ε, y) = ⎛ ⎜ ⎝ Ax − λx − εxjei eT
k x − 1
J f (x, λ)y ⎞ ⎟ ⎠ = 0 . Again an inclusion is calculated. In this case, the rank of the Jacobian J f (x, λ) = ( A − λI −x eT
k
0 ) is proved to be n and we can also prove that the eigenvalue is of geometric multiplicity one.
Verified error bounds for multiple roots 30 / 42
1
Principle of self-validating methods
2
Multiple roots of polynomial systems
3
Numerical experiments
Verified error bounds for multiple roots 31 / 42
Consider f (x) = (sin(x) − 1)(x − α) for α ∶= π 2 (1 + ε) . Te function f has a double root ̂ x = π/2 with another simple root α of relative distance ε to π/2. Hence we expect the inclusion E of the offset e for regularization to be a narrow inclusion of zero.
ε X E 10−2 1.5707963267949 ± 1.8 ⋅ 10−14 [−3.5, 1.8] ⋅ 10−18 10−3 1.5707963267948 ± 1.7 ⋅ 10−13 [−3.5, 1.8] ⋅ 10−19 10−4 1.570796326795 ± 1.6 ⋅ 10−12 [−3.5, 1.8] ⋅ 10−20 10−5 1.57079632679 ± 1.2 ⋅ 10−10 [−3.5, 1.8] ⋅ 10−21 10−6 1.5707963268 ± 1.5 ⋅ 10−9 [−3.5, 1.8] ⋅ 10−22 10−7 1.570796327 ± 1.6 ⋅ 10−8 [−3.5, 1.8] ⋅ 10−23 10−8 failed Table : Inclusions for the double root ̂
x = π/2 and a nearby simple root α for f
Verified error bounds for multiple roots 32 / 42
Consider now f (x) = (sin(x) − 1)(x − α)2 for α ∶= π 2 (1 + ε) , so that there is a double root α near the double root ̂
distance ε of about
4
√ε ∼ 10−4 the four roots behave like a quadruple root. Tis is confirmed by the results in the Table.
ε X E 10−2 1.57079632679488 ± 1.2 ⋅ 10−14 [−2.8, 5.5] ⋅ 10−20 10−3 1.5707963267948 ± 2.4 ⋅ 10−13 [−2.8, 5.5] ⋅ 10−22 10−4 1.570796326794 ± 2.8 ⋅ 10−12 [−2.8, 5.5] ⋅ 10−24 10−5 failed Table : Inclusions for the double root ̂
x = π/2 and a nearby double root α for f
Verified error bounds for multiple roots 33 / 42
Te first test function is f (x1, x2) = ( ex1x2 − sin(x2
1 − 2x1x2)
x1(x1 − cosh(x2)) + x1atan(x2) − α ) = 0 , where we choose the parameter α such that the system has a nearly double root. For example, for α = 0.4 the zero contour lines look like in Figure.
Verified error bounds for multiple roots 34 / 42
Figure : Zero contour lines of f (x1, x2) for two different parameter values α. X1 X2 X E 1.32889962186
28
1.3288995157
48
1.328899568390716
5
−0.0272980567
59
−0.0272979298
88
−0.0272979927587941
34
[-5.2,-5.0]⋅10−14 Table : Inclusions X1, X2 for two single roots and X for a nearly double root for f and
α = 0.4003120447407.
Verified error bounds for multiple roots 35 / 42
X1 X2 X E −0.2919733091
44
−0.291973361
57
−0.2919733331276441
29
1.195005123
00
1.195004869
53
1.195004985750992
87
[-1.17,-0.96]⋅10−14 Table : Inclusions X1, X2 for two single roots and X for a nearly double root for f and
α = 0.35653033083794.
Verified error bounds for multiple roots 36 / 42
Consider Brown’s almost linear function f ∶ Rn → Rn with fk(x) = xk +
n
∑
j=1
xj − (n + 1) for 1 ≤ k ≤ n − 1 , fn(x) = (
n
∏
j=1
xj) − 1 − e , where the last function is shifed by some e. One verifies that for e = (1 − 1 n2)
n−1
(1 + 1 n) − 1 and xk = 1 − 1
n2
for 1 ≤ k ≤ n − 1 , xn = 1 + 1
n
the vector (1, . . . , 1, −n) is in the kernel of the Jacobian of f .
Verified error bounds for multiple roots 37 / 42
Tus x is not a simple root of f . More precisely it is verified that there exists ̂ x ∈ X and ̂ ε ∈ E such that f (̂ x) − (̂ ε, . . . , 0) = 0 and the Jacobian J f (̂ x) of f at ̂ x is singular. n X1⋯n−1 Xn E 10 0.990000 ± 1.0 ⋅ 10−14 1.100000 ± 1 ⋅ 10−14 [−3.5, 5.8] ⋅ 10−15 20 0.997500 ± 4.0 ⋅ 10−14 1.050000 ± 1 ⋅ 10−14 [−1.4, 2.2] ⋅ 10−14 50 0.996000 ± 2.1 ⋅ 10−13 1.020000 ± 2 ⋅ 10−14 [−0.1, 1.9] ⋅ 10−13 100 0.999900 ± 8.2 ⋅ 10−13 1.010000 ± 2 ⋅ 10−14 [−5.4, 2.9] ⋅ 10−13 200 0.999975 ± 3.3 ⋅ 10−12 1.005000 ± 5 ⋅ 10−14 [−1.3, 2.0] ⋅ 10−12 500 0.999996 ± 1.9 ⋅ 10−11 1.002000 ± 1 ⋅ 10−13 [−0.6, 1.3] ⋅ 10−11 1000 0.999999 ± 7.5 ⋅ 10−11 1.001000 ± 2 ⋅ 10−13 [−1.1, 6.4] ⋅ 10−11
Table : Inclusions of a double root for different dimensions.
Verified error bounds for multiple roots 38 / 42
Conclusion: Efficient algorithms for computing verified and narrow error bounds with the property that a slightly perturbed system is proved to have a double root within the computed bounds Applied those to univariate polynomials, to multivariate polynomials and also to eigenvalue problems Numerical experiments have confirmed the performance of our algorithms Future work: Detecting singular matrices Applications to approximate coprimeness
Verified error bounds for multiple roots 39 / 42
Siegfried M. Rump and Stef Graillat. Verified error bounds for multiple roots of systems of nonlinear equations.
Siegfried M. Rump. Verification methods: Rigorous results using floating-point arithmetic. Acta Numerica (2010), pp. 287-449. Bo Einarsson. Accuracy and Reliability in Scientific Computing. Sofware-Environments-Tools. SIAM, Philadelphia, PA, 2005.
Verified error bounds for multiple roots 40 / 42
Nicholas J. Higham. Accuracy and stability of numerical algorithms. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, second edition, 2002. Jean-Michel Muller et al. Handbook of Floating-Point Arithmetic. Birkhäuser, 2010.
Introduction to Interval Analysis. SIAM, 2009.
Verified error bounds for multiple roots 41 / 42
Verified error bounds for multiple roots 42 / 42