Real Parameterized and 2 Real Parameterized and 2nd
nd
Order Complexity Theory: Order Complexity Theory: From Computability in Analysis From Computability in Analysis to Numerical Practice to Numerical Practice
Martin Ziegler Martin Ziegler
Real Parameterized and 2 Order Complexity Theory: Order Complexity - - PowerPoint PPT Presentation
Real Parameterized and 2 nd nd Real Parameterized and 2 Order Complexity Theory: Order Complexity Theory: From Computability in Analysis From Computability in Analysis to Numerical Practice to Numerical Practice Martin Ziegler Martin
nd
Martin Ziegler Martin Ziegler
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
Folklore: Folklore: Folklore: Folklore: For x∈ the following are equivalent: a) x has a decidable binary expansion b) x has a recursive signed-digit expansion c) There exists a recursive sequence (an) ⊆ s.t. | x – an/2n+1 | ≤ 2-n d) There exist recursive sequences (pn),(qn)⊆ s.t. supn pn = x = infn qn
i) only uniformly ii) no running time bound
Folklore: Folklore: Folklore: Folklore: Every computable f:[0;1]→ with f(0)·f(1)<0 has a computable root.
numerics / iRRAM
Obstacles to practice:
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
Function f:[0,1]→ computable computable if some TM can, on input of n∈ and of
=:ρ.name ≡ ρsd
p p
in time in time t
a) + +, , × ×, , exp
b) f
n∈ ∈L L 4
n iff L
*
decidable in time in time t
i) If If ƒ
computable ⇒ ⇒ continuous. continuous. ii ii) ) If If f
computable in in time time t
, then then O O(
is a a modulus modulus of uniform
continuity of
.
polytime polytime. c) 1/
c) sign
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
n |
q( (n n) ) :
TM
decides set set L
if
inputs x
prints 1 1 and and terminates terminates, ,
inputs x
prints 0 0 and and terminates terminates. .
runs in in polynom
. time if if ∃
[
input x
n makes
makes at at most most p
steps / / uses uses at at most most p
bits of
memory. . / /space space
decidable in in polynomial polynomial time time }
in in polynomial polynomial time time }
decidable in in polyn polyn. . space space }
decidable in exponential time in exponential time }
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
!
polytime.computable iff
x x ƒ
in general no computable solution z
for ƒ∈
1
for ƒ∈
k
even even when when restricting restricting to to ƒ∈
∞
[Friedman&Ko'82] [Friedman&Ko'82] [Kawamura'10, [Kawamura'10, Kawamura Kawamura et al] et al]
another class between and
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
∞ function
L:[0,1]
L|
[0,t t] ] again
0.2 0.4 0.6 0.8 1
0.5 1
〈N,M〉
N=1
N=2
N=3
N=4
N=5
M=0
M=0
M=0,1,2
M=0..3
t=1 t=½
t=⅓
t=¼
M=1
tln(1/t)
N
∈V
φ φ( (t t) = ) = exp( exp(-
t² ²/1 /1-
t² ²) )
C C∞
∞ ' 'pulse'
pulse' function function
polytime polytime computable computable
∞
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
%
polytime.computable iff
x x ƒ
in general no computable solution z
for ƒ∈
1
for ƒ∈
k
even when restricting to ƒ∈
∞
non. . uniform uniform [ [Friedman&Ko Friedman&Ko] ]
#
[Kawamura'10, Kawamura Kawamura et al] et al] [ [N.M N.Mü üller ller] ]
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
a) natural emergence of multivaluedness (aka non.extensionality) b) Uniform computation may require
― which yield canonical C++ declarations c) Parameterized uniform upper complexity bounds (as well.established in Discrete Complexity)
that numerical scientists might be interested in / should be aware of
(Brattka&Z, ) Finding an eigenvector (basis) to a given real symmetric d×d matrix A is computable; becomes computable when knowing Card σ(A).
→ ε.semantics of "<"
but +, exp computable in time polynomial
in n on [0;1];
independent of independent of x
x
dom dom
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
a) A partial F:⊆{0,1}ω→{0,1}ω is computable computable in in time time t:→
τ=F(σ)
b) For spaces X,Y equipped with representations α,β, (multivalued partial) f:⊆X ⇒Y is (
(α α, ,β β) )–
–computable computable in time in time t
t
if it admits an (α,β)–realizer F computable in time t. c) A parameter parameter to a space X with representation α is a mapping k:dom(α)→. d) For X,Y spaces with representations α,β and parameters
k,ℓ,
it admits an (α,β)–realizer F and a polynomial p such that a Type.2 machine can compute F on inputs σ within
p(n+k(σ)) steps
) Some computably reasonable (e.g. admissible) representations induce trivial notions of complexity. [Weihrauch'03] and [Schröder'04] have devised (meta.) conditions on representations to avoid such degeneracies. call f as above fully fully polytime polytime (
(α α, ,k k, ,β β, ,ℓ ℓ) )–
–computable computable if if a Type.2 machine can convert σ∈dom(F) to s.t. the n.th symbol of τ appears within t(n) steps. call and ℓ◦F≤p◦k holds.
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
access
Evaluation Eval:(f,x)→f(x) a) requires ≥µ(n) steps, µ:→ mod. of continuity to f. "Parameter" "Parameter" µ
µ( (f f) ) is
is not not . .valued valued but but
.valued valued! ! b) Even restricted to the compact domain 1 := { f:[0;1]→[0;1] 1.Lipschitz } there exists no representation δ:⊆{0,1}ω→1 rendering Eval computable in exponential time.
Parameter to representation α: a mapping k:dom(α)→. Representations α,β and parameters k,ℓ: (α,β)–realizer F required computable on inputs σ within poly(n+k(σ)) steps.
2-n
≈2n-1 'hats'
≥22.1 functions pairwise differing when evaluating up to error 2. but only 2() different initial segments of δ.names that can be read within t(n) steps. q.e.d.
?
(Arzela.Ascoli)
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
access
Evaluation Eval:(f,x)→f(x) a) requires ≥µ(n) steps, µ:→ mod. of continuity to f. Parameter Parameter µ
µ( (f f) ) is
is not not . .valued valued but but
.valued valued! ! b) Even restricted to the compact domain 1 := { f:[0;1]→[0;1] 1.Lipschitz } (Arzela.Ascoli) there exists no representation δ:⊆{0,1}ω→1 rendering Eval computable in exponential time. Kawamura&Cook'10 (based on Cook&Kapron'96): Kawamura&Cook'10 (based on Cook&Kapron'96): Remedy Remedy to a): to a): . .order
complexity theory theory Remedy Remedy to b): to b): . .order
representations
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
*# &$
Evaluation Eval:(f,x)→f(x) a) requires ≥µ(n) steps, µ:→ mod. of continuity to f. Parameter Parameter µ
µ( (f f) ) is
is not not . .valued valued but but
.valued valued! !
A second second. .order
polynomial P
P( (n n, ,λ λ) ) is
is a a term term built built from from
, ×
×, integer constants and (first
, integer constants and (first. .order) variable
n (rang
(rang. . ing ing over
) ) and second and second. .order variable
λ (ranging
(ranging over
).
a) Second.order polynomials are closed under kinds of composition
and (Q▫P)(n,λ) := Q(n,P(·,λ)) b) For λ∈[n], P(n,λ) is an ordinary polynomial. λ³(λ(n²)·n+λ²(n))+n17
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
| |ψ ψ| |(
(n n) )
:=| :=|ψ ψ( (1 1n
n)|
)|
*#
Recall that ({0,1}*){0,1}* denotes the set { ψ:{0,1}* →{0,1}* } For F:⊆({0,1}*){0,1}*→({0,1}*){0,1}*, oracle Turing machine ? computes computes F if ψ on input v∈{0,1}* outputs w=F(ψ)(v). Call ψ∈({0,1}*){0,1}* length length. .monotone monotone if |ψ(v)|≤|ψ(w)| ∀|v|≤|w|. ? runs in 2 2nd
nd.
.order
polytime if, for some 2nd.order poly. nomial P, ψ on input v∈{0,1}* makes ≤P(|v|,|ψ|) steps.
A second second. .order
polynomial P
P( (n n, ,λ λ) ) is
is a a term term built built from from
, ×
×, integer constants and (first
, integer constants and (first. .order) variable
n (rang
(rang. . ing ing over
) ) and second and second. .order variable
λ (ranging
(ranging over
). {0,1}{0,1}* ∋ Q can be computed in 2nd.order polytime by a non.determin.
→ ( {0,1}* ∋ v → ∃u∈{0,1}|v|: Q〈v,u〉 )
but by a deterministic one.
provably
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
{ length.monotone ψ }
access
A second second. .order
representation of X is a surjective partial mapping ∆:⊆LM→X
A ( (R R, ,Γ Γ) ). .realizer realizer of f:⊆X⇒Y is a mapping F:LM→LM s.t.…
Even on compact 1 = { f:[0;1]→[0;1] 1.Lipschitz } there is no representation δ:⊆{0,1}ω→1 rendering Eval computable in subexponential time.
For F:⊆({0,1}*){0,1}*→({0,1}*){0,1}*, oracle Turing machine ? computes computes F if ψ on input v∈{0,1}* outputs w=F(ψ)(v). Call ψ∈({0,1}*){0,1}* length length. .monotone monotone if |ψ(v)|≤|ψ(w)| ∀|v|≤|w|. ? runs in 2 2nd
nd.
.order
polytime if, for some 2nd.order poly. nomial P, ψ on input v∈{0,1}* makes ≤P(|v|,|ψ|) steps.
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
+#
a) An ordinary representation δ:⊆{0,1}ω→X induces induces a 2nd.order representation ∆ where ψ:{0,1}*→{0,1} is a ∆.name of x∈X iff
b) a ρ ρ
.
.name name of f∈C[0;1] as a ψ∈LM s.t.
A 2 2nd
nd.
.order
representation of X is a surjective ∆:⊆LM→X Call ψ∈({0,1}*){0,1}* length length. .monotone monotone if |ψ(v)|≤|ψ(w)| ∀|v|≤|w|. ? runs in 2 2nd
nd.
.order
polytime if, for some 2nd.order poly. nomial P, ψ on input v∈{0,1}* makes ≤P(|v|,|ψ|) steps.
, , a) a) Polytime Polytime δ
.computability computability is is uniformly uniformly equivalent equivalent to 2 to 2nd
nd.
.order
polytime ∆
.computability computability b) Evaluation b) Evaluation (
is (
×
.computable computable. .
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
b) Evaluation b) Evaluation (
is (
×
.computable computable. .
and µ a modulus
+#
b) a ρ ρ
.
.name name of f∈C[0;1] as a ψ∈LM s.t.
c) a ρ ρ
⊓
⊓Lip Lip. .name name of f∈Lip2ℓ[0;1] as
for a ρ.name ψ of f. d) A [ [ρ→ρ ρ→ρ] ]. .name name of f∈C[0;1]
for ρ.name ψ of f.
? runs in 2 2nd
nd.
.order
polytime if, for some 2nd.order poly. nomial P, ψ on input v∈{0,1}* makes ≤P(|v|,|ψ|) steps.
, , c) Evaluation on c) Evaluation on Lip[0;1]
2 2nd
nd.
.order
polytime polytime ( (ρ ρ
⊓
⊓Lip Lip×Ρ ×Ρ, ,Ρ Ρ) ). .computable computable d) and 2 d) and 2nd
nd.
.order
polytime ([ ([ρ→ρ ρ→ρ] ]×Ρ ×Ρ, ,Ρ Ρ) ). . computable computable on
.
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
in (LiCS'03)
pp.3.21 in ! " vol./0 (2003).
pp.443.459 in ! " vol.1 (2004).
#$%&' (FOCS'05).
Functions with Absolute Error", pp.246.249 in (%! (ASCM 2005)
)% (CCA'07), vol.++ (2008)
Dimensional Plane", pp.121.135 in *vol.++ (2008)
pp.207.217 in +') %(CCA 2008), ENTCS vol.++
$')% (CCA 2009)
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
Functions", pp.155.164 in ',- ./ %/ , Vieweg+Teubner (2009).
Polynomial.Space Complete", pp.305.332 in ) vol.0+ (2010)
Polynomial Time over Unbounded Domains", pp.170.181 in 0$ !&' (MFCS'2011), Springer LNCS vol.203
vol./+ (2012), article 5.
Smooth Differential Equations", pp.578.589 in 0( !&'(MFCS'2012), Springer LNCS vol.3/2/
pp.1459.1477 in ! vol.4+4 (2012).
pp.1.11 in 1' (CiE'2013).
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
+ hardware support / large data / high.dim matrices − − − − heuristics, ad.hoc approaches, unspecified class of permitted inputs, non.guaranteed behavior, vague/ inconsistent semantics, various notions of error, not closed under composition, empirical "proofs" of correctness & performance, const..factor acceleration
nag_opt_one_var_deriv nag_opt_one_var_deriv (e04bbc) (e04bbc) normally normally computes computes a a sequence sequence of x
values which which tend tend in in the the limit limit to a to a minimum minimum of F x
subject to to the the given given bounds bounds
"The iterative methods used to solve problems of nonlinear "The iterative methods used to solve problems of nonlinear programming differ according to whether they evaluate programming differ according to whether they evaluate Hessians, gradients, or only function values. While evaluating Hessians, gradients, or only function values. While evaluating Hessians and gradients improves the rate of convergence, Hessians and gradients improves the rate of convergence, such evaluations increase the such evaluations increase the computational complexity computational complexity (or computational cost) of each iteration. In some cases, (or computational cost) of each iteration. In some cases, the the computational complexity computational complexity may be excessively high." may be excessively high."
Wikipedia Wikipedia
Real Parameterized and 2nd-Order Complexity Theory: From Computability in Analysis to Numerical Practice
TECHNISCHE UNIVERSITÄT DARMSTADT, Martin Ziegler
$ 6$ 6$ ! !
guaranteed error bounds
high accuracy test of conjectures in classical analysis
fully specified algorithms with runtime bounds consistent semantics closed under composition
modular software development of certified libraries
concepts such as multivaluedness and enrichment/information theory (TTE)
canonical interface declaration of implementation
Alan Turing Alan Turing / / also a also a Numerical Numerical Scientist! Scientist! Let's collaborate with, and approach, e.g. the Let's collaborate with, and approach, e.g. the
and Computer
community
Bloch, Feigenbaum, Bremble Bloch, Feigenbaum, Bremble. .Hilbert Hilbert
Kreinovich,Yap Kreinovich,Yap Revol, Plum, Revol, Plum, v.Gudenberg? v.Gudenberg?