Why dont we move slower? On the cost of time in the neural control - - PowerPoint PPT Presentation

Why don’t we move slower?

On the cost of time in the neural control of movement Fr´ ed´ eric Jean1 Bastien Berret2

1ENSTA ParisTech

(and Team GECO, INRIA Saclay)

2University Paris Sud

IHP, November 26, 2014

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 1 / 27

Warning This is not a non-holonomic talk. . . not even a mathematical one. Application of optimal control to the modelling of human motor control.

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 2 / 27

Cost of time in human motions

Outline

1

Cost of time in human motions

2

Recovering g

3

Inverse optimal control

4

Experimental results

5

From self paced to slow/fast motions

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 3 / 27

Cost of time in human motions

What is the duration of a natural movement? (Natural = realize one task, without constraint of time, precision,. . . ) Usual theory: duration results from the minimization of a compromise between the cost of the time and the cost of the motion.

linear quadratic hyperbolic exponential - exponential +

Duration [s] Costs [a.u.] Cost of time [a.u.]

a b

?

0.25 0.5 0.75 1 1.25 1.5 1.75 2 5 10 15 20 25

Duration [s]

total cost (effort + time) effort cost time cost 0.25 0.5 0.75 1 1.25 1.5 1.75 2 5 10 15 20 25 30 35 40

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 4 / 27

Cost of time in human motions

Cost of the motion (in fixed time): solution of an inverse optimal control problem, large literature in the case of the arm [Flash, Shadmehr, Berret, . . . ] Cost of the time: lot of different modelling,

psychologists → hyperbolic costs (concave functions) economists, behaviourists → exponential costs (convex functions)

Only interpretations/explanations, no quantitative results.

linear quadratic hyperbolic exponential - exponential +

Duration [s] Costs [a.u.] Cost of time [a.u.]

a b

?

0.25 0.5 0.75 1 1.25 1.5 1.75 2 5 10 15 20 25

Duration [s]

total cost (effort + time) effort cost time cost 0.25 0.5 0.75 1 1.25 1.5 1.75 2 5 10 15 20 25 30 35 40

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 5 / 27

Cost of time in human motions

Modelling

Dynamics of the motion: ˙ x = f(x, u) Paradigm Any registered trajectory x(·) from x0 to xf is an optimal solution of min

u

tu (g(t) + L(xu(t), u(t))) dt, among all u(·) defined on [0, tu] s.t. xu(0) = x0, xu(tu) = xf. T g(t)dt : cost of the time T T L(xu, u) : cost of the motion in fixed time T

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 6 / 27

Cost of time in human motions

Modelling

Dynamics of the motion: ˙ x = f(x, u) Paradigm Any registered trajectory x(·) from x0 to xf is an optimal solution of min

u

tu (g(t) + L(xu(t), u(t))) dt, among all u(·) defined on [0, tu] s.t. xu(0) = x0, xu(tu) = xf. T g(t)dt : cost of the time T ← what we are looking for! T L(xu, u) : cost of the motion in fixed time T

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 6 / 27

Recovering g

Outline

1

Cost of time in human motions

2

Recovering g

3

Inverse optimal control

4

Experimental results

5

From self paced to slow/fast motions

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 7 / 27

Recovering g

Necessary condition

Fix xf. Value function of the problem in fixed time: V (t, x) = inf{ t L(xu, u) : for xu joining x to xf in time t} Set T = movement time from x0 to xf. Then T ∈ argmin

t∈[0,+∞)

t g(s)ds + V (t, x0)

• ,

and so g(T) = − ∂V

∂t (T, x0).

More precisely, g(T) = −H0(x∗(T), p∗(T), u∗(T)) where: H0(x, p, u) = p, f(x, u) + L(x, u) normal Hamiltonian in fixed time, (x∗(·), u∗(·)) optimal solution in time T with adjoint vector p∗(·).

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 8 / 27

Recovering g

Remark: requires two technical assumptions on the fixed time problem: existence of minimizers no abnormal minimizers (property of the dynamics) Recovering g from experimental data: Fix xf and choose initial conditions x0(a), a ∈ [a1, a2]. Experiments − → T(a) = time of motion from x0(a) to xf. Assume a → T(a) is invertible and set a∗(t) = T −1(t) [Ex: a amplitude ⇒ T ր] Then g(t) = −∂V ∂t (t, x0(a∗(t))).

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 9 / 27

Recovering g

Conclusion Given the cost of motion L(x, u), the cost of time g can be deduced from simple experiments. Problems: how to determine L(x, u)? Robustness of the construction of g?

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 10 / 27

Inverse optimal control

Outline

1

Cost of time in human motions

2

Recovering g

3

Inverse optimal control

4

Experimental results

5

From self paced to slow/fast motions

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 11 / 27

Inverse optimal control

Inverse optimal control

(Direct) Optimal control problem Given a dynamic ˙ x = f(x, u), a cost C(xu) and a pair of points x0, x1, find a trajectory xu∗ solution of inf{C(xu) : xu traj. s.t. xu(0) = x0, xu(T) = x1}. Inverse optimal control problem Given ˙ x = f(x, u) and a set Γ of trajectories, find a cost C(xu) such that every γ ∈ Γ is solution of inf{C(xu) : xu traj. s.t. xu(0) = γ(0), xu(T) = γ(T)}. Applications to analysis/modelling of human motor control (physiology) → looking for optimality principles

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 12 / 27

Inverse optimal control

Inverse problem: Choose a class C of reasonable costs and let Φ : C ∈ C → Γ optimal synthesis. Inverse optimal control problem = find an inverse Φ−1. Well-posed problem? Φ injective? Continuity (and stability) of Φ−1? → Very few general results: Calculus of Variations case [Krupkova, Prince,. . . 1990-2000’s], Linear-Quadratic case [Kalmann 64, Nori-Frezza 04], numerical methods [Mombaur-Laumond 2010, Pauwels-Henrion-Lasserre 2014]

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 13 / 27

Inverse optimal control

Theoretical result

Dynamics = the one of the 1 doF arm motion (single-input linear system) Let SC = set of smooth functions L(x, u) such that

∂2L ∂u2 > 0

(strict convexity) (0, 0) unique minimum, L(0, 0) = 0, and ∂2L

∂u2 (0, 0) = 1

(normalization) Theorem There exists a dense subset Ω s.t. Φ injective on Ω. (Proof based on Thom transversality) Open questions: continuity of (Φ|Ω)−1? Φ(Ω) dense in Φ(SC)?

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 14 / 27

Inverse optimal control

Practical point of view

Linear dynamics: ˙ x = Ax + Bu, x ∈ Rn. Class of admissible costs = class of quadratic costs, C = {L(x, u) = uT Qu + xT Rx + 2xT Su, Q ≻ 0, L sym, 0}. → optimal controls in time T of the form u(t) = KT (t)x(t). Remark: {KT (·), T > 0} uniquely determined by a pair (K−, K+),

• ptimal solutions in time T of the form:

x(t) = e(A+BK+)ty+ + e(A+BK−)(t−T)y−. Hyp: Single input case, i.e. u ∈ R

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 15 / 27

Inverse optimal control

Theorem (adapted from Nori-Frezza, 2004), single-input case For any L ∈ C, there exists a unique k ∈ Rn s.t. Φ(L) = Φ

• (u − kT x)2

. Moreover, (K−, K+) → k continuous (k = −KT

+).

→ the inverse optimal control problem is well-posed. Application to the computation of g Find (K−, K+) by identification from experimental data; set k = −KT

+ and L(x, u) = (u − kT x)2;

compute the function ∂V

∂t (t, x) using the Hamiltonian;

set g(t) = − ∂V

∂t (t, x0(a∗(t))).

→ Method robust w.r.t. perturbations of the data and w.r.t. the choice of cost.

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 16 / 27

Inverse optimal control

Quantitative result: Assume xf = 0 and x0(a) = ax0(1). Then:

∂V ∂t (T, x0) = −ua(T)2,

where ua(·) = optimal control from x0(a) to xf in time T. ua(T) = aν(T), where ν(T) = (K+ − K−)

• e−(A+BK−)T − e−(A+BK+)T −1

x0(1). ⇒ g(t) = ν(t)2a∗(t)2

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 17 / 27

Experimental results

Outline

1

Cost of time in human motions

2

Recovering g

3

Inverse optimal control

4

Experimental results

5

From self paced to slow/fast motions

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 18 / 27

Experimental results

Two kind of motions: Pointing motions of the arm in a horizontal plane (1 degree of freedom), Saccadic eye movements. In both cases: the dynamic is of the form: θ(n) + cn−1θ(n−1) + · · · + c0θ = u, → linear with a state x = (θ, ˙ θ, . . . , θ(n−1)) (n = 2 or 3 in general) Initial and final states are equilibria, typically: x0(a) = (a, 0, . . . , 0) and xf = 0.

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 19 / 27

Experimental results

Observations: For arm pointing motions, T(a) affine function ⇔ a∗(t) = αt − β.

S1 S2 S3 S4

Duration (s) Amplitude (°) 20 40 60 80 100 0.5 1 1.5 2 Duration (s) Amplitude (°) 20 40 60 80 100 0.5 1 1.5 2 Duration (s) Amplitude (°) 20 40 60 80 100 0.5 1 1.5 2 Duration (s) Amplitude (°) 20 40 60 80 100 0.5 1 1.5 2

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 20 / 27

Experimental results

Computation of g

For arm pointing motions: cost of time neither hyperbolic nor exponential...

0 0.5 1 1.5 2 2.5 3 0.0e+00 2.0e+03 4.0e+03 6.0e+03 0 0.5 1 1.5 2 2.5 3 0.0e+00 1.0e+03 2.0e+03 0 0.5 1 1.5 2 2.5 3 0.0e+00 2.0e+04 4.0e+04 0 0.5 1 1.5 2 2.5 3 0.0e+00 2.0e+03 4.0e+03 6.0e+03 0 0.5 1 1.5 2 2.5 3 0.0e+00 5.0e+02 1.0e+03 1.5e+03 0 0.5 1 1.5 2 2.5 3 0.0e+00 2.0e+02 4.0e+02 0 0.5 1 1.5 2 2.5 3 0.0e+00 5.0e+03 1.0e+04 1.5e+04 0 0.5 1 1.5 2 2.5 3 0.0e+00 5.0e+02 1.0e+03

b

Duration (s) Duration (s) Duration (s) Duration (s)

S1 S2 S3 S4 S1 S2 S3 S4

a

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 21 / 27

Experimental results

Computation of g

For eye saccadic movements: cost of time hyperbolic, as expected

100 200 300 400 500 1 2 3 4 5

Duration (ms) Cost of time

Affine relationship True data points

1 2 3 4 5 2 4 6 8 10

Duration (s) Cost of time

Asymptotic cost of time = implicit reward value of the task:

a b

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 22 / 27

From self paced to slow/fast motions

Outline

1

Cost of time in human motions

2

Recovering g

3

Inverse optimal control

4

Experimental results

5

From self paced to slow/fast motions

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 23 / 27

From self paced to slow/fast motions

Interpretation

Form of the infinitesimal cost (with xf = 0): g(t) + u2 + xT Rx [+ eventually mixed terms xT Su] g(t) = cost of the time u2 = cost of the particular motion (objective cost) xT Rx = cost related to the target (penalization) ⇒ change of the instructions (“faster/slower”) = change of R Example: for 1 doF motions, xT Rx = s0θ2 + s1 ˙ θ2 + · · · + sn−1(θ(k−1))2 → “Go quickly to the goal” = increase s0 → “Go slowly” = increase s1

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 24 / 27

From self paced to slow/fast motions

Let Ts0,...,sn−1(a) = duration of an optimal solution between x0(a) and xf. Lemma (Sensitivity analysis) s0 → Ts(a) ց s1 → Ts(a) ր Moreover, when s0 → ∞, θ∗(t) ∼ aCe−λt for t ∈ [Ts(a)/2, Ts(a)) Consequence: if the target centered at 0 has a width w, the motion stops at a time T s.t. θ(T) = w/2 ⇒ T ∼ 1 λ log( a w) + log 2C λ . → Fitts’s law

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 25 / 27

From self paced to slow/fast motions

Self-paced (gray) or maximal speed (black) movements: duration = affine w.r.t. amplitude a or ID= a/w

0.2 0.4 0.6 0.8 1 20 40 60 80 100 0.2 0.4 0.6 0.8 1

• 400
• 300
• 200
• 100

10 20 30 40 50 60 70 80 90 100 0.4 0.6 0.8 1 1.2 1.4 1.6

Amplitude (deg) Index of difficulty (ID) Duration (s) Duration (s) Time (s) Time (s) Velocity (deg/s) Position (deg)

0.4 0.6 0.8 1 1.2 1.4 1.6 2 3 4 5

a b c

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 26 / 27

From self paced to slow/fast motions

Aim Show that the above interpretation may explain the picture below. Data from [Young 2009]: time vs amplitude with constant ID= a/w

• F. Jean (ENSTA)

Why don’t we move slower? IHP, 2014 27 / 27