Kolmogorov-Chaitin Complexity of Linear Digital Controllers - - PowerPoint PPT Presentation

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic

James F. Whidborne∗, John Mckernan∗, Da-Wei Gu†

dag@leicester.ac.uk

†Department of Engineering, University of Leicester ∗Department of Aerospace Sciences, Cranfield University

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 1/24

Introduction

• Modern control design methods may give high-order digital controllers

— regarded as complex, thus costly & difficult to implement

• Rounding errors occur in digital controller implementations — magnified

with controller order — often solved using costlier longer word-length or floating point processors

• Hence control system designers motivated to produce low-order

controllers

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 2/24

Introduction

• Modern control design methods may give high-order digital controllers

— regarded as complex, thus costly & difficult to implement

• Rounding errors occur in digital controller implementations — magnified

with controller order — often solved using costlier longer word-length or floating point processors

• Hence control system designers motivated to produce low-order

controllers But do low-order controllers have lower complexity?

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 2/24

Introduction

For example consider : K1(z) = 1 (z + 1)3 and K2(z) = 1 z + 0.1

• controller K1 has order 3 & controller K2 has order 1
• normal assumption: K2 has lower complexity than K1

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 3/24

Introduction

For example consider : K1(z) = 1 (z + 1)3 and K2(z) = 1 z + 0.1

• controller K1 has order 3 & controller K2 has order 1
• normal assumption: K2 has lower complexity than K1

BUT

• K1 requires 0 multiplication operations & can be implemented exactly
• K2 requires 1 multiplication operation & cannot be implemented exactly!

(0.1 in base-2 is 0.000110011 · · · recurring — hence cannot be represented exactly by binary computer with finite wordlength)

• hence K2 has a higher complexity than K1

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 3/24

Introduction

For example consider : K1(z) = 1 (z + 1)3 and K2(z) = 1 z + 0.1

• controller K1 has order 3 & controller K2 has order 1
• normal assumption: K2 has lower complexity than K1

BUT

• K1 requires 0 multiplication operations & can be implemented exactly
• K2 requires 1 multiplication operation & cannot be implemented exactly!

(0.1 in base-2 is 0.000110011 · · · recurring — hence cannot be represented exactly by binary computer with finite wordlength)

• hence K2 has a higher complexity than K1

Controller order is only an indicator of controller complexity! Required word-length and the arithmetic operations are other factors that determine controller complexity

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 3/24

Introduction

• To measure complexity accurately, we use the idea of

Kolmogorov-Chaitin complexity: the complexity of a computable object is measured by the length, in bits, of the shortest program that computes the object

• We consider the complexity of linear, fixed-point arithmetic, digital

controllers from a Kolmogorov-Chaitin perspective

• The complexity of designs for the restricted complexity controller

benchmark problem are investigated

• It will be seen that, from a Kolmogorov-Chaitin viewpoint, higher-order

controllers with a shorter word-length may have a lower complexity but a better performance than lower-order controllers with longer word-length

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 4/24

Kolmogorov-Chaitin complexity

• Measures the complexity of a computable object by the length, in bits, of

the shortest program that computes the object.

Definition 1. The Kolmogorov complexity ΓU(x) of a computable finite-length binary string, x, with respect to a universal Turing machine U is defined as

ΓU(x) = min

p:U(p)=x ℓ(p)

where p denotes a program interpreted by U and ℓ(p) denotes the description length of

p.

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 5/24

Kolmogorov-Chaitin complexity

• Measures the complexity of a computable object by the length, in bits, of

the shortest program that computes the object.

Definition 1. The Kolmogorov complexity ΓU(x) of a computable finite-length binary string, x, with respect to a universal Turing machine U is defined as

ΓU(x) = min

p:U(p)=x ℓ(p)

where p denotes a program interpreted by U and ℓ(p) denotes the description length of

p.

• Digital controllers are not implemented on universal Turing machines —

the following well-known theorem relates Kolmogorov complexity to the length complexity using some other computing machine

Theorem 1. If U is a universal computer, then for any other computer A,

ΓU(x) ≤ ΓA(x) + cA

for all binary strings x where the constant cA does not depend on x.

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 5/24

Kolmogorov-Chaitin complexity

• For a digital controller, at each sample period, the digital computer

calculates a binary string u(v) (the control) which is dependent on another binary string v (concatenation of measurement and some previous sample period controller states)

• Actually need complexity of controller implementation for all possible

measurements & controller states and hence of all controls

• Controller maps V(the set of all possible v) to U(the set of all possible u)
• A practical complexity measure is (minimum length of the stored

program for all possible v) + (maximum length of v) — gives a measure

• f the total storage requirements for the program, the input data and the

stored data

Definition 2. The complexity Γ of a controller implementation with respect to a controller implementation machine K is defined as

Γ = max{ℓ(v) : v ∈ V} + min{ℓ(p) : K(p) = u(v), for all v ∈ V}

where p denotes program interpreted by K and ℓ(p) is length, in bits, of p

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 6/24

Complexity of state-space realizations

State space representation Ks = (A, B, C, D), with n states, p inputs and q

• utputs, where at the ith time step

x(i + 1) = Ax(i) + By(i), u(i) = Cx(i) + Dy(i) Let k be the concatenated binary string of the parameters of Ks

• If the chosen word-length is w, then the length of k is given by

ℓ(k) = (n + p)(n + q)w and the length of v is given by ℓ(v) = (n + p)w and thus the required data storage of Ks is ℓ(k) + ℓ(v)

• This is not a satisfactory complexity measure, because it does not

consider the possibility of sparse realizations, where some of the controller parameters are zero or unity

• No multiplication or addition operation if a controller parameter is zero
• No multiplication operation if a controller parameter is unity

Hence need to measure complexity using Definition 2 (i.e. measure program length, not data storage)

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 7/24

Evaluating complexity of LTI controllers

Definition 2 complexity of a linear time-invariant controller easily evaluated:

• v has constant length
• program is independent of u, y and x
• hence program length ℓ(p) is independent of v and u

Length of the program ℓ(p) also depends on implementation machine K i.e. is there hardware support for floating point calculations, multiplication

• perations etc? If so, what length “overhead” is given for the use of more

complex operations over simpler operations such as addition or barrel shift?

• Simple computing devices preferred for reasons of cost and reliability
• In this paper, assume device has fixed point arithmetic and multiplication

is by sequence of additions and barrel shifts (i.e. one addition + one shift per unity bit of the constant multiplicand)

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 8/24

Complexity of state-space realizations

Thus the following complexity measure is proposed:

Definition 3. The controller complexity Γ(Ks) of a state space controller realization

Ks = (A, B, C, D) is given by Γ(Ks) = ℓ(v) +

ℓ(k)

• i=1

ki

where the concatenated binary string of the parameters of Ks is given by

k = [k1, k2, . . . , kℓ(k)], ki ∈ {0, 1}. This gives a measure of the length of program in terms of number of

• perations and data storage requirements

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 9/24

Other controller realizations

State space realization is over-parameterized, in that there are a large number of degrees of freedom in the choice of the (A, B, C, D) terms Other common realizations are:

• Parallel and the cascade forms — considered here
• The direct form (obtained directly from the difference equation) is

numerically unreliable for medium and high-order controllers — hence not considered here

• Open-loop stability of the lattice form can be determined by inspection

— advantageous in pre-MATLAB era — hence not considered here

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 10/24

Complexity of parallel and cascade realizations

Parallel and cascade realizations decompose controller into first and second-order direct-form filter blocks: F(z) = a0 + a1z−1 1 + b1z−1 and F(z) = a0 + a1z−1 + a2z−2 1 + b1z−1 + b2z−2

• The parallel structure used here is (for SISO controllers)

Kp(z) =

m

• i=1

Fi(z) where each Fi is a first or second order transfer function

• The cascade structure is given by

Kc(z) =

m

• i=1

Fi(z)

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 11/24

Complexity of cascade realization

• Assume m filter blocks — all second order
• c is the concatenated binary string representation of all filter block

parameters a0, a1, a2, b1, b2

• Length of c is thus ℓ(c) = 5mw where w is the word-length
• Each filter block requires two previous outputs and two previous inputs

to be stored, but two previous outputs of each block (except the last) are equal to the two previous inputs of the next block

• Hence length of v is given by ℓ(v) = (2m + 3)w (measurement must also

be stored)

Definition 4. The controller complexity Γ(Kc) of a cascade controller realization Kc is given by

Γ(Kc) = ℓ(v) +

ℓ(c)

• i=1

ci

where c = [c1, c2, . . . , cℓ(c)], ci ∈ {0, 1}.

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 12/24

Restricted complexity controller benchmark

• Plant is a hydro-suspension system subjected to a disturbance
• Objective is to design the simplest controller that meets some control

specifications K G ✲ ✲ ✲ ✲ ❄ u y Gp + + ❄ up p

• The objective of the control is to reduce the residual force, y, at the first

and second vibration modes of the primary plant, Gp

• The residual force y is measured and fed back to the secondary plant via

a digital controller, K

• Controller gain should be zero at Fs/2, which is achieved by ensuring

the controller includes the term (z−1 + 1)

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 13/24

Restricted complexity controller benchmark

Control specifications are

• output sensitivity function Syp = (1 + KG)−1 constrained in frequency

domain by function Fyp(ω)

• input sensitivity function Sup = −K(1 + KG)−1 constrained in frequency

domain by function Fup(ω)

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 14/24

Restricted complexity controller benchmark

Control specifications are

• output sensitivity function Syp = (1 + KG)−1 constrained in frequency

domain by function Fyp(ω)

• input sensitivity function Sup = −K(1 + KG)−1 constrained in frequency

domain by function Fup(ω) Two performance measures are thus defined as φy = max

ω∈[0,ωs/2] (Syp(ω) − Fyp(ω))

φu = max

ω∈[0,ωs/2] (Sup(ω) − Fup(ω))

where ωs = is sampling frequency

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 14/24

State-space realization controllers

• A set of non-dominated state-space realization controllers that satisfy

criteria φy(Ks) < 0 and φu(Ks) < 0

• btained using Fonseca and Fleming’s multi-objective genetic algorithm
• All controllers either 7th order or 5th order with complexity Γ
• We will see that the superiority of the performance of the higher order

controllers is clear

• The lower complexity of the lower order controllers is less clear

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 15/24

State space realization solution set: φy versus φu

−0.16 −0.14 −0.12 −0.1 −0.08 −0.06 −0.04 −0.02 −4 −3.5 −3 −2.5 −2 −1.5 −1 −0.5

10 10 10 11 12 13 13 13 13 13 13 14 14 14 14 14 15 15 15 15 15 15 15 16

φy (dB) φu (dB)

each solution labeled with word-length more complex solutions shown darker 5th order controllers marked ◦

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 16/24

State space realization solution set: φy versus Γ

−0.16 −0.14 −0.12 −0.1 −0.08 −0.06 −0.04 −0.02 100 150 200 250 300 350 400 450 500 φy (dB) Γ

5th order controllers marked ⊙

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 17/24

State space realization solution set: φu versus Γ

−4 −3.5 −3 −2.5 −2 −1.5 −1 −0.5 100 150 200 250 300 350 400 450 500 φu (dB) Γ

5th order controllers marked ⊙

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 18/24

Parallel and cascade realization controllers

• Similarly a set of non-dominated parallel and cascade realization

controllers are obtained

• All controllers either 7th order or 5th order with complexity Γ
• Again the superiority of the performance of the higher order controllers

is clear

• The lower complexity controllers are not necessarily the lower order

controllers

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 19/24

Parallel & cascade realization solution set: φy vs φu

−0.16 −0.14 −0.12 −0.1 −0.08 −0.06 −0.04 −0.02 −4 −3.5 −3 −2.5 −2 −1.5 −1 −0.5

10 12 12 12 12 16 16 16 16 20 20 20 20 24 24 24 24 8

φy (dB) φu (dB)

each solution labeled with word-length more complex solutions shown darker 5th order controllers marked ◦

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 20/24

Parallel & cascade realization solution set: φy vs Γ

−0.16 −0.14 −0.12 −0.1 −0.08 −0.06 −0.04 −0.02 100 150 200 250 300 350 400 φy (dB) Γ

5th order controllers marked ⊙

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 21/24

Parallel & cascade realization solution set: φu vs Γ

−4 −3.5 −3 −2.5 −2 −1.5 −1 −0.5 100 150 200 250 300 350 400 φu (dB) Γ

5th order controllers marked ⊙

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 22/24

Output and input sensitivity functions for lowest complexity controller (8-bit word-length, 7th order) and the performance constraints

50 100 150 200 250 300 350 400 −6 −4 −2 2 4 6 f (Hz) |Syp| (dB) Output sensitivity function and performance constraint 50 100 150 200 250 300 350 400 −80 −60 −40 −20 20 f (Hz) |Sup| (dB) Input sensitivity function and performance constraint

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 23/24

Conclusions

• We consider controller complexity based on idea of Kolmogorov-Chaitin

complexity

• Practical measures of complexity developed for state-space, parallel and

cascade realizations

• Controllers that are low-complexity in this sense will have low

word-length, low number of arithmetic operations and simple realizations — the most important considerations for practical implementations (especially for field-programmable gate arrays)

• Measure is related to the controller order but the lowest complexity

controller is not necessarily the lowest order

• Higher-order controllers provide an extra degrees of freedom to help
• btain controller realizations that have a shorter word-length and simpler

realization structure, and thus a lower complexity

Kolmogorov-Chaitin Complexity of Linear Digital Controllers Implemented using Fixed-point Arithmetic – p. 24/24