Logic, language and the brain Michiel van Lambalgen Cognitive - - PowerPoint PPT Presentation

Logic, language and the brain Michiel van Lambalgen Cognitive Science Center Amsterdam http://staff.science.uva.nl/˜michiell

Aim and program

• aim: explain the use of computational logic in cognitive science

– the domain is language comprehension and production – show how logical modelling leads to testable predictions, both for behaviour and brain imaging – show how logical modelling connects to biological issues, e.g. neural substrate of linguistic processing, and evolutionary considerations

• lecture 1: time, tense and biology
• lecture 2: the event calculus
• lecture 3: verb tense and closed world reasoning
• lecture 4: predictions for EEG
• lecture 5: executive function and behavioural predictions for autism

and ADHD; neural network architecture

Warming-up: tense/aspect and goals

• consider ‘Mary was writing a letter when her sister spilled coffee over

the paper’

• the syntactic structure of ‘write a letter’ seems to suggest a transitive

verb with a direct object

• but ‘a letter’ is not a direct object in the sense of ‘a ball’ in ‘kick a

ball’ – e.g. it need not exist, or only partially

• whether it can be assumed to exist depends on tense/aspect
• it will be fruitful to view ‘a letter’ as goal to be achieved
• ‘The semantics of tense and aspect is profoundly shaped by concerns

with goals, actions and consequences . . . temporality in the narrow sense of the term is merely one facet of this system among many.’ (Steedman, Temporality)

Introducing the event calculus

• language comprehension was characterised as a mapping

discourse → discourse model

• the discourse model contains causal information imported from world

knowledge

• the mapping discourse → discourse model is non-monotonic
• the discourse model will be viewed as the minimal model (w.r.t. well-

founded semantics) of a (constraint) logic program which consists of – axioms for causality – clauses expressing the meaning of the lexical items in the discourse – ‘goals representing the sentences in the discourse’

• the backbone of this logic program is furnished by the event calculus,

a theory of causation developed by Kowalski in a legal context and by Shanahan to apply to robotics

Event calculus: general logical characteristics

• formulated in many-sorted predicate logic; primitive predicates for

causal concepts, connected by axioms

• how can such a formalism ever be computationally feasible?
• the logical reflex: look at modal logics, considered as subsystems of

predicate logic (modal formulas correspond to predicate logical formu- las involving a single binary R)

• which are expressively rich qua iterability of the modal operators, but

the language itself is poor

• another option: rich language, but restrictions on the recursive defini-

tion of wffs

• (representational versus procedural semantics)

Event calculus: ontology

• obviously the event calculus is about events, but there is a distinction

in the event calculus between different kinds of events (‘perfect’ and ‘imperfect’ nominals – PToE ch. 12) – action/event types: e, e′ . . . (for example ‘break’, ‘ignite’) [perhaps a further distinction between actions and events is necessary – gov- erned by separate axioms?] – (there are good reasons for having both event types (‘lightning’) and tokens (‘lightning on August 7, 2008, 8.25am’); e.g. perfect nominalisation yields event types) – implicitly time-varying properties or fluents: f, f′ . . . (for example ‘being broken’, ‘walking’), possibly with parameters – one can obtain these from imperfect nominalisation

• event types (or tokens?)

cause changes in time-varying properties (instantaneous change (Hume))

• sometimes a fluent causes another fluent to change: pushing in ‘push

a cart’ changes the position of the cart – continuous change (Kant)

Jean-Yves Girard on event ontology Il y a d’autres intuitions de base qui ont ´ et´ e ´ evacu´ ees par la logique, ainsi la distinction essentielle entre parfait et imparfait, distinction rendu en fran¸ cais par le choix des temps, en russe par le changement de verbe. Cette nuance n’existe pas dans le monde v´ eriste.’ (Girard, La logique comme g´ eom´ etrie du cognitif ) (There are other basic intuitions that have been kicked out by logic, for example the essential distinction between perfective and imperfective aspect, a distinction captured in French by verb tenses, and in Slavic languages by verb pairs. This subtle distinction does not exist in logics

• bsessed with truth.)

Event calculus: auxiliary ontology

• individual objects (‘John’) – although many individuals will be mod-

elled as fluents, not as objects

• (objects can be viewed as temporally extended events)
• instants of time, interpreted as ‘real numbers’ – technically variables

for time take values in a ‘real-closed field’

• (a ’real-closed field’ (Tarski) is a model of the set of axioms for the real

numbers in the language <, +, × (e.g. ‘a polynomial of odd degree has a root’) – these axioms are complete)

• this choice does not reflect an ontological commitment to a particular

structure of time (e.g. a continuum of points): there are also many countable structures satisfying the axioms for real-closed fields, in some

• f these all ‘reals’ are computable, and hence approximable
• various other real quantities for e.g. position, velocity, degree of some

quality (such as state of completion of a house in the process of being built) [with the same proviso as for time]

Event calculus: logical aspects

• instants of time, interpreted as ‘real numbers’ – technically variables

for time take values in a ‘real-closed field’

• a ’real-closed field’ (Tarski) is a model of the set of axioms for the real

numbers in the language <, +, × (e.g. ‘a polynomial of odd degree has a root’) – these axioms are complete

• completeness follows from quantifier elimination: every quantified for-

mula in this language is equivalent to a Boolean combination of poly- nomial equalities and inequalities (‘constraints’)

• (gives good decision procedure)
• most importantly: definable sets have a very simple structure – e.g.

all definable subsets of the real line are finite unions of intervals

• (technically: definable sets are semi-algebraic)

Event calculus: primitive predicates for instantaneous change

• relations and functions such as <, +, × over the reals
• event calculus predicates for instantaneous (Humean) change
• 1. Initially(f) (‘fluent f holds at the beginning of the discourse’)
• 2. Happens(e, t) (‘event type e has a token at t’)
• 3. Initiates(e, f, t) (‘the causal effect of event type e at time t is the

fluent f’)

• 4. Terminates(e, f, t) (‘the causal effect of event type e at time t is

the negation of the fluent f’)

• 5. Clipped(s, f, t) (roughly, ‘an event type terminating f has a token

between times s and t’)

• 6. the ‘truth predicate’ HoldsAt(f, t) (see below)

More on event types and fluents

• in standard first order logic there is an absolute distinction between

terms and formulas

• terms are constructed from variables (x, y, z, x1, . . .), constants

(a, b, c, a1, . . .) and function symbols (f, g, . . .) for each arity; e.g. f(x1, a) is a term

• formulas are built up from atomic formulas (see below) using the

logical operations ¬, ∧, ∨, ∀, ∃

• an atomic formula is constructed from predicates A(x1, . . . , xn) by

substitution of terms t1, . . . , tn for the variables x1, . . . , xn

• what is not allowed is a ‘formula’ of the form A(B(x, b), t), i.e. where

a formula is substituted for a variable

• event types and fluents are terms which can be seen as codes for

formulas via reification (also called G¨

• delization) – what is this?

More on event types and fluents

• events – e.g.

shaking hands, the destruction of natural habitats – seem to act like terms somehow derived from natural language ex- pressions

• verb tenses seem to need a transformation of V(erb)P(hrases) into

various kinds of events

• hence if one treats natural language formally, i.e. as a formal lan-

guage with a formal semantics, one needs to have a transformation of formulas into terms

• this transformation must be iterable: one can say

(1) Halting the destruction of natural habitats will prove to be diffi- cult.

• furthermore there must actually be two such transformations

– from ‘x destroys natural habitats’ to ‘the destruction of natural habitats’ [perfect nominal] – from ‘x destroys natural habitats’ to ‘destroying natural habitats’ [imperfect nominal]

More on event types and fluents

• there is a general procedure to transform formulas into terms: G¨
• del

numbering – originally devised for treating self-reference

• standard notation: if ϕ is a formula, then ϕ is its G¨
• del number
• in AI this procedure is called reification
• we still have to bring in the distinction between perfect (‘noun-like’)

and imperfect (‘verb-like’), which has to do with time – in a ‘verb-like’ nominal time is an internal argument

• assume all verbs come with a variable over time (not over events, as

in Davidson): destroy(x, y, t)

• the imperfect nominal corresponds to destroy(x,y,t)
• the perfect nominal corresponds to ∃tdestroy(x,y,t)
• in the event calculus, fluents are formed analogous to destroy(x,y,t),

event types analogous to ∃tdestroy(x,y,t)

Axioms for the event calculus, instantaneous change only Assume given a set of axioms for the reals with +, ×, < (‘axioms for real-closed fields’). Axioms specific to the event calculus (EC) are (all variables universally quantified): Axiom’ 1 Initially(f) ∧ ¬Clipped(0, f, t) → HoldsAt(f, t) Axiom’ 2 Happens(e, t) ∧ Initiates(e, f, t) ∧ t < t

′ ∧ ¬Clipped(t, f, t ′)

→ HoldsAt(f, t

′)

Axiom’ 3 Happens(e, s) ∧ t < s < t

′ ∧ Terminates(e, f, s)

→ Clipped(t, f, t

′)

General models for EC are just structures for the many-sorted language

• f EC which satisfy the axioms, but . . .
• without axioms for HoldsAt there is no connection between fluents

and time; even with axioms for HoldsAt, there are many unintended models

HoldsAt as a truth predicate.

• binary truth predicate T(•, •) (’satisfaction’) is characterised by

M | = ϕ(a) ⇐ ⇒ M | = T(ϕ(x), a)

• HoldsAt is like T(•, •), but the second argument always stands for

time

• we need it because in the event calculus formulas may also occur as
• bjects, in particular fluents
• ever since G¨
• del 1931: coding formulas as terms
• in our case we made a formula ϕ(t) act as a term (function or set)

ϕ(s) = {s | ϕ(s)}, which can be viewed as a fluent

• nominalization is strictly analogous to G¨
• del numbering
• HoldsAt establishes a correspondence between fluents and sets of in-

stants via M | = ϕ(t) ⇐ ⇒ M | = HoldsAt({s | ϕ(s)}, t)

Additional axioms for HoldsAt of the following type Axiom’ 4 HoldsAt(f1, t) ∧ HoldsAt(f2, t) → HoldsAt(f1 ∧ f2, t) Axiom’ 5 ¬HoldsAt(f, t) → HoldsAt(¬f, t) Etc. Consistency of these axioms with the event calculus is not easy to show – because of the Liar Paradox ψ(t) ↔ HoldsAt(¬ψ(s), t)

• ne actually needs 3-valued logic.

But one needs the truth predicate to establish contact between the event language and natural language.

Typical models of the event calculus: instantaneous change

✲

R e0 e1 ( ] e2 e3 ( ] e4 e5 ( ] f

P P P P P P P P P P P P P P P P P P P P P P P P P P P P ✐ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ■ ✑✑✑✑✑✑✑✑✑✑✑✑✑✑✑ ✑ ✸

• intuitively, fluents are represented by intervals because of inertia [not

yet quite captured by the axioms themselves]

• inertia seems to rule out a situation in which f holds only on rational

numbers

• intervals left-open because fluent f does not hold at the moment it is

initiated

More on the structure of time assumed by the event calculus Caution: logic!!

• the variable t is assumed to range over the reals – does this mean we

assume the uncountable continuum as the ‘true’ structure of time?

• no: we are always considering computations over cognitive time – it

is sufficient if the results are consistent with ‘real time’

• in particular, using the reals does not take a stand on the question

whether time ‘really’ consists of instants or on the contrary of (tem- porally extended) events only

• in fact, a consequence of inertia is that event types and fluents corre-

spond to finite sets of intervals (with computable end points), which is consistent with the assumption that events are basic

• since the end points are computable, they can be approximated – one

need not assume infinite precision

• the continuum can thus be seen as facilitating computation, but ulti-

mately unnecessary

Event calculus: states and scenarios The event calculus can be made computationally manageable because

• f limitations on formation rules for formulas. Note that the axioms are

implications whose consequent is always atomic.

• a goal of the form ?

HoldsAt(f, t), . . . or ? Happens(e, t), . . . for given f, e

• which is short for:

‘can we make it to be the case that Hold- sAt(f, t), . . . [Happens(e, t), . . .] for some t?’

• the scenario describes the cognitive representation of agent and envi-

ronment in the language of event calculus

• a scenario must be a theory (a finite set of sentences) of a specific

syntactic form to be plausible as memory structure – basically Horn clauses with negations allowed in the antecedent

• syntactic form of scenario defined in two steps

Scenarios: formalizing world knowledge Definition 1 A state S(t) at time t is a conjunction of literals involving

• nly
• 1. literals of the form (¬)HoldsAt(f, t)
• 2. equalities between fluent terms, and between event terms
• 3. equations and inequalities involving real numbers

Definition 2 A scenario is a conjunction of statements of the form

• 1. Initially(f),
• 2. ∀t(S(t) → Initiates(e, f, t)),
• 3. ∀t(S(t) → Terminates(e, f, t)),
• 4. ∀t, s(S(t, s) ∧ Happens(e0, s) → Happens(e, t)),

where the S(t), . . . are states in the sense of definition 1.

Exercise Consider a room in which there are two lights and two switches, each serving one light. We have the following event types and fluents:

• 1. a1 = switch1 on
• 2. e1 = switch1 off
• 3. a2 = switch2 on
• 4. e2 = switch2 off
• 5. f1 = light1 on
• 6. f2 = light2 on

At time 5 light 1 is switched on, at time 10 it is switched off. Light 2 is on initially. Write a scenario for this setup. Can you describe uniquely the light situation at times t for 0 ≤ t ≤ 15?

Scenario for this situation

• 1. ¬HoldsAt(f1, t) → Initiates(a1, f1, t) [universal qu. over t]
• 2. ¬HoldsAt(f2, t) → Initiates(a2, f2, t)
• 3. HoldsAt(f1, t) → Terminates(e1, f1, t)
• 4. HoldsAt(f2, t) → Terminates(e2, f2, t)
• 5. Happens(a1, 5)
• 6. Happens(e1, 10)
• 7. Initially(f2)

How to derive the times the lights are on/off? Example: light 2. Here we we seem to be in a position to apply Axiom 1 with f = f2 Initially(f2) ∧ ¬Clipped(0, f2, t) → HoldsAt(f2, t) But what to do with ¬Clipped(0, f2, t)? Axiom 3 seems inapplicable: Happens(e, s) ∧ t < s < t

′ ∧ Terminates(e, f, s) → Clipped(t, f, t ′)

The Frame Problem (McCarthy), or the Robot’s Dilemma (Dennett)

• intuitively 1: events might also occur at other times (e.g. switch1 off

at t = 6, but also collision with an asteroid at t = 7)

• intuitively 2: events might have causal effects not mentioned in the

scenario, e.g. Terminates(a1, f2, t)

• formally: the scenario allows a great many models which are not in-

tended

• formally: in classical logic, nothing interesting follows from axioms

plus scenario, because these premisses allow too many models

• the robot’s dilemma is therefore that it is impossible to decide what

to do, since there are so many possibilities to take account of

CWR: restricting the set of models by employing the completion of the scenario

• we want to say that the only occurrences of events are: at time 5 light

1 is switched on, at time 10 it is switched off

• formally the completion of Happens is defined by

∀e, t(Happens(e, t) ↔ (t = 5 ∧ e = a1) ∨ (t = 10∧ = e1))

• similarly the completion of Initially is defined as

∀f(Initially(f) ↔ f = f2)

• the completion of Clipped is [∀ over f, t, t

′ suppressed]

∃s(Happens(e, s)∧t < s < t

′∧Terminates(e, f, s)) ↔ Clipped(t, f, t ′)

• this gives information about ¬Clipped(0, f2, t): true for all t
• we will have to further restrict the class of models by using the com-

pletion of the axioms (i.e. HoldsAt) to get right result for light 1

The frame problem and nonmonotonic inference

• classical definition of validity

ϕ1 . . . ϕn | = ψ iff for all models M: if M | = ϕ1 ∧ . . . ∧ ϕn, then M | = ψ

• useless for reasoning about goals, plans and actions
• reasoning with the completion . . .

ϕ1 . . . ϕn | ≈ ψ iff comp(ϕ1 ∧ . . . ∧ ϕn) | = ψ

• . . . is nonmonotonic

Happens(a1, 5), Happens(a2, 10) | ≈ ¬Happens(a2, 7) Happens(a1, 5), Happens(a2, 7), Happens(a2, 10) | ≈ ¬Happens(a2, 7)

Event calculus: primitive predicates (also continuous change)

• predicates and functions such as +, ×, < over the reals
• event calculus predicates for instantaneous change:

Initially(f), Happens(e, t), Initiates(e, f, t), Terminates(e, f, t), Clipped(s, f, t), HoldsAt(f, t)

• event calculus predicates for continuous change
• 1. Releases(e, f, t) (‘event type e has the effect of freeing f from the

law of inertia’)

• 2. Trajectory(f1, t, f2, d) (‘if f1 holds between t and t+d, then f2 will

hold at t + d’)

• Clipped(s, f, t) gets extended meaning: ‘an event type releasing or

terminating f has a token between times s and t’

Causation and continuous change

• axioms for instantaneous change formalize principle of inertia: after

the cause has stopped acting, the caused state does not change

• this principle is not valid for continuous causation: think of gravity

acting on a falling object

• Releases(e, f, t) stipulates that when e happens, f is no longer subject

to the principle of inertia

• example of continuous change: crossing the street

HoldsAt(distance(x), t) → Trajectory(crossing,t,distance(x + d),d)

• example of continuous change: drawing a circle

HoldsAt(arc(x), t) → Trajectory(drawing,t,arc(g(x, d)),d) for some function g

• this will give a solution to the problem of partial objects we identified

in the Davidson/Parsons approach

Axioms for the event calculus Assume given a set of axioms for the reals with +, ×, < (‘axioms for real-closed fields’). Axioms specific to the event calculus are (all variables universally quan- tified): Axiom 1 Initially(f) ∧ 0 < t ∧ ¬Clipped(0, f, t) → HoldsAt(f, t) Axiom 2 Happens(e, t) ∧ Initiates(e, f, t) ∧ t < t

′ ∧ ¬Clipped(t, f, t ′)

→ HoldsAt(f, t

′)

Axiom 3 Happens(e, t) ∧ Initiates(e, f1, t) ∧ t < t

′ ∧ t ′ = t + d ∧

Trajectory(f1, t, f2, d) ∧ ¬Clipped(t, f1, t

′) → HoldsAt(f2, t ′)

Axiom 4 Happens(e, s) ∧ t < s < t

′ ∧

(Terminates(e, f, s) ∨ Releases(e, f, s)) → Clipped(t, f, t

′)

Scenarios (full version) Definition 3 A scenario is a conjunction of statements of the form

• 1. Initially(f),
• 2. ∀t(S(t) → Initiates(e, f, t)),
• 3. ∀t(S(t) → Terminates(e, f, t)),
• 4. ∀t(S(t) → Releases(e, f, t)),
• 5. ∀t, s(S(t, s) ∧ Happens(e0, s) → Happens(e, t)),
• 6. S(f1, f2, t, d) → Trajectory(f1, t, f2, d),

where the S(t), . . . are states in the sense of definition 1.

Example: lexical entry for the accomplishment ‘cross the street’ (roughly) Vocabulary: fluents one–side, other–side, crossing, distance(x); event types start, reach

• 1. Initially(one–side)
• 2. Initially(distance(0))
• 3. HoldsAt(distance(m), t) ∧ HoldsAt(crossing, t)→ Happens(reach, t)
• 4. Initiates(start, crossing, t)
• 5. Releases(start, distance(0), t)
• 6. Initiates(reach, other–side, t)
• 7. Terminates(reach, crossing, t)
• 8. HoldsAt(distance(x), t) →Trajectory(crossing,t,distance(x + d),d)
• 9. HoldsAt(distance(x1),t) ∧ HoldsAt(distance(x2), t) → x1 = x2

[strictly speaking this is not in the right syntactic form, but that can be remedied]

Plans contained in a lexical entry

• consider the goal ?HoldsAt(other–side,t), t ≥ now
• want to derive plan for achievement of this goal
• do this by CWR using axioms of the event calculus and the scenario
• e.g. by axiom 2 reach event must have occurred,
• by scenario 3 this can only be if distance m has been covered
• by axiom 3 this distance can be covered only if the activity crossing

persists for sufficiently long, etc.

• compare this semantic representation with set theoretic representation,

such as (?){(a, b) | cross(a, b)}, s= ‘the street’

• which cannot capture the semantic contribution of the ‘incremental

theme’ (Dowty)

Event calculus: model of the above

• Above scenario represents present progressive ‘John is crossing the

street’

• Comes with goal ?Happens(reach,t), t ≥ now.
• Goal can (only) be achieved in minimal model of scenario, defined by

the completion of the axioms plus scenario.

• E.g.

Happens(e, t)

• (e = start ∧ t = t0) ∨(e = reach ∧HoldsAt(distance(m), t) ∧

HoldsAt(crossing, t))