Cognition for Intelligent Robotics Architectures and Action - - PowerPoint PPT Presentation

Joanna J. Bryson

University of Bath, United Kingdom

Cognition for Intelligent Robotics

Architectures and Action Selection

Why Action Selection?

Why Action Selection?

Functionalist Assumption: All we care about is producing intelligent behaviour.

Why Action Selection?

Functionalist Assumption: All we care about is producing intelligent behaviour.

• Physical Symbol System Hypothesis (Newell

& Simon 1963); Qualia, Chalmers “hard problem” (1995).

Why Action Selection?

Functionalist Assumption: All we care about is producing intelligent behaviour.

• Physical Symbol System Hypothesis (Newell

& Simon 1963); Qualia, Chalmers “hard problem” (1995).

• Consciousness as epiphenomena

(Churchland 1988, Brooks 1991).

Why Action Selection?

Functionalist Assumption: All we care about is producing intelligent behaviour.

• Physical Symbol System Hypothesis (Newell

& Simon 1963); Qualia, Chalmers “hard problem” (1995).

• Consciousness as epiphenomena

(Churchland 1988, Brooks 1991). We’ll build it if we need it. Scienc

Why Action Selection?

Functionalist Assumption: All we care about is producing intelligent behaviour.

• Physical Symbol System Hypothesis (Newell

& Simon 1963); Qualia, Chalmers “hard problem” (1995).

• Consciousness as epiphenomena

(Churchland 1988, Brooks & Stein 1993). Science: We’ll build it to see if we need it.

Outline

• Introduction:

Intelligence, Cognition & Architecture

• A Brief History of AI Cognitive

Architectures

• Behavior Oriented Design

Intelligence

• What matters is expressing the right

behavior at the right time: action selection.

• Conventional AI planning searches for an

action sequence, requires set of primitives.

• Learning searches for the right parameter

values, requires primitives and parameters.

• parameter: variable state.
• Evolution and development are learning.

Combinatorics

• If . . .

– an agent knows 100 actions (e.g. eat, drink, sleep, step, turn, lift, grasp, poke, flip...), and – it has a goal (e.g. go to Madagascar)

• Then . . .

– Finding a one-step plan may take 100 acts. – A two-step plan may take 1002 (10,000). – For unknown number of steps, may search forever, missing critical steps or sequence.

Intelligence & Design

Intelligence & Design

• Combinatorics is the problem, search is the
• nly solution.

Intelligence & Design

• Combinatorics is the problem, search is the
• nly solution.
• The task of intelligence is to focus search.
• Called bias (learning) or constraint (planning).
• Most `intelligent’ behavior has no or little real-

time search (non-cognitive).

Intelligence & Design

• Combinatorics is the problem, search is the
• nly solution.
• The task of intelligence is to focus search.
• Called bias (learning) or constraint (planning).
• Most `intelligent’ behavior has no or little real-

time search (non-cognitive).

• For artificial intelligence, most focus from design.

Cognition

Cognition

Definition: Cognition is on-line (real-time) search.

Cognition

Definition: Cognition is on-line (real-time) search. Consequence: Cognition is bad.

Cognition

Cognition

• Why is cognition / individual search bad?
• Slow
• Uncertain

Cognition

• Why is cognition / individual search bad?
• Slow
• Uncertain
• Unpopular in most species.
• e.g. Plants

Cognition

• When is cognition useful?
• Deeply dynamic environments -- change

faster than learning or evolution can adapt.

• Baldwin Effect -- fast & noisy search

facilitates (speeds up) slower & more reliable learning processes (Baldwin 1896, Hinton & Nowlan 1987).

Cognition, Learning & Design

• When is cognition useful?
• Baldwin Effect -- fast & noisy search

facilitates (speeds up) slower & more reliable learning processes.

• Behaviour Oriented Design -- cognition &

learning can facilitate building a working robot.

Architecture

• Where do you put the cognition?
• Really: How do you bias / constrain / focus

cognition so that it works?

Architecture

• What parts do you put together where?

Cognitive Architecture

• Where do you put the cognition?

Cognitive Architecture

• Where do you put the cognition?
• Really: How do you bias / constrain / focus

cognition (learning, search) so it works?

Outline

• Introduction

Intelligence, Cognition & Architecture

• A Brief History of AI Cognitive

Architectures

• Behavior Oriented Design

References

References

• Cyril Brom and Joanna J. Bryson, “Action Selection for

Intelligent Systems”, white paper for euCognition, 7 August 2006.

References

• Cyril Brom and Joanna J. Bryson, “Action Selection for

Intelligent Systems”, white paper for euCognition, 7 August 2006.

• Joanna J. Bryson, “Cross-Paradigm Analysis of

Autonomous Agent Architecture”, Journal of Experimental and Theoretical Artificial Intelligence 12(2): 165-190, 2000.

References

• Cyril Brom and Joanna J. Bryson, “Action Selection for

Intelligent Systems”, white paper for euCognition, 7 August 2006.

• Joanna J. Bryson, “Cross-Paradigm Analysis of

Autonomous Agent Architecture”, Journal of Experimental and Theoretical Artificial Intelligence 12(2): 165-190, 2000.

• Joanna J. Bryson and Lynn Andrea Stein, “Architectures

and Idioms: Making Progress in Agent Design”, The Seventh International Workshop on Agent Theories, Architectures and Languages (ATAL), Boston, 2000.

“History as Evolution” Hypothesis

(Bryson JETAI 2000)

“History as Evolution” Hypothesis

• If an architecture is around for a while,

and it changes, the change was probably selected, adaptive. (Bryson JETAI 2000)

“History as Evolution” Hypothesis

• If an architecture is around for a while,

and it changes, the change was probably selected, adaptive.

• This is particularly likely if the change

goes against the stated theories of the architectureʼs makers. (Bryson JETAI 2000)

“History as Evolution” Hypothesis & Correlary

• If similar features occur in a lot of

architectures with different phylogenies, those features are probably adaptive. (Bryson & Stein ATAL 2000)

“History as Evolution” Hypothesis & Correlary

• If similar features occur in a lot of

architectures with different phylogenies, those features are probably adaptive.

• If you want to make a contribution to a

field, describe your best innovations in terms of well-known systems. (Bryson & Stein ATAL 2000)

Productions

• From sensing to action (c.f. Skinner;

conditioning; Witkowski 2007.)

• These work -- basic component of

intelligence.

• The problem is choice (search).
• Requires an arbitration mechanism.

Production-Based Architectures

• Expert Systems: allow choice of

policies, e.g. recency, utility, random.

• SOAR: problem spaces (from GPS),

impasses, chunk learning.

• ACT-R: (Bayesian) utility, problem

spaces (reluctantly, from SOAR/GPS.)

Soar

• Productions operate
• n predicate

database.

• If conflict, declare

impasse, reason (search).

• Remember

resolution: chunk

!" !"#$%&'($&#)* +,-./ 0".%* 1-%/&"# 2.3"% 4-/(5$/ 67.895- 0:/$-8/ +895-8-#$.$&"# #$%&'$& ()*+,'-. /-*0 1,2,34,356 7)89.*., :-;,<,35, 1,5=9=-3>:650, 7-*<?>@ ABBB 7/C$ &*5D=<@7-*< EFD@7-*< CG2<-H,4 C3.,<I*5,9 &:JK&+ F<*22,< 7-*<L>@ ABBM D=<@7-*< C39.<)5.-@7-*< N=O; PII=5=,356 7-*<M>@ ABBQ : D=<@7-*< N,<-@7-*< PR.,<3*0 &*9+9 1,9.<)5.=H, $2,<*.-<9 7=3O0,>7.*., 7-*<!>@ AB?B P&@7-*< SJ@7-*< PR.,<3*0> E,0,*9, %&: 7-*<T>@ AB?M /,3,<*0 J,*<3=3O EA@7-*< :;)3+=3O 7-*<U>@ AB?T %3=H,<9*0 7)8O-*0=3O EA@7-*< 162*<@7-*< $V7! J=92 V<,I,<,35,9 7)8O-*09 7-*<Q>@ AB?U 76G8-0 769.,G9 N,)<=9.=5 7,*<5; V<-4)5.=-3 769.,G9 %3=H,<9*0 F,*+>#,.;-4 WDV7>Q J=92 F,*+ #,.;-49 7-*<A>@ AB?Q &-6>&*9+9 V<-80,G> 72*5,9

Soar

!" !"#$%&'($&#)* +,-./ 0".%* 1-%/&"# 2.3"% 4-/(5$/ 67.895- 0:/$-8/ +895-8-#$.$&"# #$%&'$& ()*+,'-. /-*0 1,2,34,356 7)89.*., :-;,<,35, 1,5=9=-3>:650, 7-*<?>@ ABBB 7/C$ &*5D=<@7-*< EFD@7-*< CG2<-H,4 C3.,<I*5,9 &:JK&+ F<*22,< 7-*<L>@ ABBM D=<@7-*< C39.<)5.-@7-*< N=O; PII=5=,356 7-*<M>@ ABBQ : D=<@7-*< N,<-@7-*< PR.,<3*0 &*9+9 1,9.<)5.=H, $2,<*.-<9 7=3O0,>7.*., 7-*<!>@ AB?B P&@7-*< SJ@7-*< PR.,<3*0> E,0,*9, %&: 7-*<T>@ AB?M /,3,<*0 J,*<3=3O EA@7-*< :;)3+=3O 7-*<U>@ AB?T %3=H,<9*0 7)8O-*0=3O EA@7-*< 162*<@7-*< $V7! J=92 V<,I,<,35,9 7)8O-*09 7-*<Q>@ AB?U 76G8-0 769.,G9 N,)<=9.=5 7,*<5; V<-4)5.=-3 769.,G9 %3=H,<9*0 F,*+>#,.;-4 WDV7>Q J=92 F,*+ #,.;-49 7-*<A>@ AB?Q &-6>&*9+9 V<-80,G> 72*5,9

Soar

• Soar has serious

engineering.

!" !"#$%&'($&#)* +,-./ 0".%* 1-%/&"# 2.3"% 4-/(5$/ 67.895- 0:/$-8/ +895-8-#$.$&"# #$%&'$& ()*+,'-. /-*0 1,2,34,356 7)89.*., :-;,<,35, 1,5=9=-3>:650, 7-*<?>@ ABBB 7/C$ &*5D=<@7-*< EFD@7-*< CG2<-H,4 C3.,<I*5,9 &:JK&+ F<*22,< 7-*<L>@ ABBM D=<@7-*< C39.<)5.-@7-*< N=O; PII=5=,356 7-*<M>@ ABBQ : D=<@7-*< N,<-@7-*< PR.,<3*0 &*9+9 1,9.<)5.=H, $2,<*.-<9 7=3O0,>7.*., 7-*<!>@ AB?B P&@7-*< SJ@7-*< PR.,<3*0> E,0,*9, %&: 7-*<T>@ AB?M /,3,<*0 J,*<3=3O EA@7-*< :;)3+=3O 7-*<U>@ AB?T %3=H,<9*0 7)8O-*0=3O EA@7-*< 162*<@7-*< $V7! J=92 V<,I,<,35,9 7)8O-*09 7-*<Q>@ AB?U 76G8-0 769.,G9 N,)<=9.=5 7,*<5; V<-4)5.=-3 769.,G9 %3=H,<9*0 F,*+>#,.;-4 WDV7>Q J=92 F,*+ #,.;-49 7-*<A>@ AB?Q &-6>&*9+9 V<-80,G> 72*5,9

Soar

• Soar has serious

engineering.

• “Evolution of Soar”

is my favorite paper (Laird & Rosenbloom 1996)

!" !"#$%&'($&#)* +,-./ 0".%* 1-%/&"# 2.3"% 4-/(5$/ 67.895- 0:/$-8/ +895-8-#$.$&"# #$%&'$& ()*+,'-. /-*0 1,2,34,356 7)89.*., :-;,<,35, 1,5=9=-3>:650, 7-*<?>@ ABBB 7/C$ &*5D=<@7-*< EFD@7-*< CG2<-H,4 C3.,<I*5,9 &:JK&+ F<*22,< 7-*<L>@ ABBM D=<@7-*< C39.<)5.-@7-*< N=O; PII=5=,356 7-*<M>@ ABBQ : D=<@7-*< N,<-@7-*< PR.,<3*0 &*9+9 1,9.<)5.=H, $2,<*.-<9 7=3O0,>7.*., 7-*<!>@ AB?B P&@7-*< SJ@7-*< PR.,<3*0> E,0,*9, %&: 7-*<T>@ AB?M /,3,<*0 J,*<3=3O EA@7-*< :;)3+=3O 7-*<U>@ AB?T %3=H,<9*0 7)8O-*0=3O EA@7-*< 162*<@7-*< $V7! J=92 V<,I,<,35,9 7)8O-*09 7-*<Q>@ AB?U 76G8-0 769.,G9 N,)<=9.=5 7,*<5; V<-4)5.=-3 769.,G9 %3=H,<9*0 F,*+>#,.;-4 WDV7>Q J=92 F,*+ #,.;-49 7-*<A>@ AB?Q &-6>&*9+9 V<-80,G> 72*5,9

Soar

• Soar has serious

engineering.

• “Evolution of Soar”

is my favorite paper (Laird & Rosenbloom 1996)

• Admits problems!

!" !"#$%&'($&#)* +,-./ 0".%* 1-%/&"# 2.3"% 4-/(5$/ 67.895- 0:/$-8/ +895-8-#$.$&"# #$%&'$& ()*+,'-. /-*0 1,2,34,356 7)89.*., :-;,<,35, 1,5=9=-3>:650, 7-*<?>@ ABBB 7/C$ &*5D=<@7-*< EFD@7-*< CG2<-H,4 C3.,<I*5,9 &:JK&+ F<*22,< 7-*<L>@ ABBM D=<@7-*< C39.<)5.-@7-*< N=O; PII=5=,356 7-*<M>@ ABBQ : D=<@7-*< N,<-@7-*< PR.,<3*0 &*9+9 1,9.<)5.=H, $2,<*.-<9 7=3O0,>7.*., 7-*<!>@ AB?B P&@7-*< SJ@7-*< PR.,<3*0> E,0,*9, %&: 7-*<T>@ AB?M /,3,<*0 J,*<3=3O EA@7-*< :;)3+=3O 7-*<U>@ AB?T %3=H,<9*0 7)8O-*0=3O EA@7-*< 162*<@7-*< $V7! J=92 V<,I,<,35,9 7)8O-*09 7-*<Q>@ AB?U 76G8-0 769.,G9 N,)<=9.=5 7,*<5; V<-4)5.=-3 769.,G9 %3=H,<9*0 F,*+>#,.;-4 WDV7>Q J=92 F,*+ #,.;-49 7-*<A>@ AB?Q &-6>&*9+9 V<-80,G> 72*5,9

Soar

• Soar has serious

engineering.

• “Evolution of Soar”

is my favorite paper (Laird & Rosenbloom 1996)

• Admits problems!
• Not enough

applications for human-like AI

Architecture Lessons (from CMU)

• An architecture needs:
• action from perception, and
• further structure to combat

combinatorics.

• Dealing with time is hard.

ACT

• R
• Learns (& executes)

productions.

• For arbitration, rely
• n (Bayesian

probabalistic) utility.

• Call it implicit

knowledge.

ACT

• R Research

Programme

• Replicate lots of

Cognitive Science results.

• See if the brain

does what you think it needs to.

• Win Rumelhart

Prize (John Anderson, 2000).

Retrieval Buffer (VLPFC) Goal Buffer (DLPFC) Manual Motor (Motor) Intentional Module (not identified) External World Matching (Striatum) Execution (Thalamus) Selection (Pallidum) Productions (Basal Ganglia) Declarative Module (Temporal / Hippocampus) Visual Buffer (Parietal) Visual Module (Occipital/Parietal) Manual Module (Motor/Cerebellum)

Architecture Lessons (from CMU)

• Architectures need productions and

problem spaces.

• Real-time is hard.
• Being easy to use can be a win.

Architecture Lessons (from CMU)

• Architectures need productions and

problem spaces.

• Real-time is hard.
• Being easy to use can be a win.

Spreading Activation Networks

• “Maes

Nets” (Adaptive Neural Arch.; Maes 1989)

• Activation spreads

from senses and from goals through net of actions.

• Highest activated

Spreading Activation Networks

Spreading Activation Networks

• Sound good:
• easy
• brain-like (priming, action potential).
• Still influential (Franklin 2000,

Shanahan 2006).

Spreading Activation Networks

• Sound good:
• easy
• brain-like (priming, action potential).
• Still influential (Franklin 2000,

Shanahan 2006).

• Canʼt do full action selection:
• Donʼt scale; donʼt converge on

comsumatory acts (Tyrrell 1993).

Tyrrell (1993)

Extended Rosenblatt and Payton Free-Flow Hierarchy

N NE E SE S SW W NW

U T Reproduce

1.4

T U

Move Actions Mate

• 0.08

Court

• P. Mate
• Rand. Dir
• P. Den
• R. Den

All Dirs Clean Leave this Sq Clean Sleep Mate Court Approach Mate Explore For Mates Explore Sleep Approach

• P. Den

Approach

• R. Den

Sleep in Den Clean Keep

Dirtiness Low Health Night Prox from Den Distance

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

Courted Mate in Sq Mate in Sq Receptive

No Den in Sq Den in Sq No Den in Sq in Sq Den

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

= small negative activation = positive activation = small positive activation = zero activation = large positive activation (1.0)

Subsumption (Brooks 1986)

Subsumption (Brooks 1986)

• Emphasis on

sensing to action (via Augmented FSM).

Subsumption (Brooks 1986)

• Emphasis on

sensing to action (via Augmented FSM).

• Very complicated,

distributed arbitration.

Subsumption (Brooks 1986)

• Emphasis on

sensing to action (via Augmented FSM).

• Very complicated,

distributed arbitration.

• No learning.

Subsumption (Brooks 1986)

• Emphasis on

sensing to action (via Augmented FSM).

• Very complicated,

distributed arbitration.

• No learning.
• Worked.

Architecture Lessons (Subsumption)

Architecture Lessons (Subsumption)

• Action from perception can provide the

further structure -- modules (behaviors).

• Modules also support iterative

development / continuous integration.

Architecture Lessons (Subsumption)

• Action from perception can provide the

further structure -- modules (behaviors).

• Modules also support iterative

development / continuous integration.

• Real time should be a core organizing

principle -- start in the real world.

Architecture Lessons (Subsumption)

• Action from perception can provide the

further structure -- modules (behaviors).

• Modules also support iterative

development / continuous integration.

• Real time should be a core organizing

principle -- start in the real world.

• Good ideas can carry bad ideas a long

way (no learning, hard action selection).

Architecture Lesson?

• Goals ordering

needs to be flexible.

Architecture Lesson?

• Goals ordering

needs to be flexible.

• Maybe spreading

activation is good for this.

SA: Layers vs. Behaviours

• Relationship not

evident except in development!

SA: Layers vs. Behaviours

• Relationship not

evident except in development!

SA: Layers vs. Behaviours

• Relationship not

evident except in development!

SA: Layers vs. Behaviours

• Relationship not

evident except in development!

SA: Layers vs. Behaviours

• Relationship not

evident except in development!

Layered or Hybrid Architectures

• 1. Incorporate behaviors/modules (action

from sensing) as “smart” primitives.

• 2. Use hierarchical dynamic plans for

behavior sequencing.

• 3. (Allegedly) some have automated

planner to make plans for layer 2.

• Examples: Firby/RAPS/3T (ʻ97); PRS

(1992-2000); Hexmoore ʻ95; Gat ʻ91-98

Belief, Desires, Intentions (BDI)

• Beliefs:

Predicates

• Desires:

goals & related dynamic plans

• Intentions:

current goal

Procedural Reasoning System

Procedural Reasoning System

• BDI

Procedural Reasoning System

• BDI
• And reactive

(responds to emergencies by changing intentions.)

Procedural Reasoning System

• BDI
• And reactive

(responds to emergencies by changing intentions.)

• Er... once or

twice (Bryson ATAL 2000).

Architecture Lessons

• Structured dynamic plans make it easier to

get your robot to do complicated stuff.

• Automated planning (or for Soar, chunking/

learning) is seldom actually used.

• To facilitate that automated planning,

modularity is often compromised. (Bryson JETAI 2000, Brom & Bryson 2006)

Soar as a 3LA

• J. Laird & P.

Rosenbloom, “The Evolution of the Soar Cognitive Architecture”, Mind Matters,

• D. Steier and
• T. Mitchell

eds., 1996.

CogAff

• Reflection on Top.
• Sense & Action

separated!

• (Davis & Sloman

1995)

• Reflection on Top.
• Sense & Action

separated!

• Hierarchy in AS;

Goal Swapping (Alarms).

• (Sloman 2000)

CogAff

• Reflection on Top.
• Sense & Action

separated!

• Hierarchy in AS,

Goal Swapping (now reactive).

• Current Web

CogAff

Separate Sense & Action

• Something we

higher mammals do.

• Central Sulcus

Chance for Cognition?

(pictures from Carlson)

Architecture Lessons (CogAff)

Architecture Lessons (CogAff)

• Maybe you don’t really want productions as

your basic representation -- you may want to come between a sense and an act sometimes.

Architecture Lessons (CogAff)

• Maybe you don’t really want productions as

your basic representation -- you may want to come between a sense and an act sometimes.

• Your architecture looks very different if you

really worry about adult human literary- level behaviour rather than just making something work.

Outline

• Introduction

Intelligence, Cognition & Architecture

• A Brief History of AI Cognitive

Architectures

• Behavior Oriented Design

Outline

• Introduction

Intelligence, Cognition & Architecture

• A Brief History of AI Cognitive

Architectures

• Behavior Oriented Design

Conclusion / Recommendations

Behavior Oriented Design

• All search (learning, planning) is done within

modules with specialized representations.

• Specialized representations promote reliability
• f search; also determine decomposition.
• Modules provide perception, action, memory.

Arbitration via hierarchical dynamic plans.

• Iterative / agile test & development cycle.

(Bryson 2001, 2003)

BOD Applications

(ATAL 1997)

BOD Applications

(ATAL 1997) (SAB 2000)

• bserve predator

BOD Applications

• 0.08

• P. Mate
• Rand. Dir
• P. Den
• R. Den

• P. Den

• R. Den

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

(ATAL 1997) (VR(J) 2000) (SAB 2000)

• bserve predator

BOD Applications

• 0.08

• P. Mate
• Rand. Dir
• P. Den
• R. Den

• P. Den

• R. Den

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

(ATAL 1997) (VR(J) 2000) (SAB 2000)

• bserve predator

BOD Applications

• 0.08

• P. Mate
• Rand. Dir
• P. Den
• R. Den

• P. Den

• R. Den

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

• Action

• monkey

• sequence

• rule-learner

• look-at
• (Animal Cog 2007

CogSci 2009)

(ATAL 1997) (VR(J) 2000) (SAB 2000)

• bserve predator

BOD Applications

• 0.08

• P. Mate
• Rand. Dir
• P. Den
• R. Den

• P. Den

• R. Den

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

• Action

• monkey

• sequence

• rule-learner

• look-at
• (Animal Cog 2007

CogSci 2009) (WRAC 2003, PTRS B 2007, BICA 2008)

(ATAL 1997) (VR(J) 2000) (SAB 2000)

• bserve predator

BOD Applications

• 0.08

• P. Mate
• Rand. Dir
• P. Den
• R. Den

• P. Den

• R. Den

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

• Action

• monkey

• sequence

• rule-learner

• look-at
• (Animal Cog 2007

CogSci 2009) (WRAC 2003, PTRS B 2007, BICA 2008) (IVA 2005, CGames 2006 IEEE SMC 2007)

Modularity is not Enough

Get Fuzzy (Conley 2006)

BOD Action Selection

Parallel-rooted, Ordered, Slip-stack Hierarchical (POSH) action selection:

• Some things need to be checked at all times:

drive collection.

• Some things only need considering in

particular context: competences.

• Some things reliably follow from others:

action patterns.

POSH plan in ABODE

(for UT: Capture the Flag)

• Advanced BOD Environment.
• Initial development funded by industry.

Current Work

• Drive / Emotion level work with latching

not adequate for reprioritizing goals.

• Thinking about interrupts (Brom 2007;

Norman & Shallice 1988) -- ‘spreading activation’ just for goals.

• Trying to make ABODE a real IDE.
• Modelling primate social behaviour.

What I Learned from Robots

• 1. Perception is hard (explains the brain).
• Lead to specialized representations

encapsulated in modules; my method of behavior-module decomposition.

• 2. Discrete action selection is compatible with

continuous acting, provided the primitive `acts’ alter ongoing behaviour supported by modules.

• e.g. motor act sends target velocity, not vector;
• multiple || devices/modules e.g. speech, motion.

Outline

• Introduction

Intelligence, Cognition & Architecture

• A Brief History of AI Cognitive

Architectures

• Behavior Oriented Design

Conclusion / Recommendations

Architecture Lessons

• Modularity: problem spaces, combat

combinatorics, allow locally-optimal representations.

• Should use ordinary (OO) code (arbitrarily

powerful but also access to primitives.)

• Hierarchical action selection for arbitration.
• Dedicated, high-frequency goal / attention

switching, compensates for hierarchical AS.

• Agile development, refactoring (Beck 2000).

What Do We Really Need from Architectures?

• Development methodologies:
• Describe ontologies / representations;
• Recommend development strategies.

We need to help the average programmer.

Joanna J. Bryson

University of Bath, United Kingdom

Cognition for Intelligent Robotics

Architectures and Action Selection

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.

• 1. Specify (high-level) what the agent will do.
• 2. Describe activities as sequences of actions.

competences and action patterns

• 3. Identify sensory and action primitives from

these sequences.

• 4. Identify the state necessary to enable the

primitives, cluster primitives by shared

• state. behavior modules
• 5. Identify and prioritize goals / drives. drive

collection

• 6. Select a first (next) behavior to implement.

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.

Simplify the Design

Use the simplest representations.

• Plans:
• primitives, action patterns, competences.
• drives only if need to always check.
• Behavior modules / memory:
• none, deictic, specialized, general.

(Bryson, AgeS 2003)

Simplify the Design

Trade off representations: plans vs. behaviors

• Use simplest plan structure unless

redundancy (split primitives for sequence, add variable state in modules).

• If competences too complicated, introduce

primitives or create more hierarchy.

• Split large behaviors, use plans to unify.
• All variable state in modules (deictic).

(Bryson, AgeS 2003)

life (D) untangle (tangled?) untangle groom (C) (want-to-groom?) (partner-chosen?) (aligned?) notify groom (being-groomed?) choose-groomer-as-partner (partner-chosen?) (touching?) notify align (partner-chosen?) notify approach (⊤) choose-partner receive (being-groomed?) tolerate-grooming explore (C) (want-novel-loc?) (place-chosen?) (there-yet?) lose-target (place-chosen?) explore-that-a-way (⊤) choose-explore-target wait (⊤) wait

Talk Outline

• Why & How Time Matters
• Emotions as Memory
• Drives and Flexible Latching
• Goals, Memory and Action Selection in

Competitive Play.

• Crude, Cheesy Second-Rate Consciousness

Why Time Matters

Combinatorics: You can’t think of everything (Simon 1972; Chapman 1987; Sipser 2005).

Combinatorics

• If . . .

– an agent knows 100 actions (e.g. eat, drink, sleep, step, turn, lift, grasp, poke, flip...), and – it has a goal (e.g. go to Madagascar)

• Then . . .

Combinatorics

• If . . .

– an agent knows 100 actions (e.g. eat, drink, sleep, step, turn, lift, grasp, poke, flip...), and – it has a goal (e.g. go to Madagascar)

• Then . . .

– Finding a one-step plan may take 100 acts. – A two-step plan may take 1002 (10,000). – For unknown number of steps, may search forever, missing critical steps or sequence.

Memory & Time

• More recent information is probably more

important, but...

• Very recent information can be incomplete.
• Hard to understand,
• sensing != perception.
• Better interpreted in light of experience.
• Recent experience is a part of context.

Memory’s Role in Context

• Recent events:
• episodic memory,
• emotions.
• “Knowledge”:
• facts,
• expectations.

Memory’s Role in Context

• Recent events:
• episodic memory,
• emotions.
• “Knowledge”:
• facts,
• expectations.

These fade, get replaced.

}

Memory’s Role in Context

• Recent events:
• episodic memory,
• emotions.
• “Knowledge”:
• facts,
• expectations.

These fade, get replaced.

}

These only build (more or less).

}

Example: Emotions as Memory

Tanguy (2006)

I’ve got good news and bad news... Code & video available online.

(Tanguy, Bryson & Willis 2007; Bryson & Tanguy 2009)

• Mood — long term.
• Emotions — shorter

term.

• Behaviour (e.g.

expressions) is altered by these.

• simplifies coding,
• increases variability.

Memory

Talk Outline

• Why & How Time Matters
• Emotions as Memory
• Drives and Flexible Latching
• Goals, Memory and Action Selection in

Competitive Play.

• Crude, Cheesy Second-Rate Consciousness

Improved Animal-Like Maintenance of Homeostatic Goals via Flexible Latching

Philipp Rohlfshagen❄ and Joanna J. Bryson

BICA @ AAAI Fall Symposia 2008

This work was supported by

EPRSC grant GR/S79299/01

❄Now of the Centre of Excellence for Research in Computational

Intelligence and Applications at the University of Birmingham

• In simulations of animal behaviour ...
• agents interact with the environment and one another
• agents need to carry out a set of tasks
• agents need to ensure their survival.
• Some behaviours are essential to the survival of the ...
• individual (e.g. obtain sufficient energy by means of food and drink)
• species (e.g. grooming, mating)
• One of the key questions:
• How to coordinate priorities to ensure survival?
• Our work is about latching ...
• a general mechanism to efficiently coordinate different priorities

• Agents are layered or hybrid and consist of ...
• modules that specify details of their behaviour,
• dynamic plans that specify cross-modular prioritisation.
• The behaviour of agents is driven by
• Parallel-rooted, Ordered Slip-stack Hierachical dynamic plans.
• We assume each agent has ...
• some internal storage for long-term states,
• the ability to express goals and their associated actions,
• the notion of a trigger for each behaviour.

BOD & POSH

• 1. No latch
• 2. Strict latch
• Trigger behaviour if internal state is below δ
• Maintain behaviour until internal state is above φ ≥ δ
• 3. Strict latching with interruptions; can be very inefficient
• Agents may persevere for minimum gain
• Inefficiency first identified by Hagen Lehmann
• 4. Flexible latch:
• Introduce a third threshold, ψ such that δ ≤ ψ ≤ φ
• Behaviour is triggered as before but if agent is interrupted:
• if internal state is below ψ: continue,
• therwise: reset latch

Experimental Conditions

((SDC life (goal (s-one_step (s-succeed 0))) (drives ((dead (trigger((s-is_dead 0))) a_stay_dead)) ((drink (trigger((s-wants_drink))) a-drink) (eat (trigger((s-wants_food))) c-eat)) ((groom (trigger((s-wants_to_groom))) c-groom)) ((explore (trigger((s-succeed))) a-explore)))) (C a-groom (goal ((s-succeed 0))) (elements ((has-no-target (trigger((s-has_groom_target 0))) a-pick_groom_target)) ((not-near-target (trigger((s-is_near_groom_target 0))) a-move_to_groom_target)) ((default-groom (trigger((s-succeed))) a-groom_with_target)))) (C a-eat (goal ((s-succeed 0))) (elements ((has-no-food (trigger((s-has_food 0))) a-pick_food)) ((not-near-target (trigger((s-is_near_food_target 0))) a-move_to_food)) ((default-feeding (trigger((s-succeed))) a-eat)))) (C a-drink (goal ((s-succeed 0))) (elements ((has-no-drink (trigger((s-has_drink 0))) a-pick_drink)) ((not-near-target (trigger((s-is_near_drink_target 0))) a-move_to_drink)) ((default-feeding (trigger((s-is_near_drink_target))) a-drink)))))

No Latch

Strict Latch (no interrupts)

Strict Latch (interrupts)

Flexible Latch

• Test and compare all variants
• Check frequency of execution of

low-priority goals

• Also frequency ratio of primary

and secondary actions

• Two simulation settings
• Controlled environment
• Random (more realistic

environment)

Experiments

          

Example Experiment

Results

  



     



   

Results

There’s lots more tables in the paper... and guidance on setting thresholds.

Conclusion

Conclusion

  



     



   

Talk Outline

• Why & How Time Matters
• Emotions as Memory
• Drives and Flexible Latching
• Goals, Memory and Action Selection in

Competitive Play.

• Crude, Cheesy Second-Rate Consciousness

How & Why Time Matters

Combinatorics: You can’t think of everything (Simon 1972; Chapman 1987; Sipser 2005).

• Memory
• Sequencing
• Pursuit of Goals

How & Why Time Matters

Combinatorics: You can’t think of everything (Simon 1972; Chapman 1987; Sipser 2005).

• Memory
• Sequencing
• Pursuit of Goals

Action Selection

}

Modularity is not Enough

Get Fuzzy (Conley 2006)

Action Selection: Sequencing

• Sequencing matters only when there is a

constraining resource (Blumberg 1996).

• Example constraints: what’s in a hand,

where character standing, what it’s saying.

• Counter examples (potentially

concurrent): perception, memory, autonomic processes.

Parallel-rooted, Ordered, Slip-stack Hierarchical (POSH) plans (Bryson 2001a,b;2003; et al 2005):

Action Selection by Dynamic Planning

• Some things need to be checked

at all times: drive collection.

• Some things only need

considering in particular context: competences.

• Some things reliably follow from
• thers: action patterns.

Parallel-rooted, Ordered, Slip-stack Hierarchical (POSH) plans (Bryson 2001a,b;2003; et al 2005):

Action Selection by Dynamic Planning

• Some things need to be checked

at all times: drive collection.

• Some things only need

considering in particular context: competences.

• Some things reliably follow from
• thers: action patterns.

— Goals

Parallel-rooted, Ordered, Slip-stack Hierarchical (POSH) plans (Bryson 2001a,b;2003; et al 2005):

Action Selection by Dynamic Planning

• Some things need to be checked

at all times: drive collection.

• Some things only need

considering in particular context: competences.

• Some things reliably follow from
• thers: action patterns.

— Goals — Sequences

Parallel-rooted, Ordered, Slip-stack Hierarchical (POSH) plans (Bryson 2001a,b;2003; et al 2005):

Action Selection by Dynamic Planning

• Some things need to be checked

at all times: drive collection.

• Some things only need

considering in particular context: competences.

• Some things reliably follow from
• thers: action patterns.

— Goals — Sequences — Sub-goals generating custom sequences

Behavior Oriented Design

• Concurrent modules provide perception,

action, memory.

• Specialized representations promote

reliability of search; determine module decomposition.

• Action Selection via hierarchical dynamic plans.
• Iterative / agile test & development cycle.

(Bryson 2001, 2003)

BOD Applications

(ATAL 1997)

BOD Applications

(ATAL 1997) (SAB 2000)

• bserve predator

BOD Applications

• 0.08

• P. Mate
• Rand. Dir
• P. Den
• R. Den

• P. Den

• R. Den

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

(ATAL 1997) (VR(J) 2000) (SAB 2000)

• bserve predator

BOD Applications

• 0.08

• P. Mate
• Rand. Dir
• P. Den
• R. Den

• P. Den

• R. Den

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

(ATAL 1997) (VR(J) 2000) (SAB 2000)

• bserve predator

BOD Applications

• 0.08

• P. Mate
• Rand. Dir
• P. Den
• R. Den

• P. Den

• R. Den

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

• Action

• monkey

• sequence

• rule-learner

• look-at
• (Animal Cog 2007)

(ATAL 1997) (VR(J) 2000) (SAB 2000)

• bserve predator

BOD Applications

• 0.08

• P. Mate
• Rand. Dir
• P. Den
• R. Den

• P. Den

• R. Den

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

• Action

• monkey

• sequence

• rule-learner

• look-at
• (Animal Cog 2007)

(WRAC 2003, PTRS B 2007, BICA 2008)

(ATAL 1997) (VR(J) 2000) (SAB 2000)

• bserve predator

BOD Applications

• 0.08

• P. Mate
• Rand. Dir
• P. Den
• R. Den

• P. Den

• R. Den

• 0.10
• 0.05
• 0.01
• 0.05
• 0.05
• 0.15

• 0.02
• 0.02
• 0.25
• 0.30
• 0.04

• Action

• monkey

• sequence

• rule-learner

• look-at
• (Animal Cog 2007)

(WRAC 2003, PTRS B 2007, BICA 2008) (IVA 2005, CGames 2006 IEEE SMC 2008)

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.
• 4. Iterate.

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.
• 4. Iterate.

• 1. Specify (high-level) what the agent will do.
• 2. Describe activities as sequences of actions.

competences and action patterns

• 3. Identify sensory and action primitives from

these sequences.

• 4. Identify the state necessary to enable the

primitives, cluster primitives by shared

• state. behavior modules
• 5. Identify and prioritize goals / drives. drive

collection; emotions / durative state

• 6. Select a first (next) behavior to implement.

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.
• 4. Iterate.

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.
• 4. Iterate.

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.
• 4. Iterate.

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.
• 4. Iterate.

Simplify the Design

Use the simplest representations.

• Plans:
• primitives, action patterns, competences.
• drives only if need to always check.
• Behavior modules / memory:
• none, deictic, specialized, general.

(Bryson, AgeS 2003)

Simplify the Design

Trade off representations: plans vs. behaviors

• Use simplest plan structure unless

redundancy (split primitives for sequence, add variable state in modules).

• If competences too complicated, introduce

primitives or create more hierarchy.

• Split large behaviors, use plans to unify.
• All variable state in modules (deictic).

(Bryson, AgeS 2003)

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.
• 4. Iterate.

BOD Development Cycle

• 1. Initial decomposition ⇒ specification.
• 2. Scale the system.
• i. Code one behavior and/or plan.
• ii. Test and debug code (test earlier plans).
• iii. Simplify the design.
• 3. Revise the specification.
• 4. Iterate.

Example: Sequencing & Goals

(Partington & Bryson 2005) Thanks: Binns, Mansfield, Drugowitsch, Brom et al.

Partington’s Video (ask me for a live demo)

Partington’s Video (ask me for a live demo)

AS Summary

• Combinatorics:

You can’t think of everything.

• Memory:
• learning involves forgetting.
• Sequencing and Goals: action selection by

hierarchical dynamic planning.

Talk Outline

• Why & How Time Matters
• Emotions as Memory
• Drives and Flexible Latching
• Goals, Memory and Action Selection in

Competitive Play.

• Crude, Cheesy Second-Rate Consciousness