EmbodiedIntelligence whenBRAIN,BODY,andENVIRONMENTinteract - - PowerPoint PPT Presentation

Embodied
Intelligence


when
BRAIN,
BODY,
and
ENVIRONMENT
interact


Mohamed
Oubba?


March,
2012


1
 Dr.
Oubba?,
March
2012


Embodied
Intelligence


when
BRAIN,
BODY,
and
ENVIRONMENT
interact


2
 Dr.
Oubba?,
March
2012


Intelligence


3
 Dr.
Oubba?,
March
2012


Can
we
design
intelligence?


4
 Dr.
Oubba?,
March
2012


I
mean
human‐like
intelligence,
not
 just
the
“appearance”
of
intelligence


Can
we
design
intelligence?


5
 Dr.
Oubba?,
March
2012


I
mean
human‐like
intelligence,
not
 just
the
“appearance”
of
intelligence


YES



Can
we
design
intelligence?


6
 Dr.
Oubba?,
March
2012


Can
we
design
intelligence?


I
mean
human‐like
intelligence,
not
 just
the
“appearance”
of
intelligence


YES



BUT


1.
we
need
to
know
how.
 2.
we
need
tools
to
implement
this
know‐how.


7
 Dr.
Oubba?,
March
2012


I
mean
human‐like
intelligence,
not
 just
the
“appearance”
of
intelligence


YES



BUT


1.
we
need
to
know
how.
 2.
we
need
tools
to
implement
this
know‐how.


This
talk
is
about
 these
two
points


Can
we
design
intelligence?


8
 Dr.
Oubba?,
March
2012


What
is
intelligence?


In
1921,
the
Journal
of
Educa?onal
Psychology
 asked
14
experts
for
defini?ons


9
 Dr.
Oubba?,
March
2012


What
is
intelligence?


Result:
14
different
defini?ons!


In
1921,
the
Journal
of
Educa?onal
Psychology
 asked
14
experts
for
defini?ons


10
 Dr.
Oubba?,
March
2012


What
is
intelligence?
 Intelligence
is
difficult
to
define



and
understand


11
 Dr.
Oubba?,
March
2012


What
is
intelligence?
 Intelligence
is
difficult
to
define



and
understand
 Maybe
one
should
try
to
build
ar?ficial
 intelligent
systems
in
order
to
understand
 intelligence


12
 Dr.
Oubba?,
March
2012


Ar?ficial
Intelligence


13
 Dr.
Oubba?,
March
2012


• “
…the
study
of
how
to
make
computers
do
things
at


which,
at
the
moment,
people
are
beber”
 A.I.
research
is



(Rich
&
Knight,
1991)


• “…the
art
of
crea?ng
machines
that
perform
func?ons


that
require
intelligence
when
performed
by
people“



(Kurzweil,
1990)


• The
modern
defini?on
of
A.I.
is
“the
study
and
design

• f
intelligent
agents”


Ar?ficial
Intelligence
 14
 Dr.
Oubba?,
March
2012


The
current
AI
has
produced
success
in
the
narrow:

 well‐defined
problems,
with
verifiable
results,
in
 predictable
environments,
using
well‐understood
 techniques.


Ar?ficial
Intelligence
 15
 Dr.
Oubba?,
March
2012


In
1997,
Deep
Blue
beat
Kasparov.


Some
AI programs
are
extremely
successful
at
 what
they
are
specialized.


Ar?ficial
Intelligence
 16
 Dr.
Oubba?,
March
2012


‐ 
natural
language
understanding
 ‐ 
general
vision
 ‐ 
understanding
a
movie
 ‐ 
wri?ng
 Although
many
AI
systems
are
beyond
human‐level
 competence
at
some
individual
tasks,
AI
has
failed
 (miserably)
in
many
domains
like


Ar?ficial
Intelligence


Failure
of
the
current
AI


17
 Dr.
Oubba?,
March
2012


To
overcome
the
current
AI
problems
we
 must
return
back
to
the
original
goal
of
AI:
 „create
systems
that
are
humanly
intelligent“



Ar?ficial
Intelligence


Failure
of
AI


18
 Dr.
Oubba?,
March
2012


To
overcome
the
current
AI
problems
we
 must
return
back
to
the
original
goal
of
AI:
 „create
systems
that
are
humanly
intelligent“



Ar?ficial
Intelligence


Failure
of
AI


aims
to
create
a
computer‐based
intelligence
at
least
as
 smart
as
a
human.


(since
2001)


19
 Dr.
Oubba?,
March
2012


20
 Dr.
Oubba?,
March
2012


• Conventional AI
(also
called
Narrow
AI,
Applied
AI,
Weak
AI

• r
Classic
AI)


(also
called
Strong
AI
,
Human‐level
AI,
Real
AI,…)


Conventional AI
vs.
AGI




Intelligence
is
the
ability
to
achieve
pre‐defined
goals,
 under
constraints.
 Intelligence
is
the
ability
to
expand
and
develop
new
 abili?es.


21
 Dr.
Oubba?,
March
2012


aims
to
create
machines
that
act
as
if
they
were
 intelligent,
i.e.
the
appearance
of
intelligence.
 aims
to
create,
approach,
and
exceed
human
intelligence.


Conventional AI
vs.
AGI




22
 Dr.
Oubba?,
March
2012


• Conventional AI
(also
called
Narrow
AI,
Applied
AI,
Weak
AI

• r
Classic
AI)


(also
called
Strong
AI
,
Human‐level
AI,
Real
AI,…)


Conventional A.I.

PROBLEM
 SPACE


[Adapted
from
Dr.
Goertzel]


Conventional AI
vs.
AGI




23
 Dr.
Oubba?,
March
2012



most
people
work
on
conventional AI


Conventional AI
vs.
AGI




24
 Dr.
Oubba?,
March
2012



most
people
work
on
conventional AI


...maybe
because
working
on
AGI
is
risky:


• 1. Maybe
it
is

“impossible”.

• 2. It
is
difficult
to
get
funding
for
risky


„ambi?ous“
research
projects
that
aim
to
go
 towards
human‐level
intelligence.


• 3. Achieving
human‐level
intelligence
means
the


possibility
to
achieve
Superintelligence
that
 could
be
out
of
control.


Conventional AI
vs.
AGI




25
 Dr.
Oubba?,
March
2012


26
 Dr.
Oubba?,
March
2012


I
consider
the
following
ques?ons


27
 Dr.
Oubba?,
March
2012


28
 Dr.
Oubba?,
March
2012


YES


29
 Dr.
Oubba?,
March
2012


YES


we
have
a
solid
proof
that
it
is
possible
—
namely


• urselves.



30
 Dr.
Oubba?,
March
2012


31
 Dr.
Oubba?,
March
2012


32
 Dr.
Oubba?,
March
2012


How
can
a
robot
learn
like
a
child?


33
 Dr.
Oubba?,
March
2012


How
can
a
robot
learn
like
a
child?
 
by
trying
to
reconstruct
the
processes
of
 development
of
the
child's
mind.


34
 Dr.
Oubba?,
March
2012


Child
Development


35
 Dr.
Oubba?,
March
2012


Child
development
is
a
 process
every
child
goes
 through.
 It
includes
 physical
development,
personal
development,

 social
development,
and
cogni?ve
development.


child
development


36
 Dr.
Oubba?,
March
2012


Child
development
is
a
 process
every
child
goes
 through.
 It
includes
 physical
development,
personal
development,

 social
development,
and
cogni?ve
development.


child
development


37
 Dr.
Oubba?,
March
2012


From
birth,
babies
begin
to
ac?vely
gather
and
 process
informa?on
from
the
surrounding
world,
 and
use
the
data
to
develop
percep?on
and
 thinking
skills.



child
development


Cogni?ve
Development


38
 Dr.
Oubba?,
March
2012


years


World
0
 World
1
 World
2
 World
3


Cogni?ve
Development


The
process
of
understanding
the
world
changes
as
a
 func?on
of
age
&
experience


child
development


39
 Dr.
Oubba?,
March
2012


years


World
0
 World
1
 World
2
 World
3


Cogni?ve
Development


The
process
of
understanding
the
world
changes
as
a
 func?on
of
age
&
experience


How
does
this
happen?


child
development


40
 Dr.
Oubba?,
March
2012


Jean
Piaget
(1896‐1980)

 (Psychologist)
 Lev
Vygotsky
(1896‐1934)
 (Psychologist)


How
children’s
cogni?on
develop?


Two
typical
sources
of
inspira?on


child
development


41
 Dr.
Oubba?,
March
2012


Jean
Piaget
(1896‐1980)

 (Psychologist)


Piaget
was
interested
in
how
a
child
 adapts
to
its
environment.

 (he
described
this
ability
as
intelligence)


child
development


42
 Dr.
Oubba?,
March
2012


Jean
Piaget
(1896‐1980)

 (Psychologist)


Piaget
was
interested
in
how
a
child
 adapts
to
its
environment.

 (he
described
this
ability
as
intelligence)


Adapta?on


Assimila?on
 Accomoda?on


child
development


43
 Dr.
Oubba?,
March
2012


Assimila?on
 Adding
new
 experiences
to
the
 exis?ng
ones,
and
 reinforce
them.


child
development


Accommoda?on
 Adjus?ng
exis?ng
 experiences
to
deal
 with
new
situa?ons.


Piaget’s
assump?ons


44
 Dr.
Oubba?,
March
2012


Development
takes
place
in
dis?nct
stages
of
cogni?ve
development.


child
development


Piaget’s
assump?ons


45
 Dr.
Oubba?,
March
2012


Development
takes
place
in
dis?nct
stages
of
cogni?ve
development.
 Sensorimotor


(~Birth to 2 yrs) achieving
simple
goals
in
 a
sensorimotor
space


child
development


Piaget’s
assump?ons


46
 Dr.
Oubba?,
March
2012


Development
takes
place
in
dis?nct
stages
of
cogni?ve
development.
 Sensorimotor


(~Birth to 2 yrs) achieving
simple
goals
in
 a
sensorimotor
space


Preopera?onal


(~2 to 6 yrs) Represen?ng
the
world,
and
go
beyond

 the
connec?on
of
sensorimotor
 informa?on.


child
development


Piaget’s
assump?ons


47
 Dr.
Oubba?,
March
2012


Development
takes
place
in
dis?nct
stages
of
cogni?ve
development.
 Sensorimotor


(~Birth to 2 yrs) achieving
simple
goals
in
 a
sensorimotor
space


Preopera?onal


(~2 to 6 yrs) Represen?ng
the
world,
and
go
beyond

 the
connec?on
of
sensorimotor
 informa?on.


Concrete
Opera?onal


(~ 7-12 yrs) reasoning
logically
about
events.


child
development


Piaget’s
assump?ons


48
 Dr.
Oubba?,
March
2012


Development
takes
place
in
dis?nct
stages
of
cogni?ve
development.
 Sensorimotor


(~Birth to 2 yrs) achieving
simple
goals
in
 a
sensorimotor
space


Preopera?onal


(~2 to 6 yrs) Represen?ng
the
world,
and
go
beyond

 the
connec?on
of
sensorimotor
 informa?on.


Concrete
Opera?onal


(~ 7-12 yrs) reasoning
logically
about
events.


Formal
Opera?onal


(adolescence to adult) reasoning
in
a
more
abstract,
 idealis?c,
and
logical
way.


child
development


Piaget’s
assump?ons


49
 Dr.
Oubba?,
March
2012


Piaget
did
not
emphasize
on
the
role


• f
social
interac?on.


child
development


Piaget’s
assump?ons


50
 Dr.
Oubba?,
March
2012


Lev
Vygotsky
(1896‐1934)
 (Psychologist)


Most
of
a
child’s
 cogni?ve
skills
evolve
 from
social
interac?ons


child
development


51
 Dr.
Oubba?,
March
2012


Lev
Vygotsky
(1896‐1934)
 (Psychologist)


Most
of
a
child’s
 cogni?ve
skills
evolve
 from
social
interac?ons


child
development


Development
occurs
within
the

 „zone
of
proximal
development“


52
 Dr.
Oubba?,
March
2012


range
of
tasks
that
are
 too
complex
to
be
 mastered
alone
but
can
 be
accomplished
with
 guidance.
 Vygotsky’s
assump?on
 
Zone
of
Proximal
Development


child
development


53
 Dr.
Oubba?,
March
2012


Child
can
 Child
cannot


Zone
of
 Proximal
 Development


Vygotsky’s
assump?on
 
Zone
of
Proximal
Development
 Learning
/
Development


54


child
development


55
 Dr.
Oubba?,
March
2012


56


Social
engagement
 Individual
capabili?es

 
 
Zone
of
Proximal
Development


Sensorimotor
 Preopera?onal
 Concrete

 Opera?onal
 Formal

 Opera?onal


Dr.
Oubba?,
March
2012


…so
how
can
we
create
a
system
 that
exhibits
autonomous
 development?


57
 Dr.
Oubba?,
March
2012


…so
how
can
we
create
a
system
 that
exhibits
autonomous
 development?


We
need
„informal“
defini?ons
for
intelligence,
 anyway…..


58
 Dr.
Oubba?,
March
2012


…so
how
can
we
create
a
system
 that
exhibits
autonomous
 development?


I
will
call
them
signs
of
intelligence
 We
need
„informal“
defini?ons
for
intelligence,
 anyway…..


59
 Dr.
Oubba?,
March
2012


• Learn
(understand)
from
experience.

• Use
reasoning
to
solve
problems.

• Adapt
quickly
and
successfully
to
new
situa?ons.

• Use
the
acquired
knowledge
to
manipulate
the


environment.


[R. J. Sternberg, editor. Handbook of Intelligence. Cambridge University Press, 2000.]

[U. Neisser, et. al., Intelligence: Knowns and unknowns. American Psychologist, 51(2):77–101, 1996.]

Signs
of
Intelligence



(possibly
their
are
many
other)


How
to
create
autonomous
development?


60
 Dr.
Oubba?,
March
2012


• Learn
(understand)
from
experience.

• Use
reasoning
to
solve
problems.

• Adapt
quickly
and
successfully
to
new
situa?ons.

• Use
the
acquired
knowledge
to
manipulate
the


environment.
 Signs
of
Intelligence



(possibly
their
are
many
other)


How
to
create
autonomous
development?


61
 Dr.
Oubba?,
March
2012


The
most
common
idea
of
these
signs
can
be
 summarized
as


intelligence
is
a
property
of
a
system
that
is
 interac?ng
successfully
with
a
problem,
or
 an
environment.
 Signs
of
Intelligence



How
to
create
autonomous
development?


62
 Dr.
Oubba?,
March
2012


To
create
autonomous
development
we
have
to
 design

1)
a
developmental
learning
process,
2)
 that
interacts
with
real
environments.


How
to
create
autonomous
development?


63
 Dr.
Oubba?,
March
2012


control
 architecture



That
can
learn
and
 develop


Environment


That
supports
this
 development


Brain


Physical
 body


Interac?ng
with
the
environment
requires
 a
physical
body


environment


How
to
create
autonomous
development?


64
 Dr.
Oubba?,
March
2012


How
important
is
the
body
to
 development?


How
to
create
autonomous
development?


65
 Dr.
Oubba?,
March
2012


The
body
facilitates
 learning
from
the
 physical
environment


iCub‐robot
(EU‐projects)


The
role
of
the
body


Explora?on,
 Experimenta?on,...


How
to
create
autonomous
development?


66
 Dr.
Oubba?,
March
2012


Robot
Kismet
(MIT)


Imita?on,
Learning
 from
demonstra?on,
 etc.


The
body
facilitates
 learning
from
social
 environment


The
role
of
the
body


How
to
create
autonomous
development?


67
 Dr.
Oubba?,
March
2012


But
the
actual
„classic“
robo?cists
 (conven?onal
AI)
never
disputed
the
 importance
of
the
physical
body
to
their
 research.


The
role
of
the
body


How
to
create
autonomous
development?


68
 Dr.
Oubba?,
March
2012


Some
people
believe
that
 just
connec?ng
a
computer
 to
sensors
and
placing
it
on
 wheels
is
sufficient
to
create
 the
„body“
which
is
required
 for
intelligence.


The
role
of
the
body


How
to
create
autonomous
development?


69
 Dr.
Oubba?,
March
2012


Intelligence
requires
a
body
which
has
to
be
able
to
 embody
cogni?ve
processes,
that
affects
and
be
 affected
by
the
environment.


The
role
of
the
body


Some
people
believe
that
 just
connec?ng
a
computer
 to
sensors
and
placing
it
on
 wheels
is
sufficient
to
create
 the
„body“
which
is
required
 for
intelligence.


How
to
create
autonomous
development?


70
 Dr.
Oubba?,
March
2012


Intelligence
requires
a
body
which
has
to
be
able
to
 embody
cogni?ve
processes,
that
affects
and
be
 affected
by
the
environment.
 This
is
the
no?on
of
Embodiment


The
role
of
the
body


Some
people
believe
that
 just
connec?ng
a
computer
 to
sensors
and
placing
it
on
 wheels
is
sufficient
to
create
 the
„body“
which
is
required
 for
intelligence.


How
to
create
autonomous
development?


71
 Dr.
Oubba?,
March
2012


Embodiment
vs.
Body
 The
fundamental
difference
lies
in
the
degree


• f
interac?on
between
the
body
and
the



brain
(control
program).


The
role
of
the
body


How
to
create
autonomous
development?


72
 Dr.
Oubba?,
March
2012


Let’s
consider
an
embodied
agent...


The
role
of
the
body


How
to
create
autonomous
development?


Dr.
Oubba?,
March
2012


assume
that
this
agent
is
interac?ng
 with
a
complex
environment


The
role
of
the
body


How
to
create
autonomous
development?


Dr.
Oubba?,
March
2012


Conventional AI
seeks
an
exact
representa?on
of
the
 external
world
in
order
to
achieve
the
desired
task.


The
role
of
the
body


How
to
create
autonomous
development?


Dr.
Oubba?,
March
2012


The
role
of
the
body


The
Body
here
 is
used
just
as
 an
observer...


Conventional AI
seeks
an
exact
representa?on
of
the
 external
world
in
order
to
achieve
the
desired
task.


How
to
create
autonomous
development?


Dr.
Oubba?,
March
2012


The
role
of
the
body


However,
the
external
world
is
too
complex,
and
 requires
not
just
con?nual
observa?on,
but
also
 reasoning
and
adapta?on.


The
Body
here
 is
used
just
as
 an
observer...


Conventional AI
seeks
an
exact
representa?on
of
the
 external
world
in
order
to
achieve
the
desired
task.


How
to
create
autonomous
development?


Dr.
Oubba?,
March
2012


The
role
of
the
body


The
Body
here
 is
used
just
as
 an
observer...


Furthermore,
Knowledge
(about
the
environment)
 is
not
everything.



Conventional AI
seeks
an
exact
representa?on
of
the
 external
world
in
order
to
achieve
the
desired
task.


How
to
create
autonomous
development?


Dr.
Oubba?,
March
2012


The
role
of
the
body


Only
by
learning
from
interac?on
the
agent
will

 be
able
to
understand
the
environment,
and
use
 this
knowlege
to
achieve
its
goals.


How
to
create
autonomous
development?


Dr.
Oubba?,
March
2012


How
to
design
a
developmental
 learning
process
?


How
to
create
autonomous
development?


80
 Dr.
Oubba?,
March
2012


Developmental
Architecture


How
to
create
autonomous
development?


81
 Dr.
Oubba?,
March
2012


Even
simple
creatures
discover
new
strategies
 and
techniques
to
get
survival
resources,
or
to
 fight
predators.


Developmental
Architecture


How
to
create
autonomous
development?


82
 Dr.
Oubba?,
March
2012


Back
to
our
agent…


Developmental
Architecture


How
to
create
autonomous
development?


Dr.
Oubba?,
March
2012
 83


What
drives
the
agent
to
learn?


Developmental
Architecture


How
to
create
autonomous
development?


Dr.
Oubba?,
March
2012
 84


What
drives
the
agent
to
learn?


Task


…by
giving
it
a
task?


Developmental
Architecture


How
to
create
autonomous
development?


Dr.
Oubba?,
March
2012
 85


Not
only
 What
drives
the
agent
to
learn?


Developmental
Architecture


…by
giving
it
a
task?


How
to
create
autonomous
development?


86
 Dr.
Oubba?,
March
2012


Task


Not
only
 What
drives
the
agent
to
learn?


Developmental
Architecture


…by
giving
it
a
task?


...but
how
to
program
an
agent
without
knowing
the
 task!?


How
to
create
autonomous
development?


87
 Dr.
Oubba?,
March
2012


Task


What
drives
the
agent
to
learn?


Developmental
Architecture


Mo?va?on?


How
to
create
autonomous
development?


88
 Dr.
Oubba?,
March
2012


Task


What
drives
the
agent
to
learn?


Developmental
Architecture


Mo?va?on?
 This
might
work...


How
to
create
autonomous
development?


89
 Dr.
Oubba?,
March
2012


Task


Mo?va?on
is
what
gets
us
to
act.


Psychology


Developmental
Architecture


How
to
create
autonomous
development?


90
 Dr.
Oubba?,
March
2012


There
are
two
main
categories
of
mo?va?on
 Intrinsic
mo?va?on


doing
something
just
 because
it
is
enjoyable.

 (eg.
curiosity)

 doing
something
because
 it
leads
to
external
 rewards.
(e.g.
hunger)



Extrinsic
mo?va?on


Developmental
Architecture


How
to
create
autonomous
development?


91
 Dr.
Oubba?,
March
2012


sa?sfac?on


+


‐
 consump?on


(Inspired
by
the
PSI
theory
proposed
by
the
psychologist
Dietrich
Dörner)


A
mo?ve
as
a
container


The
filling
state
of
 the
container
 represents
the
 degree
of
 sa?sfac?on.


Developmental
Architecture


How
to
create
autonomous
development?


92
 Dr.
Oubba?,
March
2012


Gathering
resources


+
 ‐


energy
consump?on


Extrinsic
mo?va?on
 Hunger


Developmental
Architecture


How
to
create
autonomous
development?


93
 Dr.
Oubba?,
March
2012


Exploring
Novelty


+
 ‐


Predictability
of
 the
enviroment


Intrinsic
mo?va?on
 Curiosity


Developmental
Architecture


How
to
create
autonomous
development?


94
 Dr.
Oubba?,
March
2012


Hunger
 Curiosity


Developmental
Architecture


Body
 Mind


How
to
create
autonomous
development?


95
 Dr.
Oubba?,
March
2012


Curiosity


Developmental
Architecture


Hunger


Mo?ves
will
drive
the
agent
to
learn
as
much
as
possible
 about
the
environment.


How
to
create
autonomous
development?


96
 Dr.
Oubba?,
March
2012


Curiosity


Developmental
Architecture


Hunger


The
needs
of
the
body
and
those
of
the
mind
may
 compete
in
certain
circumstances.


How
to
create
autonomous
development?


97
 Dr.
Oubba?,
March
2012


Curiosity


Developmental
Architecture


Hunger


Associa?ng
importance
to
one
mo?ve
over
the
other
is
 constantly
re‐evaluated
by
experience.


How
to
create
autonomous
development?


98
 Dr.
Oubba?,
March
2012


Curiosity


Developmental
Architecture


Hunger


Learning
how
to
balance
between
mo?ves
will
provide
the
agent
the
 ability
to
solve
a
faced
problem
in
several
ways.
 Thus,
more
flexibility
(authority)
in
handling
situa?ons.


How
to
create
autonomous
development?


99
 Dr.
Oubba?,
March
2012


how
to
ar?culate
preferences
 among
mo?ves?



Developmental
Architecture


How
to
create
autonomous
development?


100
 Dr.
Oubba?,
March
2012


how
to
ar?culate
preferences
 among
mo?ves?



Developmental
Architecture


We
consider
this
as
a
mul?objec?ve
 problem.


How
to
create
autonomous
development?


101
 Dr.
Oubba?,
March
2012


Developmental
Architecture


We
propose
a
hierarchical
adap?ve‐cri?c
 design
(ACD)
that
allows
an
embodied
agent

 to
adapt
its
behavior
in
presence
of
several
 mo?ves.


How
to
solve
mul?objec?ve
problems?


How
to
create
autonomous
development?


102
 Dr.
Oubba?,
March
2012


The
proposed
architecture
is
mainly
based
on
reservoir
compu?ng.



Developmental
Architecture


Reservoir


Only
the
output
 connec?ons
are
 to
be
trained


How
to
solve
mul?objec?ve
problems?


How
to
create
autonomous
development?


103
 Dr.
Oubba?,
March
2012


Developmental
Architecture


How
to
solve
mul?objec?ve
problems?
 Cri?c
1
 Cri?c
2
 Cri?c
N


A
higher
level
reservoir
may
serve
as
a
cri?c
to
 learn
separate
value
func?ons
for
each
mo?ve
 simulatneously.



High‐level
ACD


How
to
create
autonomous
development?


104
 Dr.
Oubba?,
March
2012


Developmental
Architecture


How
to
solve
mul?objec?ve
problems?
 Cri?c
1
 Cri?c
2
 Cri?c
N


• 1. This
would
have
an
advantage
to
learn
from
separate
state


spaces
speed
up
learning.


High‐level
ACD


How
to
create
autonomous
development?


105
 Dr.
Oubba?,
March
2012


Developmental
Architecture


How
to
solve
mul?objec?ve
problems?
 Cri?c
1
 Cri?c
2
 Cri?c
N
 2.

The
ability
to
learn
from
heterogenous
data
with
the
same
 weights
(i.e.
memory
is
in
the
recurrent
ac?va?on,not
only
in
 the
synap?c
weights)
the
reservoir
would
be
able
to
cope
with
 the
conflic?ng
demands
imposed
by
the
mo?ves.



High‐level
ACD


How
to
create
autonomous
development?


106
 Dr.
Oubba?,
March
2012


Developmental
Architecture


How
to
solve
mul?objec?ve
problems?


Low‐level
ACDs


At
a
low
level
several

 reservoirs
may
have
the
role
to
 learn
op?mal
behavior
from
 the
con?nuous
and
uncertain
 state
space
to
sa?sfy
the
 selected
mo?ves.


γ
 How
to
create
autonomous
development?


107
 Dr.
Oubba?,
March
2012


Developmental
Architecture


How
to
solve
mul?objec?ve
problems?


Low‐level
ACDs


γ


Thus,
our
agent
runs
several
 ACDs
in
parallel
that
learn
 from
different
state
spaces.


How
to
create
autonomous
development?


108
 Dr.
Oubba?,
March
2012


Developmental
Architecture


Hierarchical
ACD
with
 reservoir
compu?ng
to
 solve
mul?objec?ve
 problems


How
to
create
autonomous
development?


109
 Dr.
Oubba?,
March
2012


M.
Oubba?,
J.
Uhlemann,
and
G.
Palm
(2012),
Adap7ve
learning
in
con7nuous
 environment
using
actor‐cri7c
design
and
echo‐state
networks.
FROM
ANIMALS
TO
 ANIMATS
(SAB12).



Developmental
Architecture


ACD
with
reservoir
compu?ng:
preliminary
results


How
to
create
autonomous
development?


110
 Dr.
Oubba?,
March
2012


Currently,
we
are
working
on
this
 developmental
learning
system



Developmental
Architecture


How
to
create
autonomous
development?


111
 Dr.
Oubba?,
March
2012


Developmental
Architecture


We
imagine
how
the
agent
explores
the
environment
in
order
to
 sa?sfy
ist
inborn
drives.
We
imagine
how
it
con?nually
wants
to
 master
the
environment,
and
how
it
doeas
ist
best
to
successfully
 face
challenges.
We
imagine
how
this
agent
asks
for
social
 interac?on
to
learn
from
humans,
what
it
cannot
learn
 independently...
This
is
what
we
want
to
realize.


112
 Dr.
Oubba?,
March
2012


113
 Dr.
Oubba?,
March
2012


114
 Dr.
Oubba?,
March
2012


115
 Dr.
Oubba?,
March
2012


116
 Dr.
Oubba?,
March
2012


117
 Dr.
Oubba?,
March
2012


118
 Dr.
Oubba?,
March
2012


119
 Dr.
Oubba?,
March
2012


120
 Dr.
Oubba?,
March
2012


Once
the
first
AGI
is
built,
it
is
difficult
to
predict
 what
further
degrees
of
intelligence
can
be
built.


121
 Dr.
Oubba?,
March
2012


122
 Dr.
Oubba?,
March
2012


123
 Dr.
Oubba?,
March
2012


124
 Dr.
Oubba?,
March
2012


If
these
goals
are
not
oriented
to
our
needs,
it
 would
lead
to
unpredictable,
and
probably
 dangerous,
consequences
for
our
survival.


125
 Dr.
Oubba?,
March
2012


126
 Dr.
Oubba?,
March
2012


127
 Dr.
Oubba?,
March
2012


128
 Dr.
Oubba?,
March
2012


129
 Dr.
Oubba?,
March
2012


130
 Dr.
Oubba?,
March
2012


Danke


131
 Dr.
Oubba?,
March
2012