Introduction to Automated Task Planning Jonas Kvarnstrm Automated - - PowerPoint PPT Presentation

▶

Feb 05, 2023 452 likes •769 views

Introduction to Automated Task Planning Jonas Kvarnstrm Automated Planning and Diagnosis Group Artificial Intelligence and Integrated Computer Systems (AIICS) / IDA jonkv@ida jonkv@ida What is Planning? Shakey 3 4 Classical example:

SLIDE 1

Introduction to Automated Task Planning

Jonas Kvarnström Automated Planning and Diagnosis Group Artificial Intelligence and Integrated Computer Systems (AIICS) / IDA

jonkv@ida

What is Planning?

Planning is thinking ahead

Not just reacting to what happens! Thinking about possible actions and their effects, formulating a complete plan that describes what to do and when, in order to achieve a goal …so when do we want computers to do that? 4

jonkv@ida

Shakey

Classical example: Shakey Used the STRIPS planner Stanford Research Institute Problem Solver One of the first planners (1969)

SLIDE 2

jonkv@ida

Miconic 10 Elevators

Schindler Miconic 10 elevators People enter their destination before they board an elevator Many elevators, many floors The goal: For everyone to be at their destination The plan tells us: Which elevators should go to which floors, in which order? The gain: Less time to wait for an elevator!

jonkv@ida

Sheet Metal Bending

Robots bending sheet metal Goal: Bend a flat sheet to a specific shape Constraints: The piece must not collide with anything when moved! (Geometric reasoning required) Optimized operation saves a lot of time = money!

jonkv@ida

NASA Earth Observing-1 Mission

Earth Observing-1 Mission Satellite in low earth orbit Can only communicate 8 x 10 minutes/day Operates for long periods without supervision CASPER software: Continuous Activity Scheduling, Planning, Execution and Replanning Dramatically increases science returns Interesting events are analyzed (volcanic eruptions, …) Targets to view are planned depending on previous observations Plans downlink activities: Limited bandwidth http://ase.jpl.nasa.gov/

jonkv@ida

Unmanned Aerial Vehicles

Unmanned aerial vehicles Multiple agents to coordinate Actions executed concurrently

SLIDE 3

jonkv@ida

UAV 1: Photogrammetry

Photograph buildings, to generate 3D models

jonkv@ida

UAV 2: Traffic Monitoring

Monitor traffic / find possible routes for emergency vehicles

jonkv@ida

UAV 3: Finding Forest Fires

Patrol large areas searching for forest fires, day after day after day…

jonkv@ida

UAV 4: Emergency Services Logistics

Assist in emergency situations Deliver packages of food, medicine, water

SLIDE 4

jonkv@ida

Why Planning?

Can’t we just solve these problems ourselves? Manual planning can be boring and inefficient Who wants to spend all day guiding elevators? Automated planning may create higher quality plans Software can systematically optimize, can investigate millions of alternatives Automated planning can be applied where the agent is Satellites cannot always communicate with ground operators Spacecraft or robots on other planets may be hours away by radio

Domain-Specific vs. Domain-Independent Planning

jonkv@ida

Introduction

Let’s say we are interested in the photogrammetry domain Multiple UAVs available Goal: To take pictures of buildings How do you create a photogrammetry plan?

jonkv@ida

Domain-Specific Planning

Domain-specific planner: Created for a particular domain Input: A map containing buildings Everything else is implicit, since the planner already knows what actions are available, etc (hardcoded!) Implementation: Calculate suitable UAV positions Call a Travelling Salesman Problem solver Generate flight commands PG Problem PG-specific Planner PG Plan

Sounds simple, and can be very efficient! But…

SLIDE 5

jonkv@ida

Domain-Specific Planning

Specialization means less flexibility! What if… you want to deliver a couple of crates at the same time? Generalize input… you have to consider refueling? Different algorithm! you have two UAVs and a UGV (ground vehicle)? Different algorithm! you want to survey an area (send a video feed of the ground)? you have dynamic no-fly-zones (”don’t fly there at 15:00-16:00”)? PG + Delivery Planner PG Planner w/ fuel Multi- TSP planner

jonkv@ida

Domain-Independent Planning

We want a generic planning system! Capable of taking a high-level description of any problem domain and then solving problems within the domain! A single planner Improvements to the planner all domains benefit High-level domain and problem definitions Easier to specify these than to write specialized algorithms Easier to change than a hard-coded optimized implementation Also useful for rapid prototyping

Satellite Domain

General Planner

Satellite problem instance Plan UAV Domain

General Planner

Photogrammetry problem instance Plan

jonkv@ida

What is Common?

What is common to all of these problems?

jonkv@ida

This is Common!

The world contains various types of objects Buildings, cards, aircraft, switches, people, … We are generally interested in properties of these objects Location of a card, whether we have a picture of a building or not, … Goals can be described as desired properties We should have a picture of each building Actions modify properties (simplified!) takeoff(uav) now the UAV is in the air photograph(uav, building) now we have a picture of the building Can only be executed if we can see the building

SLIDE 6

jonkv@ida

Example 1

In the Emergency Services Logistics domain:

1. What objects exist?

Helicopters! heli = { surv1, surv2, …, cargo1, cargo2, … } Boxes! box = { box1, box2, … } Locations! loc = { basestation, person1, person2, …, depot1, depot2, … } Let’s generalize: Both helicopters and boxes are objects as well.

jonkv@ida

Example 2

2. What properties can different types of objects have?

(Defined using state variables)

An object is or isn’t at a certain location: at(object, loc) A helicopter is or isn’t at a certain location: at(heli, loc) A helicopter is or isn’t carrying a certain box: carrying(heli, box)

jonkv@ida

Example 3

3. What properties do the objects of the current problem have now?

Every box is at a certain location: at(box1, depot1), at(box2, depot1), … Every helicopter is at a certain location: at(surv1, basestation), at(surv2, person1), … Some helicopters are carrying boxes: carrying(cargo1, box4)

4. What properties should they have?

This is our goal!

at(box1, person1), at(box2, person2), …

Situation we want to achieve Situation right now

jonkv@ida

Example 4

4. What can we do about it? Actions!

Example: fly(heli, loc1, loc2) Preconditions: at(heli, loc1) loc2 is free – no other helicopter there fuel(heli) > dist(loc1, loc2) * fuelUsage(heli) Effects: at(heli, loc1) is no longer true at(heli, loc2) is true instead fuel(heli) decreases by dist(loc1, loc2) * fuelUsage(heli) Time requirements: distance(loc1, loc2) / speed(heli) Take off / land Hover at the current location Position the camera at angle θ Take a visual picture / Take an infrared picture Lift a package using a winch / Deliver a package using winch L1 L2

SLIDE 7

jonkv@ida

Domain-Independent Planning

Input 1: Planning domain Input 2: Problem instance

Object Types: There are UAVs, boxes … Properties: Every UAV has a maxSpeed, … Actions: Definition of fly, pickup, … Objects: Current UAVs are {UAV1,UAV2} Initial State: Box locations, … Goal: Box b1 at location l1, …

But what language should we use? 27

jonkv@ida

Desirable Properties

Desirable properties of a domain-independent planner: Should be as general as possible Handle a wide variety of domains Should be as efficient as possible Generate plans quickly Should generate plans of the highest quality Fewer actions, lower cost, faster to execute, … Should support the user as much as possible Provide useful high-level structures such as actions that a user can easily specify

Many of these properties are in direct conflict!

jonkv@ida

Classical Planning

For efficiency, planners generally exploit domain structure This implies constraining the expressivity of the planner and its input language! Classical planning uses a common set of constraints… Sometimes called "STRIPS planning" Demonstrated using a simple domain: The Blocks World Also introduces PDDL, the Planning Domain Definition Language General language Subsets are supported by most modern planners

SLIDE 8

jonkv@ida

Blocks World

What you can do: Pick up a block unless something is on top of it Put a block that you’re holding…

n the table (space for all blocks)
n another block,

unless something is on top of it Which actions will achieve your goals?

Your greatest desire What you know You

C B A C B A

jonkv@ida

PDDL: Domain and Problem Definition

PDDL: Lisp-like language, expressions in parentheses Every domain is named, associated with explicit list of expressivity requirements (define (domain blocksworld) (:requirements :adl ;; Conditional effects, quantification, … :equality ;; Use of “=” …) ;; Remaining domain information goes here! ) Problem instance has a name, belongs to a domain (define (problem MyThreeBlockProblem) (:domain blocksworld) … ) Colon before many keywords, in order to avoid collisions when new keywords are added

jonkv@ida

1: Objects

In classical planning, there is a finite set of objects Objects are generally declared in the problem instance Different objects in different instances! (define (problem MyThreeBlockProblem) (:domain blocksworld) (:objects A B C) … ) We can choose to model the table as a special block Always present in all instances defined in the domain (define (domain blocksworld) (:requirements :adl …) (:constants Table) … )

C B A

jonkv@ida

2: Finite States

Classical planning: World modeled using finite states A finite set of state variables (predicates or non-boolean fluents), with values from a finite domain A finite (but gigantic!) set of possible states of the world Each state gives every state variable a specific value Many ways to model any domain! One example for blocks world: We must know if a block is on something else, or not We must know whether an object is the table, which we modeled as a ”special” block (define (domain blocksworld) (:requirements :adl …) (:predicates (on ?X ?Y) (istable ?X)) … )

C B A

Variables are prefixed with “?”

SLIDE 9

jonkv@ida

2b: State Example

Three different states

in the Blocks World

Can be described graphically Can be described in text

(the following 3 slides!)

C B A C B A C B A

jonkv@ida

3: Complete Initial Information

We assume complete information about the initial state All state variables are given values in the problem instance BW: on(A,B), on(B,Table), on(C,Table)… etc.

C B A

jonkv@ida

3b: Initial State in PDDL

How do we completely specify the initial BW state? (and (not (on A A)) (not (on A B)) (on A C) (not (on A Table) (not (on B A)) (not (on B B)) (not (on B C)) (on B Table) (not (on C A)) (not (on C B)) (not (on C C)) (on C Table) (not (on Table A)) (not (on Table B)) (not (on Table C)) (not (on Table Table)) (not (istable A)) (not (istable B)) (not (istable C)) (istable Table)) Observation: Given complete info, most facts are false! A block can only be on one other block: Given any ?x, at most one instance of (on ?x ?y) is true (Given 1000 blocks, ?x is on 1, not on 999)

C B A

jonkv@ida

3c: Conventions for Initial States

Convention: Only specify what is true in the initial state (define (problem MyThreeBlockProblem) (:domain blocksworld) (:objects A B C) (:init (on A C) (on C Table) (on B Table) (istable Table)) …) This is a shorthand notation, and is only used in :init. PDDL is for planners that cannot handle unknown initial information, so for convenience, an atom that is not mentioned is by definition false

C B A

SLIDE 10

jonkv@ida

4: Operators and Actions

A set of single-step operators defines what you can do Part of the domain description Many ways of modeling any domain! One example: “put” should take a block from where it is, and put it on another block Each operator has parameters: put(x,y) Instantiated into actions: put(A,B), put(A,C), … Each operator has a precondition “There must be no other block on top of x”, … Each operator has effects Afterwards, x will be on top of y

put(A,B)
C

B A C B A

jonkv@ida

4b: Operators and Actions

Each action is executed as a single step If the precondition holds in the current state: The action can be executed A single new state results, where the effects hold. UAV domain: Flying from (100,100) to (300,300) is basically modeled as “teleporting” in a single step We don’t model the fact that the UAV moves through intervening locations Sufficient for sequential plans

put(A,B)
C

B A C B A

jonkv@ida

4c: Operator Preconditions

Precondition: Must hold for an action to be applicable Classical planning: We must know what an action requires! (define (domain blocksworld) … (:action put :parameters (?moved ?dest) :precondition (and ;; Don't put the table on top of something (not (istable ?moved)) ;; Don't put a block on top of itself (not (= ?moved ?dest)) ;; There is nothing on top of the block we move (not (exists (?other) (on ?other ?moved))) ;; Either the destination is the table, ;; or there is nothing on top of it (or (istable ?dest) (not (exists (?other) (on ?other ?dest)))) )

Connectives: (and …) (or …) (imply …) (not …) Quantifiers: (exists (?v) …) (forall (?v) …)

Operators are called actions in PDDL…

jonkv@ida

4d: Effects

Effects: What changes if you perform an action? Classical planning: We must know all effects! Can’t state that an action may fail: (on x y) or (on x Table) Can’t model outcome probabilities: 95% (on x y), 5% (on x Table) Instead, complete information: Only (and …) is possible! Include atoms that become true: (on ?moved ?dest) Include negated atoms that become false: (not (on ?moved ?other)) :effect (and ;; ?moved is now on ?dest. (on ?moved ?dest) ;; If ?moved was on some ?other, it no longer is. (forall (?other) (when (on ?moved ?other) (not (on ?moved ?other))) )

Uses quantified and conditional effects (ADL): (when (condition-in-invocation-state) (effects-in-result-state))

SLIDE 11

jonkv@ida

5: No Other Agents

Classical planning assumes the state of the world can only be modified

by actions generated by the planner

No agents are moving blocks around unless we say so in the plan! When considering different potential plans, the planner can predict the exact end result of every action in the plan without worrying about someone "disturbing" its execution

jonkv@ida

6: Goals

Goals only constrain the final state of a plan Specify the end result, not how you get there Specified in the problem instance Goal for our example instance One possibility: (define (problem ABC) (:domain blocksworld) … (:goal (and (on A B) (on B C) (on C Table))))

C B A

jonkv@ida

6b: Goals

A complete goal specification is generally not required Some predicates can "follow from others" (define (problem ABC) (:domain blocksworld) … (:goal (and (on A B) (on B C) (on C Table)))) Here we don't specify that (on A C) must be false This just follows from the fact that a block can't be on two different blocks at the same time

C B A

jonkv@ida

6c: Goals

A goal specification can usually correspond to many states An arbitrary conjunction is allowed We do not assume that predicates left out must be false! (define (problem ABC) (:domain blocksworld) … (:goal (and (on B C) (on C Table)))) We don't say whether (on A B) is true, so either of the two following states would be considered goal states! Sometimes disjunctions may be allowed as well

C B A C B A

SLIDE 12

Using Object Types

Same Expressivity, More Convenient

jonkv@ida

Example Domain: Logistics

The "traditional" logistics domain Unlike the blocks world, many object types We have a number of packages at given locations Move packages within a city using trucks Move packages between cities using airplanes Airplanes can only move between certain airport locations Goal: Packages should be at their destinations

jonkv@ida

Logistics Domain in PDDL

First attempt at a logistics domain: (define (domain logistics) (:predicates (incity ?L ?C) (at ?X ?L) (in ?P ?V)) (:action load :parameters (?P ?V ?L) :precondition (and (at ?P ?L) (at ?V ?L)) :effect (and (in ?P ?V) (not (at ?P ?L))) (:action drive-truck :parameters (?T ?L1 ?L2 ?C) :precondition (and (incity ?L1 ?C) (incity ?L2 ?C) (at ?T ?L1)) :effect (and (not (at ?T ?L1)) (at ?T ?L2))) …) Problem: No type checking!

load plane7 onto city5 at truck2
drive city9 from truck1 to airplane2

jonkv@ida

Type Predicates

Solution 1: Type predicates

(define (domain logistics)

(:predicates (incity ?L ?C) (at ?X ?L) (in ?P ?V) (location ?L) (airport ?A) (vehicle ?V) (truck ?T) (airplane ?P) (package ?P)) (:action load :parameters (?P ?V ?L) :precondition (and (package ?P) (vehicle ?V) (location ?L) (at ?P ?L) (at ?V ?L)) :effect (and (in ?P ?V) (not (at ?P ?L)))) …)

(define (problem MyLogisticsProblem)

(:domain logistics) (:objects plane1 plane2 truck1 truck2 truck3 loc1 loc2 …) (:init (incity loc1 linköping) … (plane plane1) (plane plane2) (location loc1) (location loc2) …))

SLIDE 13

jonkv@ida

Explicit Types

Solution 2: "True" types (many modern planners)

(define (domain logistics)

(:requirements :typing) (:types truck airplane – vehicle airport – location package vehicle location city (:predicates (incity ?L – location ?C – city) (at ?X – package ?L – location) (in ?P – package ?V – vehicle)) (:action load :parameters (?P – package ?V – vehicle ?L – location) :precondition (and (at ?P ?L) (at ?V ?L)) :effect (and (in ?P ?V) (not (at ?P ?L))) …)

(define (problem MyLogisticsProblem)

(:domain logistics) (:objects plane1 plane2 – airplane truck1 truck2 truck3 – truck loc1 loc2 – location …)

Unusual syntax:

subtype subtype … - supertype

All “top-level” types must be

declared last!

jonkv@ida

The 4-Operator Blocks World

Planning will be demonstrated using the Blocks World A more common variation, with 4 operators (pickup ?x) – takes ?x from the table (putdown ?x) – puts ?x on the table (unstack ?x ?y) – takes ?x from on top of ?y (stack ?x ?y) – puts ?x on top of ?y Plans are twice as long, but each operator is less complex Additional predicates used to avoid quantification (ontable ?x) – ?x is on the table (clear ?x): – ?x is an ordinary block whose top is free (holding ?x) – the robot is holding block ?x (handempty) – the robot is not holding a block pickup(B) putdown(A) unstack(A,C) stack(B,C)

C B A

jonkv@ida

State Space

Any classical planning problem corresponds to a state space This state space is a directed graph Each node is a world state Each directed edge corresponds to an executable action A B C D A B C D A B C D A B C D A B C D A B C D A B C D A small part of a BW state space, for a tiny problem instance. Notice that the BW lacks dead ends. This is not true for all domains!

SLIDE 14

jonkv@ida

Sequential Plans

For simple sequential (not concurrent or timed) plans: A plan is a path in the state space from the initial state to any of the goal states (recall that many states may satisfy the goal)! A B C D A B C D A B C D A B C D A B C D A B C D A B C D So how do we find such a path?

Forward State-Space Search (Progression Search)

jonkv@ida

Progression Search 1

Progression (Forward-chaining): “If I’m at this state now, what actions are applicable, and where will they take me?” Begin at the initial state Search forward, hoping to find a goal state Example: Initial state Goal state C B A D C B A D

jonkv@ida

Progression Search 2

Four potential ways of achieving the goal: 1) We may already have achieved the goal (not true here) 2) Unstack(A,C), then continue 3) Pickup(B), then continue 4) Pickup(D), then continue A B C D A B C D A B C D A B C D A B C D

SLIDE 15

jonkv@ida

Progression Search 3

Hypothetically: Suppose we unstack(A,C). What then? 1) Already achieved the goal? No. 2) May stack(A,C) – cycle, skip this. 3) May stack(A,B), continue 4) May stack(A,D), continue 5) May putdown(A), continue A B C D A B C D A B C D A B C D A B C D A B C D A B C D A B C D A B C D

jonkv@ida

Progression Search 4

Result: A search tree structure One start state, possibly many goal states Each node is associated with a search state (in this case = world state) Branching rule: Given a node, there is one outgoing branch for each applicable action G G

jonkv@ida

Depth First

Remaining choice: search algorithm Decides which actions to “simulate”, in which order One possibility: Depth first

cycle cycle

G G

jonkv@ida

Breadth First

Another possibility: Breadth first Same search tree, different search order Plans will be optimal in the number of actions Sometimes faster, sometimes slower, usually too much memory Can also use iterative deepening and other blind search methods

G G

SLIDE 16

The Difficulty of Planning

jonkv@ida

Forward-chaining Search

Seems intuitive – but what are the results? Movie 4 / no-rules This search procedure is “blind”: It just tries all variations! Is this really a problem? Computers are fast! What if we have 100 possible actions in each step, not 4 or 5? What if a plan requires 100 steps? 100 plans having 1 step 10 000 plans having 2 steps 1 000 000 plans having 3 steps 100 000 000 plans having 4 steps 10 000 000 000 plans having 5 steps … Emergency Service Logistics / 20 helicopters, 100 locations 200 boxes: 7 * 10415 potential plans…

70000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 0000000000000000 plans – but computers are fast… Assume 1020 (100 billion billion) plans per second: 20000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000 years – but we have multiple cores…

Suppose every particle in the universe is a computer: 2000000000000000000000000000000000000000000 000000000000000000000000000000000000000000 000000000000000000000000000000000000000000 000000000000000000000000000000000000000000 000000000000000000000000000000000000000000 000000000000000000000000000000000000000000 000000000000000000000000000000000000000000 000000 years

jonkv@ida

Hopeless?

Is there still hope for planning? Of course there is! Our trivial planning method uses blind search – tries everything! We wouldn’t choose such silly actions – so why should the computer? Planning is part of Artificial Intelligence! Developing methods to judge what actions are promising given goals Rapid progress in the last decade We are still far from true human-level understanding But planners are thousands of times faster than 10-15 years ago, even on the same hardware, despite being much more expressive A very active research area

Heuristic Progression Search and Domain-Dependent Guidance

SLIDE 17

jonkv@ida

Heuristic Search (1)

One approach: heuristic functions For each node, (under)estimates the cost to reach the goal Usually not perfect (can’t always show the best way to go) Example: Can’t always choose the optimal block to unstack Would be equivalent to solving the original planning problem! …together with heuristic search methods A*, hill climbing, many specialized methods

jonkv@ida

Heuristic Search (2)

A very simple domain-independent heuristic: Count the number of predicates with the “wrong” value Note: Only local information used (current state) A B C A B C A B C 7

ntable(C)
n(A,C)

¬on(C,A) ¬on(A,B) clear(A) clear(B) ¬clear(C) C A B

on(A,C)
clear(A)
¬clear(C)

+ holding(A) + handempty

clear(B)

+ ¬ontable(B) + holding(B) + handempty

A B C

¬on(A,B)
holding(A)
handempty

+ clear(A)

holding(A)
handempty

+ clear(A) + ontable(A)

jonkv@ida

Heuristic Search (3)

A perfect solution? No! Often useful compared to blind search Modern heuristics are more sophisticated – can still be "misleading" A B C D A B C D A B C D A B C D A B C D 5 8 5 9

ntable(B)

¬on(A,B)

n(A,C)

¬on(B,C) clear(B) B A C D

B A C D

on(A,C)

+ ¬clear(A) + clear(C) + ¬handempty + holding(A)

jonkv@ida

Heuristic Search (4)

Hand-tailored domain-dependent heuristics: 2 actions for any block that is not "on" its destination,

r that is above some other block that is not on its destination

1 action for any block we are holding, to put it down If we hold a block whose destination is not yet ready, we must put it on the table – and later, 2 actions to move it again A B C D A B C D A B C D A B C D A B C D

Must move B; holding A 2*1 + 1+0 = 3 Must move A; holding B; dest not ready 2*1+ 1+2 = 5 Must move A, B; holding D 2*2 + 1+2 = 7 Must move A, B 2*2 + 0+0 = 4

SLIDE 18

jonkv@ida

Domain-Dependent Guidance

Other forms of domain-specific guidance: Rules that help the planner recognize "stupid" actions "Don't unload packages before you reach the destination" "Don't drive to a destination where there's nothing to pick up" Sometimes a small set of common sense rules will suffice Examples later – TALplanner A B C D A B C D A B C D A B C D A B C D Moves a block that had to be moved, in order to put something else on C. GOOD. B was moved, but it did not block anything and its destination is not yet

ready. BAD.

As above. BAD.

Backward State-Space Search (Regression Search)

jonkv@ida

Regression Search (1)

Search method #2: Regression “If this is the goal, what action could I use to achieve part of the goal, and what would I have to do before that?” Begin at the goal Search backwards, hoping to find the initial state Example: Initial state Goal C B A D C B A D

jonkv@ida

Regression Search (2)

Three potential ways of achieving the goal: 1) It might already have been achieved – not in this case 2) The last action was stack(A,B) Before this, we’d have the same situation, except A would be held. 3) The last action was putdown(D) Before, D would have been held. A B C D A B C D A B C D A B C D

SLIDE 19

jonkv@ida

Regression Search (3)

How may we achieve the state where we’re holding A? pickup(A) unstack(A,D) unstack(A,B) Cycle! A B C D A B C D A B C D A B C D A B C D A B C D A B C D

jonkv@ida

No Current State during Search!

Forward search has a "current state" Backwards search has a "current goal" goal putdown(D) unstack(D,F) g1 g2 initial s2 stack(A,C) unstack(A,B) s1

jonkv@ida

Backward and Forward Search

FORWARD SEARCH

Problematic when:

There are many applicable actions

in the "average state" high forward branching factor need guidance

You know if an action is applicable,

but not if it will contribute to the goal

BACKWARD SEARCH

Problematic when:

There are many relevant actions

for the "average goal" high backward branching factor need guidance

You know if an action

contributes to the goal, but not if you can achieve its preconditions

Blind backward search

is generally better than blind forward search: Relevance tends to provide better guidance than applicability

But heuristics are still needed

SLIDE 20

jonkv@ida

Background 1

State-space planning commits immediately to action order Selects an action that seems “useful” According to heuristics According to relevance, in regression planning Puts it in its final place in the plan Continues from the resulting state (progression)… … Or from the resulting goal (regression) goal putdown(D) unstack(D,F) g1 g2 initial s2 stack(A,C) unstack(A,B) s1

jonkv@ida

Background 2: Example

Heuristic forward-chaining might begin as follows: Luckily, the heuristic always selected useful actions Picking up b1 is necessary, driving is necessary, putting b1 down is necessary But we still have a problem! At some time, the robot also has to pick up block b2 The best plan would be to do that right after pickup(b1,A): The robot has two arms Can't insert a new action there in state space search!

put(b1, B) effects: at(b1,B), ¬holding(b1) We want: at(b1,B) at(b2,B) We have: robot-at(A) at(b1,A) at(b2,A) drive(A,B) effects: ¬robot-at(A), robot-at(B) pickup(b1,A) effects: holding(b1), ¬at(b1,A) robot-at(A) holding(b1) at(b2,A) robot-at(B) holding(b1) at(b2,A) robot-at(B) at(b1,B) at(b2,A)

jonkv@ida

Background 3: Alternative A

The planner can backtrack, wasting some of its effort,

r continue from where it is:

put(b1, B) effects: at(b1,B), ¬holding(b1) We want: at(b1,B) at(b2,B) We have: robot-at(A) at(b1,A) at(b2,A) put(b2, B) effects: at(b2,B), ¬holding(b2) drive(A,B) effects: ¬robot-at(A), robot-at(B) pickup(b1,A) effects: holding(b1), ¬at(b1,A) pickup(b2,A) effects: holding(b2), ¬at(b2,A) drive(B,A) effects: ¬robot-at(B), robot-at(A) drive(A,B) effects: ¬robot-at(A), robot-at(B) robot-at(A) holding(b1) at(b2,A) robot-at(B) holding(b1) at(b2,A) robot-at(B) at(b1,B) at(b2,A) robot-at(A) at(b1,B) at(b2,A) robot-at(A) at(b1,B) holding(b2) robot-at(B) at(b1,B) holding(b2) robot-at(B) at(b1,B) at(b2,B) Results in a suboptimal plan! We could have done it in 5 actions.

jonkv@ida

Background 4: What We Prefer

We want: at(b1,B) at(b2,B) We have: robot-at(A) at(b1,A) at(b2,A) pickup(b1,A) effects: holding(b1), ¬at(b1,A)

Since the first three actions we found were actually useful for the goal, we would prefer if we could just slide the last ones over…

robot-at(A) holding(b1) at(b2,A) put(b1, B) effects: at(b1,B), ¬holding(b1) drive(A,B) effects: ¬robot-at(A), robot-at(B) robot-at(B) holding(b1) at(b2,A) robot-at(B) at(b1,B) at(b2,A) pickup(b2,A) effects: holding(b2), ¬at(b2,A) robot-at(A) at(b1,B) holding(b2)

…and then insert another pickup action inbetween! This is no longer state-space search! Requires different techniques…

SLIDE 21

Generating Partial-Order Plans

jonkv@ida

No Current State during Search!

Forward search has a "current state" Backwards search has a "current goal" With partial-order plans: No “current” state or goal! What is true after stack(A,B) below? Dependson the order in which other actions are executed Changes if we insert new actions before stack(A,B)! stack(A,B) putdown(D) unstack(D,F) goal putdown(D) unstack(D,F) g1 g2 initial s2 stack(A,C) unstack(A,B) s1 goalaction initaction

jonkv@ida

Search Nodes are Partial Plans

No current state search nodes can’t correspond to states! A node in the search space has to be a partial plan Each successor is a modified, refined partial plan Use search strategies, backtracking, heuristics, ... to search this space! initial search node initaction goalaction initaction goalaction

putdown(D)

initaction goalaction

unstack(B,C)

initaction goalaction

putdown(D) putdown(K)

This is one form of ”plan-space” planning! 84

jonkv@ida

POCL vs Forward-Chaining

What is the initial search node? Forward state space search: The initial state POCL: The initial plan, consisting of: A special initial action whose effects specify the initial state A special goal action whose preconditions specify the goal A precedence constraint: the initial action is before the goal action

goalaction

initaction

SLIDE 22

jonkv@ida

POCL vs Forward-Chaining

What are the successors of a search node? Progression: One successor per action executable in the current state Preconditions satisfied, effects consistent, … POCL: One successor for every way that a flaw in the plan can be fixed Every way of supporting an open goal Every way of removing a threat initial search node initaction goalaction initaction goalaction

putdown(D)

initaction goalaction

unstack(B,C)

jonkv@ida

Flaw Type 1: Open Goals

Open goal: An action a has a precondition p that we haven’t decided how to establish (it has no incoming causal link) For example, the preconditions of the goal action! To resolve the flaw: Find an action b that causes p to become true Can be a new action Can be an action already in the plan, if we can make sure it precedes a Create a causal link

Partial order! This was not possible in backward search…

Even if there is already an action that causes p, you can still add a new action that also causes p!

goalaction

jonkv@ida

Resolving Open Goals 1

In this initial Blocks World plan we have six open goals We can choose to find support for clear(A): From the initaction From a new instance of unstack(B,A), unstack(C,A), or unstack(D,A) From a new instance of stack(A,B), stack(A,C), stack(A,D), or putdown(A) We can choose to find support for on(A,B): Only from a new instance of stack(A,B) …

initaction

goalaction

We haven't decided how to achieve any of these six goals!

jonkv@ida

Resolving Open Goals 2

Suppose we add stack(A,B) to support (achieve) on(A,B) Must add a causal link for on(A,B) Dashed line Must also add precedence constraints The plan looks totally ordered Because it actually only has one “real” action…

initaction
goalaction
stack(A,B)

Causal link says: This instance of stack(A,B) is responsible for achieving on(A,B) for the goalaction

SLIDE 23

jonkv@ida

Resolving Open Goals 3

Now we have 7 open goals We can choose to find support for clear(A): From the initaction From the instance of stack(A,B) that we just added From a new instance of stack(A,B), stack(A,C), stack(A,D), or putdown(A) From a new instance of unstack(B,A), unstack(C,A), or unstack(D,A) …

initaction
goalaction
stack(A,B)

jonkv@ida

Flaw Type 2: Threats

Second flaw type: A threat initaction supports clear(B) for stack(A,B) – there’s a causal link pickup(B) deletesclear(B), and may occur between initaction and stack(A,B) So we can’t be certain that clear(B) still holds when stack(A,B) starts!

initaction
goalaction
stack(A,B)
stack(B,C)
pickup(B)

jonkv@ida

Resolving Threats 1

How to make sure that clear(B) holds when stack(A,B) starts? Alternative 1: The action that disturbs the precondition is placed after the action that has the precondition Only possible if the resulting partial order is consistent (acyclic)!

initaction
goalaction
stack(A,B)
stack(B,C)
pickup(B)

jonkv@ida

Resolving Threats 2

Alternative 2: The action that disturbs the precondition is placed before the action that supports the precondition Only possible if the resulting partial order is consistent – not in this case!

initaction
goalaction
stack(A,B)
stack(B,C)
pickup(B)

SLIDE 24

jonkv@ida

Resolving Threats 3

Only causal links can be threatened! Below, pickup(B) does not threaten the precond clear(B) of stack(A,B) We haven’t decided yet how to achieve clear(B): No incoming causal link So we can’t claim that its achievement is threatened!

initaction
goalaction
stack(A,B)
stack(B,C)
pickup(B)

jonkv@ida

The PSP Procedure

Plan-Space Planning: PSP(π) flaws OpenGoals(π) ∪ Threats(π) if flaws = ∅ then return π select any flaw φ ∈ flaws resolvers FindResolvers(φ, π) if resolvers = ∅ then return failure // Backtrack nondeterministically choose a resolver ρ ∈ resolvers π’ Refine(ρ, π) // Actually apply the resolver return PSP(π’) end Call PSP(the initial plan) PSP is both sound and complete It returns a partially ordered solution plan Any total ordering of this plan will achieve the goals

Not a backtracking point! Resolving one flaw cannot prevent us from resolving other flaws. This is a backtracking point. For example, a resolver might add an action that solves this local flaw, but that cannot be part of a solution. The plan is complete exactly when there are no remaining flaws (no open goals, no threats)

jonkv@ida

Conclusions

Partial-order planning delays commitment to action ordering Lower branching factor More efficient in some situations Many POP planners still assume sequential execution The intention was to find plans quickly, not to find partially constrained plans POCL planning was long considered more efficient than FC With new heuristics, forward-chaining planners took the lead At this point, many of the fastest planners are forward-chaining, but partial-order planners have also improved

jonkv@ida

Other Search Spaces for Planning

Many other search spaces: Parallel progression and regression Start from both ends; hope to meet in the middle Encoding: Transform into another problem, e.g. SAT (boolean satisfiability) … Consult the literature for more information!

SLIDE 25

jonkv@ida

Sequential Plans: Blocks World

Sequential plans are OK for blocks world There is only one robot… pickup(B) putdown(A) unstack(A,C) stack(B,C)

jonkv@ida

Sequential Plans: Logistics

Consider sequential

planning for Logistics:

Two plans with the same

number of actions

Do you see the difference?

load(P1,T1) load(P2,T1) load(P3,T1) drive(T1,A,B) unload(P1,T1) drive(T1,B,C) unload(P2,T1) drive(T1,C,D) unload(P3,T1) load(P1,T1) load(P2,T2) load(P3,T3) drive(T1,A,B) unload(P1,T1) drive(T2,A,C) unload(P2,T2) drive(T3,A,D) unload(P3,T3)

100

jonkv@ida

The Linear Plan: Logistics

The first plan uses three trucks Could easily be executed concurrently This is what we want! How should sequential planners see

that we prefer the first plan?

SLIDE 26

101

jonkv@ida

Plan Structures

The plan structure should depend on: The abilities of the execution system If it can perform many tasks concurrently, we want concurrent plans Otherwise, we might be satisfied with simple sequences The desired solution properties Is timing important, or can it be ignored? The abilities of the algorithms Most algorithms are tightly bound to plan structures May settle for a less expressive plan structure, if this allows you to use an algorithm which is better in other respects

102

jonkv@ida

Temporal Concurrent Plans

Temporal concurrent plans: Explicit timing Predict the amount of time required for each action [0,10] load(P1,T1) [10,570] drive(T1,A,B) [570,580] unload(P1,T1) [0,10] load(P2,T2) [10,420] drive(T2,C,D) [420,430] unload(P2,T2) [0,10] load(P3,T3) [10,845] drive(T3,E,F) [845,855] unload(P3,T3)

103

jonkv@ida

Action Durations in PDDL

PDDL supports action durations …And some planners implement this, generating temporal plans. (:durative-action turn :parameters (?current-target ?new-target - target) :duration (= ?duration (/ (angle ?current-target ?new-target) (turn-rate))) :condition (and (at start (pointing ?current-target)) (at start (not (controller-in-use)))) :effect (and (at start (not (pointing ?current-target))) (at start (controller-in-use)) (at start (vibration)) (at end (not (controller-in-use))) (at end (not (vibration))) (at end (pointing ?new-target))))

SLIDE 27

105

jonkv@ida

TALplanner

TALplanner: A planner developed at IDA Forward-chaining and uses depth-first search Allows us to use highly expressive models Arbitrary preconditions Conditional effects Timing, concurrency and resources Depth first requires guidance and goal-directedness No built-in heuristics Uses domain-dependent rules to guide towards the goal Can also specify domain-dependent heuristics Did well in international competitions

106

jonkv@ida

TALplanner Example Domain

Example Domain: ZenoTravel Planes move people between cities (board, debark, fly) Planes have limited fuel level; must refuel Example instance: 70 people 35 planes 30 cities

107

jonkv@ida

ZenoTravel Problem Instance

A smaller problem instance

108

jonkv@ida

What Just Happened?

No additional domain knowledge specified yet! Pure depth first…

initial node

ne of the

goal nodes

SLIDE 28

109

jonkv@ida

Control Rules

TALplanner uses control rules The plan generates a sequence of states Plan: step 1: board(p1, plane1, city4), step 2: board(p2, plane1, city4) States: time 0: initial state time 1: after p1 boarding time 2: after p2 boarding Control rules are formulas that must hold in this state sequence Rules can also refer to the goal goal(f) true if f must hold in all goal states at(p1, city4) at(p2, city4) board(p1, plane1, city4) board(p2, plane1, city4) in(p1, plane1) at(p2, city4) in(p1, plane1) in(p2, plane1)

110

jonkv@ida

Multiple Uses of Control Rules

Control rules have multiple uses Prune parts of the tree that do not contain solutions No need to investigate dead ends Some domains, like ZenoTravel, have no dead ends Prune parts of the tree that contain “bad” solutions Plans that achieve the goal, but are unnecessarily expensive Applicable to this domain! Prune parts of the tree that contain redundant solutions In optimizing planning, we don’t stop when the first plan is found No point in visiting many plans with equal cost

111

jonkv@ida

Control Rules

First problem in the example: Passengers debark whenever possible. Rule: "At any timepoint, if a passenger debarks, he is at his goal.” #control :name "only-debark-when-in-goal-city" forall t, person, aircraft [ [t] in(person, aircraft) [t+1] in(person, aircraft) ∨ exists city [ [t] at(aircraft, city) ∧ goal(at(person, city)) ] ]

112

jonkv@ida

Control Rules

Second problem in the example: Passengers board planes, even at their destinations Rule: "At any timepoint, if a passenger boards a plane, he was not at his destination.” #control :name "only-board-when-necessary" forall t, person, aircraft [ ([t] !in(person, aircraft) ∧ [t+1] in(person, aircraft)) exists city1, city2 [ [t] at(person, city1) ∧ goal(at(person, city2)) ∧ city1 != city2 ] ]

SLIDE 29

113

jonkv@ida

Zeno Travel, second attempt

114

jonkv@ida

What's Wrong This Time?

Only constrained passengers Forgot to constrain airplanes Which cities are reasonable destinations?

1. A passenger’s destination
2. A place where a person wants to leave
3. The airplane’s destination

115

jonkv@ida

Control Rules

#control :name "planes-always-fly-to-goal“

forall t, aircraft, city [ [t] at(aircraft, city) ([t+1] at(aircraft, city)) | exists city2 [ city2 != city & ([t+1] at(aircraft, city2)) & [t] reasonable-destination(aircraft, city2) ]]

#define [t] reasonable-destination(aircraft, city):

[t] has-passenger-for(aircraft, city) | exists person [ [t] at(person, city) & [t] in-wrong-city(person) ] | goal(at(aircraft, city)) & [t] empty(aircraft) & [t] all-persons-at-their-destinations-or-in-planes ]

116

jonkv@ida

Zeno Travel, third attempt

SLIDE 30

117

jonkv@ida

Positive Side Effects

Positive effects of using domain knowledge: Better performance than domain-independent heuristics When you’ve found the right control rules! Can use more detailed domain models With control rules, this has a smaller impact on performance Can generate better plans Can prune non-solutions, but also bad solutions Negative effects of using domain knowledge: Requires more knowledge Requires more domain engineering Rules are sometimes easy to find, sometimes difficult As always: A tradeoff!