Reducing the Search Space of Resource Constrained DCOPs T. Matsui 1) - - PowerPoint PPT Presentation

Reducing the Search Space of Resource Constrained DCOPs

• T. Matsui1), M. Silaghi2), K. Hirayama3), M. Yokoo4),
• B. Faltings5) and H. Matsuo1)

1) Nagoya Institute of Technology, 2) Florida Institute of Technology, 3) Kobe University, 4) Kyushu University, 5) Swiss Federal Institute of Technology, Lausanne (EPFL)

1

Contribution

 Resource Constrained Distributed Constraint Optimization Problem (RCDCOP) is a dedicated representation for multi-agent cooperation constrained by shared resources.  A Dynamic programming method based on pseudo-trees can be applied to the RCDCOP.  Its large search space increases the size of tables

• f the dynamic programming method.

 We apply combinations of several efficient methods to reduce the search space for the RCDCOP solver.

2

Outline

RCDCOP

Problem definition Pseud-trees ignoring Resource Constraints A solver based on dynamic programming

Reducing search space

Reducing redundancy Improving accuracy of infeasibility/feasibility

Experiments Conclusions

3

Distributed Constraint Optimization Problem (DCOP)

x0 x1 x2 x3 x4 Constraint network for DCOP

4

Constraint (objective function) Variable (Agent)

State/decision of an agent is represented as a variable. Relationship between agents is represented as a constraint. Multi-agent cooperation is formalized as constraint optimization.

: variable/agent : global optimal cost : constraint between

i

x

i

D : domain of

: objective function for

j i

c ,

j i

f ,



j i

f ,

i

x

j i

c ,

i

x

j

x

and

Distributed search algorithms solve the global optimal solution.

min

s constraint all for

j i

c ,

Resource Constrained DCOP (RCDCOP)

5

x0 x1 x2 x3 x4 r0 r1 RDCCOP Resource constraint

In RCDCOP, Resource Constraint defines limitations of resource use. A resource constraint is a global (n-ary) constraint. : resource.

a

r

: resource use of

) , (

i i

d r u

: resource constraint s.t. ≤

a

R

: capacity of

a

R

A resource has its capacity.

Each agent requires related resources



∈

a

r i i a i

d r u

require which nodes

) , (

Total use of a resource must not exceed its capacity.

a

r

a

c

) , (

i i d

x

[Matsui et al. 08]

Pseudo-tree ignoring resource constraints

x0 x1 x2 x3 x4 r0 r1 Back edge Tree edge based on a DFS tree

not arrowed original constraints arrowed resource constraints

6

[Matsui et al. 08]

Several exact solvers for RCDCOPs employ extended Pseudo-trees that are based on spanning-trees (e.g. DFS trees) for the problems.

Constraints between sub-trees are:

A resource constraint is just a Boolean constraint that depends on the summation of unary usage functions.

Additional hyper edge for resource constraint

The summation only depends on tree edges. The Boolean constraint is evaluated later using the summation.

Solver based on dynamic programming

x0 x1 x2 x3 x4 r0 r1 1) computation of optimal cost 2) computation of optimal assignment x0 x1 x2 x3 x4 r0 r1 COST(UTIL) message

A dynamic programming DPOP (for DCOPs) employs pseudo trees.

7

[Kumar et al. 09] [Petcu et al. 05] Infeasible assignments are eliminated.

An extension of DPOP solves DCOPs with specific resources. We employ another DPOP for RCDCOPs (RC-DPOP).

VALUE message

In the following, we mainly focus on the computation of the optimal cost. The processing consists of two phases:

Computation of cost values

r0 r1 x1 x2 x3 x0 x0 g1 x1 g2 x0 x1 g3 vr0 vr1 vr0' vr1' vr1''

8

Each agent i computes/propagates a table of:

 assignments for ancestor variables that relate with sub-tree rooted at i  cost values gi for the assignments  combination of resource use that are represented by virtual variables vrl

1 5 3 2 4 1

For the same assignments of variables, different combinations of cost value and resource use can exist.

(The cost values implicitly depend on the resource use.)

Computation of cost values (contd.)

r0 r1 x1 x2 x3 x0 x0 g1 x1 g2 x0 x1 g3 vr0 vr1 vr0' vr1' vr1''

9

The aggregation of cost values is based on:

1) summation of cost values for each combination of assignments and resource use 2) minimization for all values of current variable

x0 vr0 x1 vr1 f0,1 + g2 +g3 1 4 1 1 2 5 9 7 4 3 8 6 3 2 f0,1 u(r0, x1) u(r0, x1) + vr0' vr1' + vr1'' summation minimization

The size of tables grows with the number of combinations of resource use.

Infeasible assignments are eliminated.

Reducing the search space

 Root node for each resource 1)  Aggregation of cost value and resource use 1)  Global lower bound of resource use 1), 2)  Replacing evaluations of resource use 2)  Aggregation of feasible assignments 1), 3) To reduce the size of the tables in DP, we employ preprocessing and add-on processing based on:

10

The methods 1) reduce redundancy or 2) improve accuracy of infeasibility/ 3) feasibility. Several ideas are based on previous studies. We mainly investigate effects of the methods for RCDPOP.

Root node for each resource

r0 r1 x1 x2 x3 x0 x0 g1 x1 g2 vr0,vr1 vr0,vr1 ε g1 vr0,vr1 r0 r1 x1 x2 x3 x0 x0 g1 x1 g2 vr0,vr1 vr0,vr1 ε g1 vr1

For resource r , the highest agent that relates to r eliminates resource use vr of r . The resource use is propagated among limited area of agents.

11

root agent of r0, r1 root agent of r1 root agent of r0 (eliminates vr0 ) (preprocessing)

Aggregation of cost value and resource use

cost value and resource use vr0 vr1 w w' vr0 vr1 w w' (a) w' is replaced by w (b) w and w' are not aggregated

For the same assignments, if combination w' of cost value and resource use are covered by another combination w, combination w' is replaced by w.

Redundant assignments in the table are eliminated.

12

x1 x0 g1 vr0,vr1 a a b c d w w' b' c' d' g1 g1

If combination w' has higher cost value and resource use than

• ther combinations, combination w' is never preferred.

Global lower bounds of resource use

Each lower bound of resource use is subtracted/propagated to the root.

x1 x2 x0 x2 1 2 1 2 3

Resource use function u for r0

x1 1 2 1 2 2 x0 1 2 1 1 2 x1 x2 x0 x2 1 2 1 2 x1 1 2 1 1 x0 1 2 1

• 1
• 1
• 1

cr0←cr0 - 3 x1 x2 x0

Capacity of the resource is reduced by the global lower bound.

Reduced capacity cr0 of r0 is propagated.

13

r0

Global lower bound

• 3

Accuracy of infeasibility is improved.

u0(r0,x0) u1(r0,x1) u2(r0,x2) (preprocessing) (preprocessing) u0(r0,x0) u1(r0,x1) u2(r0,x2)

Replacing evaluations of resource use

x0 x1 x2 x3 r0 (a) original problem (b) equivalent problem x0 x1 x2 x3 r0 u0(r0,x0) x0 2 5 u0(r0,x0) x0 a b c a b c u0,2(r0,x0) x0 1 2 u0,3(r0,x0) x0 1 3 a b c a b c

Each resource use function is decomposed into agents that are:

1) constrained with the agent that originally has the resource use and 2) one of the lowest agent for each path of the pseudo-tree.

14

Accuracy of infeasibility is improved in the lower agents.

(preprocessing)

Aggregation of feasible assignments

cost and resource use cost vr0 w w' cost vr0 w w' In both cases, w' is replaced by w. (Only cost values are compared.) x1 x2 x1 x2 exact resource use for current sub-tree upper bound (maximum use) for other parts cr0 capacity of r0

15

x0 r0 x0 r0 (preprocessing)

If the global upper bound of resource use does not exceed the capacity, assignments are aggregated ignoring the resource use.

The aggregation complementally effects with the pruning based on infeasibility.

Experiments

Efficiency of the proposed methods are evaluated using example RCDCOPs:  n ternary variables  1.1n binary constraints (low density)  r n-ary resource constraints  each resource is shared by n/r agents  range of cost values is {1, …, 5}  range of the resource use for each variable is {1, …, u}  total amount of the resource is c・u・(n/r) (c: parameter) Combinations of the proposed methods are compared. The number of assignments in tables is evaluated.

16

Effects of additional methods

17

1 10 100 1000 10000 vi vir vira viras virasd virasdf vi vir vira viras virasd virasdf vi vir vira viras virasd virasdf r=1 r=4 r=10 total max. vi: base line vir: +trimming root nodes vira: +aggregation of assign. viras: +global lower bounds virasd: +replacing evaluation virasdf: +feasibility

• num. of assignments

While most methods reduces the number of assignments, using “feasibility” has no effect because of low feasibility (c=0.25).

(#variables: n=20, max. res. use: u=5, capacity of res.: c=0.25) (#resources)

1 10 100 1000 10000 min.0.25 0.5 0.75 1 min.0.25 0.5 0.75 1 min.0.25 0.5 0.75 1 r=1 r=4 r=10 ratio c for capacity of resources virasd: without feasibility (total) (max.) virasdf: +feasibility (total) (max.)

Effects of feasibility

18

In the case of relatively feasible problems, using “feasibility” reduces the number of assignments.

(#variables: n=20, max. res. use: u=5) infeasible feasible

• num. of assignments

(#resources)

Influence of num. of combination of resource use

19

1 10 100 1000 10000 1 5 10 1 5 10 1 5 10 r=1 r=4 r=10

• max. use u of resources

total max. virasd: without feasibility (#variables: n=20, capacity of res.: c=1 … completely feasible problem)

Wide range of resource use increases the number of combination of the resource use in assignments.

• num. of assignments

(#resources)

Influence of the correlation between cost and resource use 20

1 10 100 1000 10000 positive random negative positive random negative positive random negative r=1 r=4 r=10 correlation total max. virasd: without feasibility (#variables: n=20, capacity of res.: c=1 … completely feasible problem)

• num. of assignments

The correlation affects the aggregation of assignments. Negative correlation decreases the opportunity of the aggregation.

(#resources)

1 10 100 1000 10000 100000 20 40 60 80

• num. of variables

c=min. c=0.25 c=0.5 c=0.75 c=1

Influence of number of variables

21

virasdf: with all methods total num. of assignments

The search space grows with the size of problem (#variable). For large and difficult problems, relaxation/approximation methods are still necessary.

(#resources: r=4, limit of max. #assign.: 105)

(capacity of res.)

Conclusions

 Efficient methods to reduce search space of Resource Constrained DCOPs have been studied.

Pre-processing/add-on methods have been applied to a DP based solver. Reducing redundancy and improving accuracy of feasibility/infeasibility reduces the size of tables of DP. There are necessity/opportunities of non-exact methods.

 Future works

Applying the proposed methods to other solvers (e.g. branch-and-bound based solvers). Non-exact methods with a bounded error and low memory use.

22