Reducing the Arity in Unbiased Black-Box Complexity Benjamin Doerr , - - PowerPoint PPT Presentation

Reducing the Arity in Unbiased Black-Box Complexity

Benjamin Doerr, Carola Winzen

Max-Planck-Institut für Informatik, Saarbrücken May 02, 2012 Supported by a Google Fellowship in Randomized Algorithms

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Rem inder: Black-Box Com plexity

• Allows an abstract view on Randomized Search Heuristics
• # queries until an optimum is queried for the first time?

Black-Box = “Oracle”

y f(y)

f

Algorithm A

(x ,f(x))

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Rem inder: Black-Box Com plexity

Black-Box = “Oracle”

f

Algorithm A

(x ,f(x)) (y ,f(y))

Expected number of function evaluations (=calls to the oracle) until an optimal solution is queried for the first time Runtim e of A for f : T (A ,f) Worst runtime of A among all functions f Runtim e of A for F : supf ∈ F T (A ,f) Best worst-case runtime among all algorithms A (Unrestricted) Black-Box Com plexity of F : infA supf ∈ F T (A,f)

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Drawbacks of the Unrestricted Black-Box Model

• NP-hard problems with low black-box complexity
• Max-Clique
• PARTITION
• .....
• Most classical test functions have “too low black-box

complexities”

• OneMax: Θ
• instead of Θ log

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Masterm ind-Version of OneMax

• Black-Box chooses ∈ 0,1
• Algorithm guesses x ∈ 0,1
• Black-Box answers

Black-Box = “Oracle” Algorithm A

OneMax 2

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Drawbacks of the Unrestricted Black-Box Model

• NP-hard problems with low black-box complexity
• Max-Clique
• PARTITION
• .....
• Most classical test functions have “too low black-box

complexities”

• OneMax: Θ
• instead of Θ log
• LeadingOnes: O log /log log instead of Θ
• ...
• Early end of black-box m odels??

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

The Unbiased Black-Box Model

• Revival of black-box studies by Lehre and Witt
• Observation: search heuristics sample unbiasedly

Black-Box = “Oracle”

x

f

Algorithm A sampled unbiasedly, i.e., in a “fair” way

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

The Unbiased Black-Box Model

• Revival of black-box studies by Lehre and Witt
• Observation: search heuristics sample unbiasedly
• Treat all bit positions and bit values in a “fair way”
• Intuitively:

“Flip the 5th bit” “Flip all zeros”

• Formally:
• queries must be sampled from an unbiased

distribution; i.e., a distribution D(.| ..) that satisfies for all x, y, … . , y, w and all ∈

• , … , ⨁ ⨁ , … , ⨁
• , … , , … ,

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

OneMax in the Unbiased Model

Unary unbiased BBC of OneMax is Ω log Unary Model [LW Gecco 20 10 ] Matched by many unary RSH

• Randomized Local Search
• (1+1) EA
• (μ λ) EAs (μ, λ constant

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

OneMax in the Unbiased Model

Unary unbiased BBC of OneMax is Ω log Unary Model [LW Gecco 20 10 ] k-ary unbiased BBC of OneMax is O/ log Arities 2 [DJKLWW Foga 20 11]

• Suggests that k-ary RSH are more

powerful than unary oned

• Not known to be matched by k-ary RSH

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

OneMax in the Unbiased Model

Unary unbiased BBC of OneMax is Ω log Unary Model [LW Gecco 20 10 ] k-ary unbiased BBC of OneMax is O/ log Arities 2 [DJKLWW Foga 20 11] k-ary unbiased BBC of OneMax is O/ Arities log 2 [DW Gecco 20 12]

"log -arity is as powerful as unrestrictedness”

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Why Should You Care?

1. Nice mathematics 

• 2. Techniques can be applied to other problems
• 3. Hope: find RSH that are provably faster than basic

algorithms

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Key techniques

1. Derandomized version of random sam pling technique Make cn/ log n random guesses For all y, z ∈ 0,1 with y z there exits an index i ∈ t such that w.h.p. Probabilistic method at its best: There exists a string-distinguishing sequence , … , of / log strings

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Key techniques

1. Derandomized version of random sam pling technique 2. Block-w ise identification of target string

• Cut board into blocks of length 2
• Apply random guessing technique to these blocks
• There are /2 blocks of length 2
• Random guessing: O(2/ log 2)= O(2/ k) guesses each
• Total number of guesses: /2 O(2/ k) = O(n/ k)

In the FOGA paper, we could handle only blocks of length k, yielding the O(n/ log k) bound

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Key techniques

1. Derandomized version of random sam pling technique 2. Block-w ise identification of target string 3. Sim ulating unrestrictedness Using k-ary operators, we can access 2k-1 bits in an almost unrestricted fashion

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Key techniques

1. Derandomized version of random sam pling technique 2. Block-w ise identification of target string 3. Sim ulating unrestrictedness 4. Storing the fitness values

creating unrestricted block “random guess” storing

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Sum m ary & Future Work

Future Work

• Lower bounds!!
• Cross-over based algorithms that match the upper bound?
• Other BB-models for higher arity algorithms
• .....

For log 2, the k-ary unbiased BBC

• f OneMax is O/

Main Result

Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

References

• Unrestricted Black-Box Model

[DJW06] Droste, Jansen, Wegener Upper and Low er Bounds for Random ized Search Heuristics in Black-box Optim ization ToCS 2006

• Unbiased Black-Box Models

[LW10] Lehre, Witt Black-Box Search by Unbiased Variation GECCO 2010 [RV11] Rowe, Vose Unbiased Black Box Search Algorithm s GECCO 2011 [DKLW11] Doerr, Kötzing, Lengler, Winzen Black-Box Com plexities of Com binatorial Problem s GECCO 2011

• OneMax-Complexities & (Derandomized) Random Sampling

[ER63] Erdős, Rényi On Tw o problem s of Inform ation Theory Magyar Tud. Akad. Mat. Kutató Int. Közl 1963 [AW09] Anil, Wiegand Black-Box Search by Elim ination of Fitness Functions FOGA 09 [DJKLWW11] Doerr, Johannsen, Kötzing, Lehre, Wagner, Winzen Faster Black-Box Algorithm s Through Higher Arity Operators FOGA 11 [DW12a] Doerr, Winzen Reducing the Arity in Unbiased Black-Box Com plexity GECCO 2012 [DW12] Doerr, Winzen Playing Masterm ind With Constant-Size Mem ory STACS 2012 [...] Many more!

• LeadingOnes-Complexities

[DW11] Doerr, Winzen Breaking the O(n log n) Barrier of LeadingOnes EA 2011

• Jumpk-Complexities & Black-Box Complexity of PARTITION

[DKW11] Doerr, Kötzing, Winzen Too Fast Unbiased Black-Box Algorithm s FOGA 11