? Stephan Schulz schulz@eprover.org Driving the State of the Art - - PowerPoint PPT Presentation

?

We know (nearly) nothing!

But can we learn? Stephan Schulz

schulz@eprover.org

Driving the State of the Art

Implementation Calculus Search Control

2

Driving the State of the Art

Implementation Calculus Search Control Where to search for proofs? What inference system to use? How to do inferences efficiently?

2

Evolution of Calculus

2000 4000 6000 8000 10000 50 100 150 200 250 300 "E 0.2 FOF/Calc" E 0.2 FOF

3

Evolution of Implementation

2000 4000 6000 8000 10000 50 100 150 200 250 300 "E 0.2 FOF" E 0.2 FOF Fast

4

Evolution of Implementation

2000 4000 6000 8000 10000 50 100 150 200 250 300 E 1.8 Best E 1.8 Slow

4

Evolution of Search Control/Clause Selection

2000 4000 6000 8000 10000 50 100 150 200 250 300 E 0.2 Goals E 0.2 Larry E 0.2 FOF E 0.2 SC

5

Evolution of Search Control/Literal Selection

2000 4000 6000 8000 10000 50 100 150 200 250 300 E 0.2 SmallestNegLit E 0.2 MaxLComplexAvoidPosPred E 0.2 FOF

6

Compare and Contrast

2000 4000 6000 8000 10000 50 100 150 200 250 300 "E 0.2 FOF/Calc" E 0.2 FOF 2000 4000 6000 8000 10000 50 100 150 200 250 300 "E 0.2 FOF" E 0.2 FOF Fast 2000 4000 6000 8000 10000 50 100 150 200 250 300 E 0.2 Goals E 0.2 Larry E 0.2 FOF E 0.2 SC 2000 4000 6000 8000 10000 50 100 150 200 250 300 E 0.2 SmallestNegLit E 0.2 MaxLComplexAvoidPosPred E 0.2 FOF 7

Improving heuristics has been the main source

• f progress in proof search!

8

. . . and our heuristics still suck!

500 1000 1500 problems 0.0 0.2 0.4 0.6 0.8 1.0 ratio Evolved 10SC11/FIFO SC11 FIFO

9

Humans are Inadequate!

◮ We are not good at keeping large amounts of data in our head ◮ We are not good at analysing large amounts of data without help ◮ We are not good visualising complex relationships

Compare “The Magical Number Seven, Plus or Minus Two”

10

Player of Games

◮ Chess

◮ State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where

◮ (Opening)

◮ Success: Capture of the king

11

Player of Games

◮ Chess

◮ State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where

◮ (Opening)

◮ Success: Capture of the king

◮ Go

◮ State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board

11

Player of Games

◮ Chess

◮ State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where

◮ (Opening)

◮ Success: Capture of the king

◮ Go

◮ State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board

◮ Saturating theorem proving

◮ State: Set of clauses ◮ Choice point: Which clause to process next?

◮ Pick term ordering, literal selection strategy

◮ Success: Derivation of the empty clause

U

g P

11

Player of Games

◮ Chess

◮ State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where

◮ (Opening)

◮ Success: Capture of the king

◮ Go

◮ State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board

◮ Saturating theorem proving

◮ State: Set of clauses ◮ Choice point: Which clause to process next?

◮ Pick term ordering, literal selection strategy

◮ Success: Derivation of the empty clause

U

g P

11

Player of Games

◮ Chess

◮ State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where

◮ (Opening)

◮ Success: Capture of the king

◮ Go

◮ State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board

◮ Saturating theorem proving

◮ State: Set of clauses ◮ Choice point: Which clause to process next?

◮ Pick term ordering, literal selection strategy

◮ Success: Derivation of the empty clause

U

g P

11

Player of Games

◮ Chess

◮ State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where

◮ (Opening)

◮ Success: Capture of the king

◮ Go

◮ State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board

◮ Saturating theorem proving

◮ State: Set of clauses ◮ Choice point: Which clause to process next?

◮ Pick term ordering, literal selection strategy

◮ Success: Derivation of the empty clause

U

g P

11

Player of Games

◮ Chess

◮ State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where

◮ (Opening)

◮ Success: Capture of the king

◮ Go

◮ State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board

◮ Saturating theorem proving

◮ State: Set of clauses ◮ Choice point: Which clause to process next?

◮ Pick term ordering, literal selection strategy

◮ Success: Derivation of the empty clause

U

g P

11

Player of Games

◮ Chess

◮ State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where

◮ (Opening)

◮ Success: Capture of the king

◮ Go

◮ State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board

◮ Saturating theorem proving

◮ State: Set of clauses ◮ Choice point: Which clause to process next?

◮ Pick term ordering, literal selection strategy

◮ Success: Derivation of the empty clause

?

11

Grand Challenge

Integrate Machine Learning and Symbolic Reasoning

12

Discussion

◮ Should we target domain-specific or more general search control

knowledge?

◮ Deep learning or hand-selected features - which is better for learning

search control knowledge?

◮ What is a better source for learning: Meta-information

(success/failure, time to success, . . . ), full proofs, or even full search protocols?

13

Discussion

◮ Should we target domain-specific or more general search control

knowledge?

◮ Deep learning or hand-selected features - which is better for learning

search control knowledge?

◮ What is a better source for learning: Meta-information

(success/failure, time to success, . . . ), full proofs, or even full search protocols?

Discuss away!

13