Part II: Enhancing ATPs with Machine Learning Course Machine - - PowerPoint PPT Presentation

▶
part ii enhancing atps with machine learning
SMART_READER_LITE
LIVE PREVIEW

Part II: Enhancing ATPs with Machine Learning Course Machine - - PowerPoint PPT Presentation

Nov 05, 2022 128 likes •814 views

Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine Part II: Enhancing ATPs with Machine Learning Course Machine Learning and Reasoning 2020 MLR 2020 1 1 Czech Technical Univeristy in Prague (CIIRC) April 3, 2020 1/56

slide-1
SLIDE 1

1/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Part II: Enhancing ATPs with Machine Learning

Course Machine Learning and Reasoning 2020 MLR 20201

1Czech Technical Univeristy in Prague (CIIRC)

April 3, 2020

slide-2
SLIDE 2

2/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

1

Automated Strategy Invention BliStr: Blind Strategy Maker BliStrTune: Hierarchical Tuning EmpireTune: Term Orderings Invention

2

ENIGMA: Efficient Inference Guidance Machine Basic Enigmas Enhancing Enigma Enigma Anonymous

slide-3
SLIDE 3

3/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Outline

1

Automated Strategy Invention BliStr: Blind Strategy Maker BliStrTune: Hierarchical Tuning EmpireTune: Term Orderings Invention

2

ENIGMA: Efficient Inference Guidance Machine Basic Enigmas Enhancing Enigma Enigma Anonymous

slide-4
SLIDE 4

4/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Using Automated Theorem Provers

Solve problems in First-Order Logic Built-in automated strategy selection E Prover: $ eprover --auto-schedule problem.tptp Vampire: $ vampire --mode casc problem.tptp

slide-5
SLIDE 5

5/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

No Success?

$ eprover --auto-schedule problem.tptp ... # Failure: Resource limit exceeded (time) # SZS status ResourceOut eprover: CPU time limit exceeded, terminating

slide-6
SLIDE 6

6/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Try your own strategy!

$ eprover --auto-schedule problem.tptp

slide-7
SLIDE 7

6/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Try your own strategy!

$ eprover --definitional-cnf=24 --oriented-simul-paramod \

  • -forward-context-sr --destructive-er-aggressive \
  • -destructive-er --prefer-initial-clauses -tAuto \
  • Garity -F1 -WSelectMaxLComplexAvoidPosPred \
  • H(1*ConjectureRelativeTermWeight(PreferProcessed,1,1,1,...), \

1*ConjectureTermPrefixWeight(SimulateSOS,1,3,0.5,10,...), \ 34*ConjectureRelativeTermWeight(DeferSOS,1,3,0.2,10,...)) \ problem.tptp

slide-8
SLIDE 8

6/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Try your own strategy!

$ eprover --definitional-cnf=24 --oriented-simul-paramod \

  • -forward-context-sr --destructive-er-aggressive \
  • -destructive-er --prefer-initial-clauses -tAuto \
  • Garity -F1 -WSelectMaxLComplexAvoidPosPred \
  • H(1*ConjectureRelativeTermWeight(PreferProcessed,1,1,1,...), \

1*ConjectureTermPrefixWeight(SimulateSOS,1,3,0.5,10,...), \ 34*ConjectureRelativeTermWeight(DeferSOS,1,3,0.2,10,...)) \ problem.tptp # Proof found! # SZS status Theorem

slide-9
SLIDE 9

7/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Our Task

Invent targeted strategies for E . . . specific for a given benchmark set . . . using machine learning methods (BliStrTune). . . . also for Vampire – EmpireTune

slide-10
SLIDE 10

8/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BliStr: Blind Strategy Maker

slide-11
SLIDE 11

9/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BliStr Basics

giraffes = strategies food = problems the better a giraffe specializes . . . . . . the more it gets fed and evolves

slide-12
SLIDE 12

10/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Strategy Invention: BliStr Loop

Input: Initial strategies & benchmark problems Output: Strategies which perform better on the benchmark All := Initials ; loop Evaluate (All, eval, min, max ) ; G := Reduce (All, tops, bests ) ; S := S e l e c t (G ) ; i f S i s undefined then r e t u r n G ; S1 := Improve (S, cutoff , imp ) ; All := All Y tS1u ; end

slide-13
SLIDE 13

11/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Step 1/4: Generation Evaluation

evaluate all the strategies on all the problems compute overall result (solved/unsolved) measure length of the proof search for each strategy, compute best-performing problems discard too easy and too hard problems

slide-14
SLIDE 14

12/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Step 2/4: Generation Reduction

consider only strategies performing best on bests problems . . . restrict the size of individuals keep only tops best strategies . . . restrict the count of individuals

slide-15
SLIDE 15

13/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Step 3/4: Strategy Selection

select a strategy to improve . . . on its best performing problems never improve a strategy on the same problems prefer strategies with more best-performing problems prefer improving strategies on diverse problems

slide-16
SLIDE 16

14/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Step 4/4: Strategy Improvement

improve a strategy on its best-performing problems using ParamILS software1 2 . . . parameter tuning and algorithm configuration different BliStr “clones” use ParamILS differently

BliStr: single ParamILS run BliStrTune: several “hierarchical” ParamILS runs EmpireTune: hierarchical runs for E, single run for Vampire

1http://www.cs.ubc.ca/labs/beta/Projects/ParamILS/ 2Frank Hutter, Holger Hoose, . . . (Uni. British Columbia)

slide-17
SLIDE 17

15/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BliStr: Basic Tuning

One ParamILS run for one strategy improvement Given: initial strategy, set of problems, parameter space Task: find a better strategy w.r.t. the objective penalty ˚ |unsolved| ` ÿ

PPsolved

processedpPq e.g. 23, 001, 234 with penalty “ 1, 000, 000

slide-18
SLIDE 18

16/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BliStrTune: Hierarchical Tuning

BliStr explores a limited E protocol space Considers fixed set of clause weight functions Only 12 fixed functions:

ConjTermWeight(ConstPrio,0,1,0.1,18,400,50,300) ConjSymbolWeight(PreferGround,0.2,50,100,5) ...

slide-19
SLIDE 19

17/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BliStrTune: Global Tuning Phase

Explore different values for top-level parameters. . .

  • tKBO6 -Garity -WSelectComplexG –oriented-simul-paramod -H’(

3 * ConjTermWeight(ConstPrio,0,1,0.1,18,400,50,300), 34 * ConjTermWeight(PreferUnits,1,1,0.1,100,9999,100,5), 8 * ConjSymbolWeight(PreferGround,0.2,50,100,5) )’

slide-20
SLIDE 20

17/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BliStrTune: Global Tuning Phase

Explore different values for top-level parameters. . .

  • tKBO6 -Garity -WSelectComplexG –oriented-simul-paramod -H’(

3 * ConjTermWeight(ConstPrio,0,1,0.1,18,400,50,300), 34 * ConjTermWeight(PreferUnits,1,1,0.1,100,9999,100,5), 8 * ConjSymbolWeight(PreferGround,0.2,50,100,5) )’

slide-21
SLIDE 21

18/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BliStrTune: Between Phases

. . . some values get improved.

  • tLPO4 -Ginvarity -WSelectComplexG -H’(

3 * ConjSymbolWeight(PreferGround,0.2,50,100,5) 4 * ConjTermWeight(ConstPrio,0,1,0.1,18,400,50,300), 23 * ConjTermWeight(PreferUnits,1,1,0.1,100,9999,100,5), 16 * ConjPrefixWeight(PreferGoals,0.2,50,100,5) )’

slide-22
SLIDE 22

18/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BliStrTune: Between Phases

Next, improve weight function arguments, . . .

  • tLPO4 -Ginvarity -WSelectComplexG -H’(

3 * ConjSymbolWeight(PreferGround,0.2,50,100,5) 4 * ConjTermWeight(ConstPrio,0,1,0.1,18,400,50,300), 23 * ConjTermWeight(PreferUnits,1,1,0.1,100,9999,100,5), 16 * ConjPrefixWeight(PreferGoals,0.2,50,100,5) )’

slide-23
SLIDE 23

19/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BliStrTune: Fine Tuning Phase

. . . and values get changed again.

  • tLPO4 -Ginvarity -WSelectComplexG -H’(

3 * ConjSymbolWeight(ConstPrio,0.4,10,10,50) 4 * ConjTermWeight(PreferGround,0,1,1.5,9,100,50,300), 23 * ConjTermWeight(PreferGoals,1,1,-0.1,200,9,100,5), 16 * ConjPrefixWeight(PreferUnits ,0.2,10,100,5) )’

slide-24
SLIDE 24

20/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BliStrTune: Experiments

Mizar @ Turing division of competition CASC’12 problems exported from Mizar 1000 training problems known beforehand 400 testing problems in the competition

slide-25
SLIDE 25

21/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BlistrTune: Progress on Testing Problems

slide-26
SLIDE 26

22/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

BlistrTune: Impact of Hierarchical Tuning

slide-27
SLIDE 27

23/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

EmpireTune: E and Vampire Tuning

select Vampire options for strategy selection . . . (Sine, saturation alg., AVATAR, inference rules) describe their possible values rule out incompatible combinations sa { discount , inst gen , l r s , o t t e r } [ o t t e r ] erd { off , i n p u t o n l y } [ i n p u t o n l y ] fde { a l l , none , unused } [ unused ] gsp { input only , o f f } [ o f f ] i n s {0 ,1 ,2 ,4 ,8} [ 0 ] . . .

slide-28
SLIDE 28

24/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Term Orderings in ATP

partial ordering on terms (from a symbol precedence) used to restrict and guide a proof search . . . ordered resolution, orient rewriting rules for n symbols, n! precedences the right ordering can have a dramatical effect however, not clear which one is the right one

slide-29
SLIDE 29

25/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Term Orderings in Vampire 4.2

Standard Vampire:

  • ccurrence - order symbols by their occurrence in the problem

arity - order symbols by their arity frequency - order symbols by frequency in the problem

slide-30
SLIDE 30

26/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Term Orderings in Vampire 4.2

EmpireTune extension: user specifies coefficients: spoc, spac, spfc valpsq “ spoc ˚ occpsq ` spac ˚ aritypsq ` spfc ˚ freqpsq symbols are order by valpsq additionally: explicitly specified precedence

slide-31
SLIDE 31

27/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Tuning Ordering in EmpireTune

use ParamILS to find the best possible ordering . . . best values for spoc, spac, spfc . . . or best explicit precedence hierarchical approach:

1

tune everything except ordering

2

tune ordering only

slide-32
SLIDE 32

28/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Example: Tuning Ordering in EmpireTune

Take the input Vampire strategy:

  • av off -awr 2:3 -bd preordered -drc off -fd preordered -fde unused
  • fsr off -nm 64 -s -1004 -sa otter -sas z3 -updr off -urr on
  • spoc 1 -spac 2 -spfc 1
slide-33
SLIDE 33

28/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Example: Tuning Ordering in EmpireTune

Phase 1: Allow only non-ordering options to be changed:

  • av off -awr 2:3 -bd preordered -drc off -fd preordered -fde unused
  • fsr off -nm 64 -s -1004 -sa otter -sas z3 -updr off -urr on
  • spoc 1 -spac 2 -spfc 1
slide-34
SLIDE 34

28/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Example: Tuning Ordering in EmpireTune

Phase 1 output: New Vampire strategy:

  • av off -awr 5:4 -bce on -bd preordered -drc off -fd preordered -fde

unused -fsr off -nm 26 -s 1004 -sa otter -sas z3 -updr off -urr on

  • spoc 1 -spac 2 -spfc 1
slide-35
SLIDE 35

28/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Example: Tuning Ordering in EmpireTune

Phase 2: Allow only ordering options to be changed:

  • av off -awr 5:4 -bce on -bd preordered -drc off -fd preordered -fde

unused -fsr off -nm 26 -s 1004 -sa otter -sas z3 -updr off -urr on

  • spoc 1 -spac 2 -spfc 1
slide-36
SLIDE 36

28/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Example: Tuning Ordering in EmpireTune

Phase 2 output: New Vampire strategy:

  • av off -awr 5:4 -bce on -bd preordered -drc off -fd preordered -fde

unused -fsr off -nm 26 -s 1004 -sa otter -sas z3 -updr off -urr on

  • spoc 0 -spuc 1 -fp identity:0:1,associator:3:2,multiply:2:3
slide-37
SLIDE 37

29/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Experiments Setting

AIM problems from CASC (LTB category) . . . 1020 training problems, 200 testing problems advantages (simplifications): . . . different conjectures in the same theory . . . small number of symbols (8+4) . . . symbols are used consistently

slide-38
SLIDE 38

30/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

EmpireTune: Impact of Tuning

10 20 30 40 50 60 70 80 E 2 . ( a u t

  • s

c h e d u l e ) V a m p i r e 4 . 2 ( C A S C m

  • d

e ) E 2 . ( E m p i r e T u n e ) V a m p i r e 4 . 2 ( E m p i r e T u n e ) 5 10 15 20 Solved [count] Tune time [days] Solved Tune time

slide-39
SLIDE 39

31/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

EmpireTune: Impact of Ordering Tuning

10 20 30 40 50 60 V a m p i r e 4 . 2 E m p i r e T u n e ( n

  • r

d e r i n g ) E m p i r e T u n e ( w i t h

  • r

d e r i n g ) 5 10 15 20 Solved [count] Tune time [days] Solved Tune time

slide-40
SLIDE 40

32/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Outline

1

Automated Strategy Invention BliStr: Blind Strategy Maker BliStrTune: Hierarchical Tuning EmpireTune: Term Orderings Invention

2

ENIGMA: Efficient Inference Guidance Machine Basic Enigmas Enhancing Enigma Enigma Anonymous

slide-41
SLIDE 41

33/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Enigma Basics

Idea: Use fast linear classifier to guide given clause selection! ENIGMA stands for. . .

slide-42
SLIDE 42

33/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Enigma Basics

Idea: Use fast linear classifier to guide given clause selection! ENIGMA stands for. . . Efficient learNing-based Inference Guiding MAchine

slide-43
SLIDE 43

34/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

LIBLINEAR: Linear Classifier

LIBLINEAR: open source library3 input: positive and negative examples (float vectors)

  • utput: model („ a vector of weights)

evaluation of a generic vector: dot product with the model

3http://www.csie.ntu.edu.tw/~cjlin/liblinear/

slide-44
SLIDE 44

35/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Clauses as Feature Vectors

Consider the literal as a tree and simplify (sign, vars, skolems). “ f x y g sko1 sko2 x Ñ ‘ “ f f f g d d f

slide-45
SLIDE 45

35/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Clauses as Feature Vectors

Features are descending paths of length 3 (triples of symbols). “ f x y g sko1 sko2 x Ñ ‘ “ f f f g d d f

slide-46
SLIDE 46

36/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Clauses as Feature Vectors

Collect and enumerate all the features. Count the clause features. ‘ “ f f f g d d f # feature count 1 (‘,=,a) . . . . . . . . . 11 (‘,=,f) 1 12 (‘,=,g) 1 13 (=,f,f) 2 14 (=,g,d) 2 15 (g,d,f) 1 . . . . . . . . .

slide-47
SLIDE 47

36/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Clauses as Feature Vectors

Take the counts as a feature vector. ‘ “ f f f g d d f # feature count 1 (‘,=,a) . . . . . . . . . 11 (‘,=,f) 1 12 (‘,=,g) 1 13 (=,f,f) 2 14 (=,g,d) 2 15 (g,d,f) 1 . . . . . . . . .

slide-48
SLIDE 48

37/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Horizontal Features

Function applications and arguments top-level symbols. ‘ “ f f f g d d f # feature count 1 (‘,=,a) . . . . . . . . . 100 “ pf , gq 1 101 f pf, fq 1 102 gpd, dq 1 103 dpfq 1 . . . . . . . . .

slide-49
SLIDE 49

38/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Static Clause Features

For a clause, its length and the number of pos./neg. literals. ‘ “ f f f g d d f # feature count/val 103 dpfq 1 . . . . . . . . . 200 len 9 201 pos 1 202 neg . . . . . . . . .

slide-50
SLIDE 50

39/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Static Symbol Features

For each symbol, its count and maximum depth. ‘ “ f f f g d d f # feature count/val 202 neg . . . . . . . . . 300 #‘pf q 1 301 #apf q . . . . . . . . . 310 %‘pfq 4 311 %apfq . . . . . . . . .

slide-51
SLIDE 51

39/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Static Symbol Features

For each symbol, its count and maximum depth. ‘ “ f f f g d d f # feature count/val 202 neg . . . . . . . . . 300 #‘pf q 1 301 #apf q . . . . . . . . . 310 %‘pfq 4 311 %apfq . . . . . . . . .

slide-52
SLIDE 52

40/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Enigma Model Construction

1 Collect training examples from E runs (useful/useless clauses). 2 Enumerate all the features (π :: feature Ñ int). 3 Translate clauses to feature vectors. 4 Train a LIBLINEAR classifier (w :: float|dompπq|). 5 Enigma model is M “ pπ, wq.

slide-53
SLIDE 53

41/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Conjecture Features

Enigma classifier M is independent on the goal conjecture! Improvement: Extend ΦC with goal conjecture features. Instead of vector ΦC take vector pΦC, ΦGq.

slide-54
SLIDE 54

42/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Given Clause Selection by Enigma

We have Enigma model M “ pπ, wq and a generated clause C.

1 Translate C to feature vector ΦC using π. 2 Compute prediction:

weightpCq “ # 1 iff w ¨ ΦC ą 0 10

  • therwise
slide-55
SLIDE 55

43/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Enigma Given Clause Selection

We have implemented Enigma weight function in E. Given E strategy S and model M. Construct new E strategy: S d M: Use M as the only weight function: (1 * Enigma(M)) S ‘ M: Insert M to the weight functions from S: (23 * Enigma(M), 3 * StandardWeight(...), 20 * StephanWeight(...))

slide-56
SLIDE 56

44/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Decision Tree Models

Idea: Use decision trees instead of linear classifier. Gradient boosting library XGBoost/LightGBM Provides C/C++ API and Python (and others) interface. Uses exactly the same training data as LIBLINEAR. We use the same Enigma features. No need for training data balancing.

slide-57
SLIDE 57

45/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

XGBoost/LightGBM Models

An model consists of a set of decision trees. Leaf scores are summed and translated into a probability.

slide-58
SLIDE 58

46/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Feature Hashing

With lot of training samples we have lot of features. LIBLINEAR/XGBoost can’t handle too long vectors (ą 105). Why? Input too big. . . Training takes too long. . . Solution: Reduce vector dimension with feature hashing. Encode features by strings and . . . . . . use a general purpose string hashing function. Values are summed in the case of a collision.

slide-59
SLIDE 59

47/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Experiments: Hammering Mizar

MPTP: FOL translation of selected articles from Mizar Mathematical Library (MML). Contains 57, 880 problems. Small versions with (human) premise selection applied. Single good-performing E strategy S fixed. All strategies evaluated with time limit of 10 seconds.

slide-60
SLIDE 60

48/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Solved problems: one looping iteration

Decision trees depth = 9 M0 is trained on problems solved by S Mn (n ą 0) is trained on problems solved by S and S d Mi (for all i ă n) and S ‘ Mi (for all i ă n) S S d M0 S ‘ M0 S d M1 S ‘ M1 solved 14933 16574 20366 21564 22839 S% +0% +10.5% +35.8% +43.8% +52.3% S` +0 +4364 +6215 +7774 +8414 S´

  • 2723
  • 782
  • 1143
  • 508
slide-61
SLIDE 61

49/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Solved problems: more loops

S S ‘ M0 S ‘ M1 S ‘ M2 S ‘ M3 solved 14933 20366 22839 23467 23753 S% +0% +35.8% +52.3% +56.5% +58.4 S` +0 +6215 +8414 +8964 +9274 S´

  • 782
  • 508
  • 430
  • 454
slide-62
SLIDE 62

50/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Solved problems: deeper trees

Increase tree depth to 12 and 16 Train the model on the same data as M3 S d M3

12

S ‘ M3

12

S d M3

16

S ‘ M3

16

solved 24159 24701 25100 25397 S% +61.1% +64.8% +68.0% +70.0% S` +9761 +10063 +10476 +10647 S´

  • 535
  • 295
  • 309
  • 183
slide-63
SLIDE 63

51/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Training Statistics: different tree depths

1.8 M features (hashed to 215) vector dimension is 216 input trains file is 38 GB . . . and contains 63 M training samples (4.2M pos x 59M neg) . . . with 5000 M non-zero values (density 0.1%) depth error real time CPU time size (MB) speed 9 0.201 2h41m 4d20h 5.0 5665.6 12 0.161 4h12m 8d10h 17.4 4676.9 16 0.123 6h28m 11d18h 54.7 3936.4

slide-64
SLIDE 64

52/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Symbol Anonymization

Replace symbol names in features by their arities. identity Ñ f0 plus(X,Y) Ñ f2(X,Y) multiply(X,Y) Ñ f2(X,Y) less(X,Y) Ñ p2(X,Y)

slide-65
SLIDE 65

53/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Additional Symbol Independent Features

Additional variable/symbols statistics. the number of variables/symbols in a clause the number of variables/with with one/more occurrences the number of occurrences of the most occurring variable . . .

slide-66
SLIDE 66

54/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Hammering Mizar Anonymously

M TPR TNR training real time [%] [%] size time params S ‘ M `% H

  • 14 966

0.0 D0 84.9 68.4 14M 2h29m X,d12 20 679 38.1 D1 79.0 79.5 29M 4h33m X,d12 23 679 58.2 D2 80.5 79.2 47M 40m L,d30,l1800 24 347 62.7

slide-67
SLIDE 67

55/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Solved in Time & Processed Clauses

slide-68
SLIDE 68

56/56 Automated Strategy Invention ENIGMA: Efficient Inference Guidance Machine

Conclusion

Thank You! Good Bye!