Get Logical with Datalog Stuart Halloway Datomic Team, - - PowerPoint PPT Presentation

▶
get logical with datalog
SMART_READER_LITE
LIVE PREVIEW

Get Logical with Datalog Stuart Halloway Datomic Team, - - PowerPoint PPT Presentation

Dec 28, 2022 359 likes •1.27k views

Get Logical with Datalog Stuart Halloway Datomic Team, Clojure/core, Relevance 1 https://github.com/clojure/core.logic https://github.com/nathanmarz/cascalog http://datomic.com/ 2 Roadmap How Why Bonus Round problem /

slide-1
SLIDE 1

Get Logical with Datalog

Stuart Halloway

Datomic Team, Clojure/core, Relevance

1

slide-2
SLIDE 2

https://github.com/clojure/core.logic https://github.com/nathanmarz/cascalog http://datomic.com/

2

slide-3
SLIDE 3

Roadmap

  • How
  • Why
  • Bonus Round
  • problem / solution form

3

slide-4
SLIDE 4

Query Anatomy

q([:find ... :in ... :where ...], input1, ... inputN);

4

slide-5
SLIDE 5

Query Anatomy

q([:find ... :in ... :where ...], input1, ... inputN);

constraints

5

slide-6
SLIDE 6

Query Anatomy

q([:find ... :in ... :where ...], input1, ... inputN);

inputs

6

slide-7
SLIDE 7

Query Anatomy

q([:find ... :in ... :where ...], input1, ... inputN);

names for inputs

7

slide-8
SLIDE 8

Query Anatomy

q([:find ... :in ... :where ...], input1, ... inputN);

variables to return

8

slide-9
SLIDE 9

Variables

?customer ?orderId ?product ?email

9

slide-10
SLIDE 10

Constants

42 “john” :email #inst "2012-02-29" :order/id

10

slide-11
SLIDE 11

Keywords

42 “john” :email #inst "2012-02-29" :order/id

11

slide-12
SLIDE 12

Namespaces

42 “john” :email #inst "2012-02-29" :order/id

12

slide-13
SLIDE 13

Extensible Reader

42 “john” :email #inst "2012-02-29" :order/id

13

slide-14
SLIDE 14

Example Database

entity attribute value 42 :email jdoe@example.com 43 :email jane@example.com 42 :orders 107 42 :orders 141

14

slide-15
SLIDE 15

[?customer :email ?email]

Constrains the results returned, binds variables

Data Pattern

15

slide-16
SLIDE 16

[?customer :email ?email]

Constrains the results returned, binds variables

Data Pattern

entity attribute value

16

slide-17
SLIDE 17

[?customer :email ?email]

Constrains the results returned, binds variables

Data Pattern

constant

17

slide-18
SLIDE 18

[?customer :email ?email]

Constrains the results returned, binds variables

Data Pattern

variable variable

18

slide-19
SLIDE 19

entity attribute value 42 :email jdoe@example.com 43 :email jane@example.com 42 :orders 107 42 :orders 141

[?customer :email ?email]

19

slide-20
SLIDE 20

[42 :email ?email] “Find a particular customer’s email”

Constants Anywhere

20

slide-21
SLIDE 21

entity attribute value 42 :email jdoe@example.com 43 :email jane@example.com 42 :orders 107 42 :orders 141

[42 :email ?email]

21

slide-22
SLIDE 22

[42 ?attribute] “What attributes does customer 42 have?

Variables Anywhere

22

slide-23
SLIDE 23

entity attribute value 42 :email jdoe@example.com 43 :email jane@example.com 42 :orders 107 42 :orders 141

[42 ?attribute]

23

slide-24
SLIDE 24

[42 ?attribute ?value] “What attributes and values does customer 42 have?

Variables Anywhere

24

slide-25
SLIDE 25

entity attribute value 42 :email jdoe@example.com 43 :email jane@example.com 42 :orders 107 42 :orders 141

[42 ?attribute ?value]

25

slide-26
SLIDE 26

[:find ?customer :where [?customer :email]]

Where Clause

data pattern

26

slide-27
SLIDE 27

[:find ?customer :where [?customer :email]]

Find Clause

variable to return

27

slide-28
SLIDE 28

[:find ?customer :where [?customer :email] [?customer :orders]] “Find all the customers who have placed orders.”

Implicit Join

28

slide-29
SLIDE 29

import static datomic.Peer.q; q("[:find ?customer :where [?customer :id] [?customer :orders]]", db);

API

29

slide-30
SLIDE 30

import static datomic.Peer.q; q("[:find ?customer :where [?customer :id] [?customer :orders]]", db);

q

30

slide-31
SLIDE 31

import static datomic.Peer.q; q("[:find ?customer :where [?customer :id] [?customer :orders]]", db);

Query

31

slide-32
SLIDE 32

import static datomic.Peer.q; q("[:find ?customer :where [?customer :id] [?customer :orders]]", db);

Input(s)

32

slide-33
SLIDE 33

:in $database ?email

In Clause

Names inputs so you can refer to them elsewhere in the query

33

slide-34
SLIDE 34

“Find a customer by email.”

Parameterized Query

q([:find ?customer :in $database ?email :where [$database ?customer :email ?email]], db, "jdoe@example.com");

34

slide-35
SLIDE 35

“Find a customer by email.”

First Input

q([:find ?customer :in $database ?email :where [$database ?customer :email ?email]], db, "jdoe@example.com");

35

slide-36
SLIDE 36

“Find a customer by email.”

Second Input

q([:find ?customer :in $database ?email :where [$database ?customer :email ?email]], db, "jdoe@example.com");

36

slide-37
SLIDE 37

“Find a customer by email.”

Verbose?

q([:find ?customer :in $database ?email :where [$database ?customer :email ?email]], db, "jdoe@example.com");

37

slide-38
SLIDE 38

“Find a customer by email.”

Shortest Name Possible

q([:find ?customer :in $ ?email :where [$ ?customer :email ?email]], db, "jdoe@example.com");

38

slide-39
SLIDE 39

“Find a customer by email.”

Elide $ in Where

q([:find ?customer :in $ ?email :where [ ?customer :email ?email]], db, "jdoe@example.com"); no need to specify $

39

slide-40
SLIDE 40

[(< 50 ?price)]

Predicates

Functional constraints that can appear in a :where clause

40

slide-41
SLIDE 41

[:find ?item :where [?item :item/price ?price] [(< 50 ?price)]]

“Find the expensive items”

Adding a Predicate

41

slide-42
SLIDE 42

[(shipping ?zip ?weight) ?cost] Take bound variables as inputs and bind variables with output

Functions

42

slide-43
SLIDE 43

[(shipping ?zip ?weight) ?cost]

Function Args

bound inputs

43

slide-44
SLIDE 44

[(shipping ?zip ?weight) ?cost]

Function Returns

bind return values

44

slide-45
SLIDE 45

[:find ?customer ?product :where [?customer :shipAddress ?addr] [?addr :zip ?zip] [?product :product/weight ?weight] [?product :product/price ?price] [(Shipping/estimate ?zip ?weight) ?shipCost] [(<= ?price ?shipCost)]]

“Find me the customer/product combinations where the shipping cost dominates the product cost.”

Calling a Function

45

slide-46
SLIDE 46

[:find ?customer ?product :where [?customer :shipAddress ?addr] [?addr :zip ?zip] [?product :product/weight ?weight] [?product :product/price ?price] [(Shipping/estimate ?zip ?weight) ?shipCost] [(<= ?price ?shipCost)]]

“Find me the customer/product combinations where the shipping cost dominates the product cost.”

Calling a Function

navigate from customer to zip

46

slide-47
SLIDE 47

[:find ?customer ?product :where [?customer :shipAddress ?addr] [?addr :zip ?zip] [?product :product/weight ?weight] [?product :product/price ?price] [(Shipping/estimate ?zip ?weight) ?shipCost] [(<= ?price ?shipCost)]]

“Find me the customer/product combinations where the shipping cost dominates the product cost.”

Calling a Function

get product facts needed during query

47

slide-48
SLIDE 48

[:find ?customer ?product :where [?customer :shipAddress ?addr] [?addr :zip ?zip] [?product :product/weight ?weight] [?product :product/price ?price] [(Shipping/estimate ?zip ?weight) ?shipCost] [(<= ?price ?shipCost)]]

“Find me the customer/product combinations where the shipping cost dominates the product cost.”

Calling a Function

call web service to bind shipCost

48

slide-49
SLIDE 49

public class Shipping { public static BigDecimal estimate(String zip1, int pounds); }

Functions can be plain JVM code.

BYO Functions

49

slide-50
SLIDE 50

[:find ?customer ?product :where [?customer :shipAddress ?addr] [?addr :zip ?zip] [?product :product/weight ?weight] [?product :product/price ?price] [(Shipping/estimate ?zip ?weight) ?shipCost] [(<= ?price ?shipCost)]]

“Find me the customer/product combinations where the shipping cost dominates the product cost.”

Calling a Function

constrain price

50

slide-51
SLIDE 51

[:find ?customer ?product :where [?customer :shipAddress ?addr] [?addr :zip ?zip] [?product :product/weight ?weight] [?product :product/price ?price] [(Shipping/estimate ?zip ?weight) ?shipCost] [(<= ?price ?shipCost)]]

“Find me the customer/product combinations where the shipping cost dominates the product cost.”

Calling a Function

return customer, product pairs

51

slide-52
SLIDE 52

Why

52

slide-53
SLIDE 53

Why Clojure?

  • Data
  • good literals
  • immutable data
  • extensible reader
  • Platform
  • extensibility
  • performance
  • Lisp

53

slide-54
SLIDE 54

Why Datalog?

  • Equivalent to Relational Model + Recursion
  • Better fit than Prolog for query
  • No clause order dependency
  • Guaranteed termination
  • Pattern-matching style easy to learn

54

slide-55
SLIDE 55

Problem: Rectangles

  • join table
  • person table
  • club table
  • id key in person table
  • person key in join table
  • club key in join table
  • id key in club table

“People can belong to multiple clubs”

55

slide-56
SLIDE 56

Structural Navigation Structural Rigidity

56

slide-57
SLIDE 57

CoC Taxonomy

purpose assessment make an arbitrary choice

  • ften helpful

default the common choice

  • ften helpful

automate boilerplate dangerous

57

slide-58
SLIDE 58

Solution: Universal Relation

“People can belong to multiple clubs”

[?person :club ?club]

58

slide-59
SLIDE 59

Did You Ever Want To...

  • Make a column name variable?
  • Make a table name variable?
  • Treat metadata as first-class data?

59

slide-60
SLIDE 60

[?person ?attr ?value]

First-Class Attributes

attribute slot isn’t special

60

slide-61
SLIDE 61

[?e :db/valueType]

Schema Made of Ordinary Data

find all attributes

61

slide-62
SLIDE 62

SELECT ID FROM CUSTOMERS;

Problem: Ambient DB

in what db?

62

slide-63
SLIDE 63

q("[:find ?customer :where [?customer :id] [?customer :orders]]", db);

Solution: Explicit DB

in this db!

63

slide-64
SLIDE 64

q([:find ?customer :in $database ?email :where [$database ?customer :email ?email]], db, "jdoe@example.com");

Benefit: Query Params

parameterized query is not a separate feature

64

slide-65
SLIDE 65

(q '[:find ?k :in [[?k]]] (System/getProperties))

What system properties are available?

Benefit: BYO Data

65

slide-66
SLIDE 66

(q '[:find ?k :in [[?k]]] (System/getProperties))

What system properties are available?

Benefit: BYO Data

bind first element of each tuple in a relation

66

slide-67
SLIDE 67

Binding Patterns

Pattern Binds Example Input Binds ?a To ?a scalar 42 42 [?a ?b] tuple [1 2] 1 [?a ...] collection [1 2] 1, 2 [[?a ?b ?c]] relation

john likes pizza jane likes pasta

john, jane

67

slide-68
SLIDE 68

(q '[:find ?v :in [[?k ?v]] :where [(.endsWith ?k "path")]] (System/getProperties))

Which system properties are path-related?

BYO Data

68

slide-69
SLIDE 69

(q '[:find ?pathElem :in [[?k ?v]] :where [(.endsWith ?k "path")] [(.split ?v ":") [?pathElem ...]]] (System/getProperties))

What path elements are mentioned in system properties?

BYO Data

69

slide-70
SLIDE 70

(q '[:find ?pathElem :in [[?k ?v]] :where [(.endsWith ?k "path")] [(.split ?v ":") [?pathElem ...]] [(.endsWith ?pathElem ".jar")]] (System/getProperties))

What JAR files are in my system property paths?

BYO Data

70

slide-71
SLIDE 71

q("[:find ?customer :where [?customer :id] [?customer :orders]]", db.asOf(lastMonth));

Benefit: Time Travel

71

slide-72
SLIDE 72

[:find ?customer ?email :in $cust $emp :where [$cust ?customer :email ?email] [$emp _ :email ?email]]

“Find me the customers who are also employees.”

Benefit: Join Across DBs

q(query, custDb, empDb);

implicit join

72

slide-73
SLIDE 73

[:find ?customer ?email :in $cust $emp :where [$cust ?customer :email ?email] [$emp _ :email ?email]]

“Find me the customers who are also employees.”

Benefit: Join Across DBs

q(query, custDb, empDb);

data patterns can be led by database names

73

slide-74
SLIDE 74

Problem: Better Views

  • Good
  • abstraction
  • relational
  • Bad
  • over there
  • rectangular
  • tool/language

choices

74

slide-75
SLIDE 75

[(relatedProduct ?p1 ?p2) [?p1 :category ?c] [?p2 :category ?c] [(!= ?p1 ?p2)]]

“Products are related if they have a common category.”

Solution: Rules

75

slide-76
SLIDE 76

[(relatedProduct ?p1 ?p2) [?p1 :category ?c] [?p2 :category ?c] [(!= ?p1 ?p2)]]

“Products are related if they have a common category.”

Rule Head

this is true...

76

slide-77
SLIDE 77

[(relatedProduct ?p1 ?p2) [?p1 :category ?c] [?p2 :category ?c] [(!= ?p1 ?p2)]]

“Products are related if they have a common category.”

Rule Body

...if all these are true

77

slide-78
SLIDE 78

q("[:find ?p2 :in $ % :where (expensiveChocolate p1) (relatedProduct p1 p2)", db, rules)

“Find all products related to expensive chocolate.”

Using Rules

rules are a kind of input

78

slide-79
SLIDE 79

q("[:find ?p2 :in $ % :where (expensiveChocolate p1) (relatedProduct p1 p2)", db, rules)

“Find all products related to expensive chocolate.”

Using Rules

rule names begin with %

79

slide-80
SLIDE 80

q("[:find ?p2 :in $ % :where (expensiveChocolate p1) (relatedProduct p1 p2)", db, rules)

“Find all products related to expensive chocolate.”

Using Rules

rule patterns can appear in :where clause

80

slide-81
SLIDE 81

[[(relatedProduct ?p1 ?p2) [?p1 :category ?c] [?p2 :category ?c] [(!= ?p1 ?p2)]] [(relatedProduct ?p1 ?p2) [?o :order/item ?item1] [?item1 :order/product ?p1] [?o :order/item ?item2] [?item2 :order/product ?p2] [(!= ?p1 ?p2)]]]

“Products are related if they have the same category, or they have appeared in the same order.”

Implicit Or

81

slide-82
SLIDE 82

Problem: Extent

Line Item 1 Order Line Item 2 Product A Product B Customer

Get “the whole order”.

82

slide-83
SLIDE 83

Problem: Extent

Get “the whole order”.

Line Item 1 Order Line Item 2 Product A Product B Customer

83

slide-84
SLIDE 84

[(extent ?x ?e ?a ?v) (?e ?a ?v) (?x ?a ?v) [(= ?e ?x)]]

Find Values :x References

Line Item 1 Order Line Item 2 Product A Product B Customer

84

slide-85
SLIDE 85

[(extent ?x ?e ?a ?v) (?e ?a ?v) (?e ?a ?x) [(= ?v ?x)]]

Finds Entities Referencing :x

Line Item 1 Order Line Item 2 Product A Product B Customer

matches ref from customer, not customer itself

85

slide-86
SLIDE 86

[(extent ?x ?e ?a ?v) (components ?x ?y) (extent ?y ?e ?a ?v)] [(components ?p ?c) (?a :db/isComponent) (?p ?a ?c)]

Recurse Component Attributes

Line Item 1 Order Line Item 2 Product A Product B Customer

86

slide-87
SLIDE 87

[(extent ?x ?e ?a ?v) (components ?x ?y) (extent ?y ?e ?a ?v)] [(components ?p ?c) (?a :db/isComponent) (?p ?a ?c)]

Recurse Component Attributes

Line Item 1 Order Line Item 2 Product A Product B Customer

recursive definition

87

slide-88
SLIDE 88

[(extent ?x ?e ?a ?v) (components ?x ?y) (extent ?y ?e ?a ?v)] [(components ?p ?c) (?a :db/isComponent) (?p ?a ?c)]

Recurse Component Attributes

Line Item 1 Order Line Item 2 Product A Product B Customer

  • nly recurse attributes

marked :db/isComponent

88

slide-89
SLIDE 89

Stuart Halloway

datomic.com @datomic_team @stuarthalloway

89