Diamond: Automating Data Management and Storage for Wide-area, - - PowerPoint PPT Presentation

▶
diamond automating data management and storage for wide
SMART_READER_LITE
LIVE PREVIEW

Diamond: Automating Data Management and Storage for Wide-area, - - PowerPoint PPT Presentation

Oct 30, 2023 386 likes •1.22k views

Diamond: Automating Data Management and Storage for Wide-area, Reactive Applications Irene Zhang Niel Lebeck Pedro Fonseca Brandon Holt Raymond Cheng Ariadna Norberg Arvind Krishnamurthy Henry M. Levy 1 Diamond:

slide-1
SLIDE 1

Diamond: Automating 
 Data Management and Storage 
 for Wide-area, Reactive Applications

Irene Zhang Niel Lebeck Pedro Fonseca 
 Brandon Holt Raymond Cheng 
 Ariadna Norberg Arvind Krishnamurthy Henry M. Levy

1

slide-2
SLIDE 2

Diamond: Automating 
 Data Management and Storage 
 for Wide-area, Reactive Applications

Irene Zhang Niel Lebeck Pedro Fonseca 
 Brandon Holt Raymond Cheng 
 Ariadna Norberg Arvind Krishnamurthy Henry M. Levy

1

slide-3
SLIDE 3

Reactive applications automatically propagate updates across mobile devices and the cloud.

2

slide-4
SLIDE 4

Pin The Advisor Irene Niel Pin The Advisor Irene Niel

Reactive applications automatically propagate updates across mobile devices and the cloud.

2

slide-5
SLIDE 5

Pin The Advisor Irene Niel Pin The Advisor Irene Niel You earned 5 pts!

Reactive applications automatically propagate updates across mobile devices and the cloud.

5

2

slide-6
SLIDE 6

Pin The Advisor Irene Niel Pin The Advisor Irene Niel 5

Reactive applications automatically propagate updates across mobile devices and the cloud.

5

2

It’s your turn!

slide-7
SLIDE 7

Pin The Advisor Irene Niel Pin The Advisor Irene Niel 5

Reactive applications automatically propagate updates across mobile devices and the cloud.

Availability

5

2

It’s your turn!

slide-8
SLIDE 8

Pin The Advisor Irene Niel Pin The Advisor Irene Niel 5

Reactive applications automatically propagate updates across mobile devices and the cloud.

Fault-tolerance Availability

5

2

It’s your turn!

slide-9
SLIDE 9

Pin The Advisor Irene Niel Pin The Advisor Irene Niel 5

Reactive applications automatically propagate updates across mobile devices and the cloud.

Fault-tolerance Availability Consistency

5

2

It’s your turn!

slide-10
SLIDE 10

Pin The Advisor

5 Irene Niel You earned 5 pts!

Which poses a challenge for app programmers.

Pin The Advisor

Irene Niel

3

slide-11
SLIDE 11

Pin The Advisor

5 Irene Niel You earned 5 pts!

player1 player1 turn irene

App

player1 player1 turn irene

App

Which poses a challenge for app programmers.

Pin The Advisor

Irene Niel

3

slide-12
SLIDE 12

Pin The Advisor

5 Irene Niel You earned 5 pts!

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

Which poses a challenge for app programmers.

Pin The Advisor

Irene Niel

3

slide-13
SLIDE 13

Pin The Advisor

5 Irene Niel You earned 5 pts!

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

Which poses a challenge for app programmers.

5

Pin The Advisor

Irene Niel

3

slide-14
SLIDE 14

Pin The Advisor

5 Irene Niel You earned 5 pts!

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

Which poses a challenge for app programmers.

5 niel

Pin The Advisor

Irene Niel

3

slide-15
SLIDE 15

Pin The Advisor

5 Irene Niel You earned 5 pts!

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

write

Which poses a challenge for app programmers.

5 niel 5 niel

Pin The Advisor

Irene Niel

3

slide-16
SLIDE 16

1

Pin The Advisor

5 Irene Niel You earned 5 pts!

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

write

Which poses a challenge for app programmers.

5 niel 5 niel

Pin The Advisor

Irene Niel

3

slide-17
SLIDE 17

1

Pin The Advisor

5 Irene Niel You earned 5 pts!

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

write

Which poses a challenge for app programmers.

5 niel 5 niel

Pin The Advisor

Irene Niel

3

Notification
 Service

slide-18
SLIDE 18

1

Pin The Advisor

5 Irene Niel You earned 5 pts!

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

write notify

Which poses a challenge for app programmers.

5 niel 5 niel

Pin The Advisor

Irene Niel

notify

3

Notification
 Service

2 3

slide-19
SLIDE 19

1

Pin The Advisor

5 Irene Niel You earned 5 pts!

r e a d

4

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

write notify

Which poses a challenge for app programmers.

5 niel 5 niel

Pin The Advisor

Irene Niel

notify

5 niel

5

3

Notification
 Service

2 3

It’s your turn!

slide-20
SLIDE 20

1

r e a d

4

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

write notify

Which poses a challenge for app programmers.

5 niel 5 niel

Pin The Advisor

Irene Niel

notify

5 niel

5

3

Notification
 Service

2 3

It’s your turn!

slide-21
SLIDE 21

1 4

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

write

Which poses a challenge for app programmers.

5 niel 5 niel

Pin The Advisor

Irene Niel

3

Notification
 Service

2 3

slide-22
SLIDE 22

1

Pin The Advisor

5 Irene Niel You earned 5 pts!

4

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

write

Which poses a challenge for app programmers.

5 niel 5 niel

Pin The Advisor

Irene Niel

3

Notification
 Service

2 3

slide-23
SLIDE 23

1

Pin The Advisor

5 Irene Niel You earned 5 pts!

4

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

write

Which poses a challenge for app programmers.

5 niel 5 niel

This is a complex, distributed data management problem!

Pin The Advisor

Irene Niel

3

Notification
 Service

2 3

slide-24
SLIDE 24

1

Pin The Advisor

5 Irene Niel You earned 5 pts!

4

irene niel turn irene

Storage

player1 player1 turn irene

App

player1 player1 turn irene

App

write

Which poses a challenge for app programmers.

5 niel 5 niel

This is a complex, distributed data management problem!

Pin The Advisor

Irene Niel

3

Notification
 Service

Conclusion: Reactive applications require end-to-end data management with strong guarantees.

2 3

slide-25
SLIDE 25

Which poses a challenge for app programmers.

4

Conclusion: Reactive applications require end-to-end data management with strong guarantees.

slide-26
SLIDE 26

Which poses a challenge for app programmers.

4

Conclusion: Reactive applications require end-to-end data management with strong guarantees.

slide-27
SLIDE 27

Which poses a challenge for app programmers.

4

Conclusion: Reactive applications require end-to-end data management with strong guarantees.

slide-28
SLIDE 28

Which poses a challenge for app programmers.

4

Conclusion: Reactive applications require end-to-end data management with strong guarantees.

slide-29
SLIDE 29

Which poses a challenge for app programmers.

4

Conclusion: Reactive applications require end-to-end data management with strong guarantees.

slide-30
SLIDE 30

Which poses a challenge for app programmers.

4

Conclusion: Reactive applications require end-to-end data management with strong guarantees.

slide-31
SLIDE 31

Diamond

Diamond is the first reactive data management service, which provides the following guarantees:

  • Ensures updates to shared data are consistent and durable
  • Coordinates and synchronizes updates reliably across mobile

clients and cloud storage

  • Automatically triggers application code in response to

updates to shared data

5

slide-32
SLIDE 32

Talk Outline

Diamond System & Programming Model
 What does Diamond provide for reactive apps?
 Diamond Guarantees & Implementation
 What does Diamond guarantee for reactive apps?
 Evaluation
 How does Diamond impact app complexity and performance?


6

slide-33
SLIDE 33

Distributed Key-Value Store

Diamond System Model

Front-end Servers

Diamond Cloud

App Process

Client Devices

libDiamond App Process libDiamond

7

slide-34
SLIDE 34

Diamond Programming Model

Diamond Cloud App libDiamond Store App libDiamond

8

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-35
SLIDE 35

Pin The Advisor

Irene Niel

PinAdvisor.cc

x

  • +
  • Simple data structures including primitives

(e.g., string, long), collections (e.g., list) and Conflict-free Data Types (e.g., counter, set)

  • Data type semantics avoid false sharing and

enable commutative operations

  • Defined in libDiamond language bindings

App libDiamond

9

Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-36
SLIDE 36

Pin The Advisor

Irene Niel

PinAdvisor.cc

x

  • +
  • Simple data structures including primitives

(e.g., string, long), collections (e.g., list) and Conflict-free Data Types (e.g., counter, set)

  • Data type semantics avoid false sharing and

enable commutative operations

  • Defined in libDiamond language bindings

DCounter player1;

player1

App libDiamond

9

Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-37
SLIDE 37

Pin The Advisor

Irene Niel

PinAdvisor.cc

x

  • +
  • Simple data structures including primitives

(e.g., string, long), collections (e.g., list) and Conflict-free Data Types (e.g., counter, set)

  • Data type semantics avoid false sharing and

enable commutative operations

  • Defined in libDiamond language bindings

DCounter player1; DCounter player2;

player1 player2

App libDiamond

9

Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-38
SLIDE 38

Pin The Advisor

Irene Niel

PinAdvisor.cc

x

  • +
  • Simple data structures including primitives

(e.g., string, long), collections (e.g., list) and Conflict-free Data Types (e.g., counter, set)

  • Data type semantics avoid false sharing and

enable commutative operations

  • Defined in libDiamond language bindings

DCounter player1; DCounter player2; DString turn;

player1 player2 turn

App libDiamond

9

Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-39
SLIDE 39

Diamond Cloud

player1 player2 turn

App libDiamond Store

player1 player2 turn

App libDiamond

Diamond Programming Model

10

Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-40
SLIDE 40

Diamond Cloud

player1 player2 turn

App libDiamond Store

player1 player2 turn

App libDiamond

Diamond Programming Model

10

Reactive Data Types (RDTs) 
 Shared, persistent data structures Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store

slide-41
SLIDE 41

Diamond Cloud

player1 player2 turn

App libDiamond Store

PinAdvisor.cc

x

  • +
  • Key abstraction for providing

flexible, shared memory

  • Gives apps control over what app

data is shared and how it is

  • rganized
  • Enables Diamond to automatically

provide availability, fault-tolerance and consistency to RDTs

11

Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store

slide-42
SLIDE 42

Diamond Cloud

player1 player2 turn

App libDiamond

irene

Store

PinAdvisor.cc

x

  • +

rmap(player1, “irene”);

  • Key abstraction for providing

flexible, shared memory

  • Gives apps control over what app

data is shared and how it is

  • rganized
  • Enables Diamond to automatically

provide availability, fault-tolerance and consistency to RDTs

11

Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store

slide-43
SLIDE 43

Diamond Cloud

player1 player2 turn

App libDiamond

irene niel

Store

PinAdvisor.cc

x

  • +

rmap(player1, “irene”); rmap(player2, “niel”);

  • Key abstraction for providing

flexible, shared memory

  • Gives apps control over what app

data is shared and how it is

  • rganized
  • Enables Diamond to automatically

provide availability, fault-tolerance and consistency to RDTs

11

Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store

slide-44
SLIDE 44

Diamond Cloud

player1 player2 turn

App libDiamond

irene niel turn

Store

PinAdvisor.cc

x

  • +

rmap(player1, “irene”); rmap(player2, “niel”); rmap(turn, “turn”);

  • Key abstraction for providing

flexible, shared memory

  • Gives apps control over what app

data is shared and how it is

  • rganized
  • Enables Diamond to automatically

provide availability, fault-tolerance and consistency to RDTs

11

Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store

slide-45
SLIDE 45

Diamond Cloud

player1 player2 turn

App libDiamond

irene niel turn

Store

player1 player2 turn

App libDiamond

Diamond Programming Model

12

Reactive Data Types (RDTs) 
 Shared, persistent data structures Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store

slide-46
SLIDE 46

Diamond Cloud

player1 player2 turn

App libDiamond

irene niel turn

Store

player1 player2 turn

App libDiamond

Diamond Programming Model

12

Reactive Data Types (RDTs) 
 Shared, persistent data structures Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store

slide-47
SLIDE 47

player1 player2 turn

App Diamond Cloud

irene niel turn

Store

libDiamond

PinAdvisor.cc

x

  • +
  • Execute application code to

update rmapped RDTs

  • Gives application programmers

control over when to synchronize shared data

  • Ensures safe concurrent access to

shared data

13

Read-write Transactions
 Read-write transactions to update shared RDTs.

slide-48
SLIDE 48

player1 player2 turn

App Diamond Cloud

irene niel turn

Store

libDiamond

begin();

PinAdvisor.cc

x

  • +
  • Execute application code to

update rmapped RDTs

  • Gives application programmers

control over when to synchronize shared data

  • Ensures safe concurrent access to

shared data

13

Read-write Transactions
 Read-write transactions to update shared RDTs.

slide-49
SLIDE 49

player1 player2 turn

App Diamond Cloud

irene niel turn

Store

libDiamond

begin(); player1 = 0;

PinAdvisor.cc

x

  • +
  • Execute application code to

update rmapped RDTs

  • Gives application programmers

control over when to synchronize shared data

  • Ensures safe concurrent access to

shared data

13

Read-write Transactions
 Read-write transactions to update shared RDTs.

slide-50
SLIDE 50

player1 player2 turn

App Diamond Cloud

irene niel turn

Store

libDiamond

begin(); player1 = 0; player2 = 0;

PinAdvisor.cc

x

  • +
  • Execute application code to

update rmapped RDTs

  • Gives application programmers

control over when to synchronize shared data

  • Ensures safe concurrent access to

shared data

13

Read-write Transactions
 Read-write transactions to update shared RDTs.

slide-51
SLIDE 51

player1 player2 turn

App Diamond Cloud

irene niel turn

Store

libDiamond

begin(); player1 = 0; player2 = 0; turn = “irene”;

irene

PinAdvisor.cc

x

  • +
  • Execute application code to

update rmapped RDTs

  • Gives application programmers

control over when to synchronize shared data

  • Ensures safe concurrent access to

shared data

13

Read-write Transactions
 Read-write transactions to update shared RDTs.

slide-52
SLIDE 52

player1 player2 turn

App Diamond Cloud

irene niel turn

Store

libDiamond

irene

begin(); player1 = 0; player2 = 0; turn = “irene”; commit();

irene

PinAdvisor.cc

x

  • +
  • Execute application code to

update rmapped RDTs

  • Gives application programmers

control over when to synchronize shared data

  • Ensures safe concurrent access to

shared data

13

Read-write Transactions
 Read-write transactions to update shared RDTs.

slide-53
SLIDE 53

Diamond Cloud

player1 player2 turn

App libDiamond

irene niel turn

Store

player1 player2 turn

App libDiamond Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store

Diamond Programming Model

14

Reactive Data Types (RDTs) 
 Shared, persistent data structures Read-write Transactions
 Read-write transactions to update shared RDTs.

irene irene irene

slide-54
SLIDE 54

Diamond Cloud

player1 player2 turn

App libDiamond

irene niel turn

Store

player1 player2 turn

App libDiamond Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store

Diamond Programming Model

14

Reactive Data Types (RDTs) 
 Shared, persistent data structures Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates.

irene irene irene

slide-55
SLIDE 55

Pin The Advisor

Irene Niel

player1 player2 turn irene

App libDiamond

PinAdvisor.cc

x

  • +
  • Key abstraction for automatically

propagating updates to local data

  • Gives apps a consistent view of

shared data and control over what to sync

  • Automatically triggers app code in

response to updates from read- write transactions to shared RDTs

15

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates.

slide-56
SLIDE 56

Pin The Advisor

Irene Niel

player1 player2 turn irene

App libDiamond

PinAdvisor.cc

x

  • +
  • Key abstraction for automatically

propagating updates to local data

  • Gives apps a consistent view of

shared data and control over what to sync

  • Automatically triggers app code in

response to updates from read- write transactions to shared RDTs

registerReactiveTxn

15

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates.

slide-57
SLIDE 57

Pin The Advisor

Irene Niel

player1 player2 turn irene

App libDiamond

irene

PinAdvisor.cc

x

  • +
  • Key abstraction for automatically

propagating updates to local data

  • Gives apps a consistent view of

shared data and control over what to sync

  • Automatically triggers app code in

response to updates from read- write transactions to shared RDTs

registerReactiveTxn (displayUI(player1, player2, turn));

15

It’s your turn!

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates.

slide-58
SLIDE 58

Diamond Cloud

player1 player2 turn irene

App libDiamond

irene niel turn irene

Store

player1 player2 turn irene

App libDiamond

Diamond Programming Model

Pin The Advisor

Irene Niel

Pin The Advisor

Irene Niel 5

16

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-59
SLIDE 59

Diamond Cloud

player1 player2 turn irene

App libDiamond

irene niel turn irene

Store

player1 player2 turn irene

App libDiamond

Diamond Programming Model

Pin The Advisor

Irene Niel

Pin The Advisor

Irene Niel

You earned 5 pts!

5

16

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-60
SLIDE 60

Diamond Cloud

player1 player2 turn irene

App libDiamond

irene niel turn irene

Store

player1 player2 turn irene

App libDiamond

Diamond Programming Model

Pin The Advisor

Irene Niel

Pin The Advisor

Irene Niel

You earned 5 pts!

5

16

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-61
SLIDE 61

Diamond Cloud

player1 player2 turn irene

App libDiamond

irene niel turn irene

Store

player1 player2 turn irene

App libDiamond

Diamond Programming Model

Pin The Advisor

Irene Niel

Pin The Advisor

Irene Niel

5 niel

You earned 5 pts!

5

begin commit

5

16

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-62
SLIDE 62

Diamond Cloud

player1 player2 turn irene

App libDiamond

irene niel turn irene

Store

player1 player2 turn irene

App libDiamond

Diamond Programming Model

Pin The Advisor

Irene Niel

Pin The Advisor

Irene Niel

5 niel

You earned 5 pts!

5

begin commit

5

16

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-63
SLIDE 63

Diamond Cloud

player1 player2 turn irene

App libDiamond

irene niel turn irene

Store

player1 player2 turn irene

App libDiamond

Diamond Programming Model

Pin The Advisor

Irene Niel

Pin The Advisor

Irene Niel

5 niel

5 niel 5 niel

You earned 5 pts!

5

begin commit

5

16

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-64
SLIDE 64

Diamond Cloud

player1 player2 turn irene

App libDiamond

irene niel turn irene

Store

player1 player2 turn irene

App libDiamond

Diamond Programming Model

Pin The Advisor

Irene Niel

Pin The Advisor

Irene Niel

5 niel

5 niel 5 niel

You earned 5 pts!

5

begin commit

5

16

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Types (RDTs) 
 Shared, persistent data structures

slide-65
SLIDE 65

Diamond Cloud

player1 player2 turn irene

App libDiamond

irene niel turn irene

Store

player1 player2 turn irene

App libDiamond

Diamond Programming Model

Pin The Advisor

Irene Niel

Pin The Advisor

Irene Niel

5 niel

5 niel 5 niel

You earned 5 pts!

5

begin commit

5 5

16

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Types (RDTs) 
 Shared, persistent data structures

It’s your turn!

slide-66
SLIDE 66

Diamond Cloud

player1 player2 turn irene

App libDiamond

irene niel turn irene

Store

player1 player2 turn irene

App libDiamond

Diamond Programming Model

Pin The Advisor

Irene Niel

Pin The Advisor

Irene Niel

5 niel

5 niel 5 niel

You earned 5 pts!

5

begin commit

5 5

16

Reactive Transactions
 Read-only transactions that re-execute app code when the read set updates. Reactive Data Map (rmap) 
 Binding between RDTs in the app and the Diamond store Read-write Transactions
 Read-write transactions to update shared RDTs. Reactive Data Types (RDTs) 
 Shared, persistent data structures

It’s your turn!

Automated end-to-end 
 data management and storage with 
 fault-tolerance, availability and consistency

slide-67
SLIDE 67

Talk Outline

17

Diamond System & Programming Model
 What does Diamond provide for reactive apps?
 Diamond Guarantees & Implementation
 What does Diamond guarantee for reactive apps?
 Evaluation
 How does Diamond impact app complexity and performance?


Automated end-to-end data management and storage.

slide-68
SLIDE 68

Diamond ACID+R Guarantees

18

Atomicity - All or no updates to shared data in a read-write transaction complete. Consistency - All accesses in a transaction (read-write or reactive) reflect a single, point-in-time view of shared data. Isolation - All transactions reflect a serial execution order over shared data. Durability - All updates in committed transactions are never lost. Reactivity - All accesses in reactive transactions will eventually reflect the latest updates.

slide-69
SLIDE 69

Diamond Isolation Levels

Read-write Isolation Level Reactive Isolation Level Strict Serializability Serializable Snapshot Snapshot Isolation Serializable Snapshot Read Committed Read Committed

Stronger Guarantees Better
 Performance

19

slide-70
SLIDE 70

Diamond Implementation

  • Data-type Optimistic

Concurrency Control

  • Multi-versioned caching
  • Data Push Notifications

Alice libDiamond Front-end Back-end read(a)

<1,[11,13)>

gettimestamp

Front-end libDiamond Bob write(b,1) commit prepare

validation

  • k
  • k

commit pub (b,1,16) notify (b,1,16) re-exec read(b) register read(b,14)

<0,[11,15)>

reg(14,[b]) sub (b,14)

execute commit

callback

Read-Write Transaction Reactive Transaction Register Notify

Diamond T ransaction Protocol Wide-area Optimizations T ake a look at the paper!

slide-71
SLIDE 71

Talk Outline

21

Diamond System & Programming Model
 What does Diamond provide for reactive apps?


Automated end-to-end data management and storage.

Diamond Guarantees & Implementation
 What does Diamond guarantee for reactive apps?
 Evaluation
 How does Diamond impact app complexity and performance?


Strong ACID+R transactional guarantees

slide-72
SLIDE 72

Evaluation Overview

  • Does Diamond simplify reactive applications?
  • How does Diamond perform compare to a hand coded implementation?
  • Testbed: Google Compute Engine VMs (5 shards x 3 replicas)
  • Workload: Retwis-based Twitter benchmark

22

slide-73
SLIDE 73

Diamond reduces the complexity and improves the guarantees of reactive apps.

Application Original LoC Diamond LoC % Saved Multi-player Game 46 34 26% Chat Room 335 225 33% Scrabble clone 8729 7603 13% Twitter clone 14,278 12,554 13%

23

slide-74
SLIDE 74

Diamond reduces the complexity and improves the guarantees of reactive apps.

Application Original LoC Diamond LoC % Saved Multi-player Game 46 34 26% Chat Room 335 225 33% Scrabble clone 8729 7603 13% Twitter clone 14,278 12,554 13%

23

No UI.
 Mostly
 sync code.

slide-75
SLIDE 75

Diamond reduces the complexity and improves the guarantees of reactive apps.

Application Original LoC Diamond LoC % Saved Multi-player Game 46 34 26% Chat Room 335 225 33% Scrabble clone 8729 7603 13% Twitter clone 14,278 12,554 13%

23

No UI.
 Mostly
 sync code. Full UI.
 Complex 
 app logic.

slide-76
SLIDE 76

Diamond reduces the complexity and improves the guarantees of reactive apps.

Application Original LoC Diamond LoC % Saved Multi-player Game 46 34 26% Chat Room 335 225 33% Scrabble clone 8729 7603 13% Twitter clone 14,278 12,554 13%

23

No UI.
 Mostly
 sync code. Full UI.
 Complex 
 app logic. +durability +durability +consistency +durability +isolation +reactivity +reactivity

slide-77
SLIDE 77

Diamond’s data management has low overhead.

Throughput (txn/sec) 0K 8K 16K 24K 32K Redis Diamond
 Read
 Committed Diamond
 Strict 
 Serializability

24

slide-78
SLIDE 78

Diamond’s data management has low overhead.

Throughput (txn/sec) 0K 8K 16K 24K 32K Redis Diamond
 Read
 Committed Diamond
 Strict 
 Serializability

  • 2.1%

24

slide-79
SLIDE 79

Diamond’s data management has low overhead.

Throughput (txn/sec) 0K 8K 16K 24K 32K Redis Diamond
 Read
 Committed Diamond
 Strict 
 Serializability

  • 2.1%

Weak
 Consistency Strong
 Consistency

24

slide-80
SLIDE 80

Diamond’s data management has low overhead.

Throughput (txn/sec) 0K 8K 16K 24K 32K Redis Diamond
 Read
 Committed Diamond
 Strict 
 Serializability

  • 2.1%
  • 48.5%

Weak
 Consistency Strong
 Consistency Linearizable
 T ransactions

24

slide-81
SLIDE 81

What does Diamond provide for reactive apps?


Automated end-to-end data management and storage.

What does Diamond guarantee for reactive apps?


Strong ACID+R transactional guarantees.

How does Diamond impact app complexity and performance?


Simplifies reactive apps with low overhead.

Summary

https://github.com/UWSysLab/diamond

25

slide-82
SLIDE 82

Related Work

  • Distributed Programming Frameworks


Meteor, Parse, Firebase, Mjolnir, Mapjax, RethinkDB

  • Client-side Programming Frameworks


React, Angular, Blaze, ReactiveX

  • Distributed Storage Systems


Redis, MongoDB, Dropbox

  • Notification/Pub-Sub/Streaming Services


Thialfi, Apache Kafka, Amazon Kinesis