C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica - - PowerPoint PPT Presentation

▶
c3 cutting tail latency in cloud data stores via adaptive
SMART_READER_LITE
LIVE PREVIEW

C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica - - PowerPoint PPT Presentation

Dec 08, 2023 421 likes •844 views

C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica Selection Lalith Suresh (TU Berlin) with Marco Canini (UCL), Stefan Schmid, Anja Feldmann (TU Berlin) Tail-latency matters Tens to Thousands One of data accesses User

slide-1
SLIDE 1

C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica Selection

Lalith Suresh (TU Berlin)

with Marco Canini (UCL), Stefan Schmid, Anja Feldmann (TU Berlin)

slide-2
SLIDE 2

2

One User Request Tens to Thousands

  • f data accesses

Tail-latency matters

slide-3
SLIDE 3

3

For 100 100 leaf servers, 99 99th

th percentile latency

will reflect in 63% 63% of user requests! One User Request

Tail-latency matters

Tens to Thousands

  • f data accesses
slide-4
SLIDE 4

4

Server performance fluctuations are the norm

Queueing delays Skewed access patterns

CDF

Resource contention Background activities

slide-5
SLIDE 5

Effectiveness of replica selection in reducing tail latency?

5

?

Client Server Server Server

Request

slide-6
SLIDE 6

Replica Selection Challenges

6

slide-7
SLIDE 7

Replica Selection Challenges

  • Service-time variations

7

Request

Client Server Server Server

4 ms 5 ms 30 ms

slide-8
SLIDE 8

Replica Selection Challenges

  • Herd behavior and load oscillations

8

Request Request Request

Client Client Client Server Server Server

slide-9
SLIDE 9

9

Impact of Replica Selection in Practice?

Dy Dynami mic Sn Snitching

Uses history of read latencies and I/O load for replica selection

slide-10
SLIDE 10

10

Experimental Setup

  • Cassandra cluster on Amazon EC2
  • 15 nodes, m1.xlarge instances
  • Read-heavy workload with YCSB (120 threads)
  • 500M 1KB records (larger than memory)
  • Zipfian key access pattern
slide-11
SLIDE 11

11

Cassandra Load Profile

slide-12
SLIDE 12

12

Also observed that 99.9th percentile latency ~ 10x median latency

Cassandra Load Profile

slide-13
SLIDE 13

13

Load Conditioning in our Approach

slide-14
SLIDE 14

C3

Adaptive replica selection mechanism that is robust to service time heterogeinity

14

slide-15
SLIDE 15

C3

  • Replica Ranking
  • Distributed Rate Control

15

slide-16
SLIDE 16

C3

  • Replica Ranking
  • Distributed Rate Control

16

slide-17
SLIDE 17

17

Client Server Client Client Server µ-1 = 2 ms µ-1 = 6 ms

slide-18
SLIDE 18

18

Client Server Client Client Server

Balance product of queue-size and service time { q q · · µ-1 }

µ-1 = 2 ms µ-1 = 6 ms

slide-19
SLIDE 19

19

Server-side Feedback

Servers piggyback {qs } } and {µν𝒕

#𝟐}

} in every response

Client Server

{ qs , , µν𝒕

#𝟐 }

slide-20
SLIDE 20

20

Server-side Feedback

  • Concurrency compensation

Servers piggyback {qs } } and {µν𝒕

#𝟐}

} in every response

slide-21
SLIDE 21

21

Server-side Feedback

  • Concurrency compensation

𝑟 &' = 1 + ¡𝑝𝑡'. 𝑥 + 𝑟'

Servers piggyback {qs } } and {µν𝒕

#𝟐}

} in every response

Outstanding requests Feedback

slide-22
SLIDE 22

22

Select server with min ¡𝑟 &' ¡. µν𝒕

#𝟐 ?

slide-23
SLIDE 23

23

Select server with min ¡𝑟 &' ¡. µν𝒕

#𝟐 ?

Server Server µ-1 = 4 ms µ-1 = 20 ms 20 requests 100 requests!

  • Potentially long queue sizes
  • What if a GC pause happens?
slide-24
SLIDE 24

24

Penalizing Long Queues

Server Server µ-1 = 4 ms µ-1 = 20 ms 20 requests 35 requests

Select server with min ¡ 𝑟 &' ¡. µν𝒕

#𝟐

b

b = 3

slide-25
SLIDE 25

C3

  • Replica Ranking
  • Distributed Rate Control

25

slide-26
SLIDE 26

26

Need for rate control

Replica ranking insufficient

  • Avoid saturating individual servers?
  • Non-internal sources of performance

fluctuations?

slide-27
SLIDE 27

27

Cubic c Rate Control

  • Clients adjust sending

rates according to cubic function

  • If receive rate isn’t

increasing further, multiplicatively decrease

slide-28
SLIDE 28

28

Putting everything together

Server Server 1000 ¡ req/s 2000 ¡ req/s Rate Limiters Replica group scheduler Sort replicas by score C3 Client { Feedback }

slide-29
SLIDE 29

29

Implementation in Cassandra

Details in the paper!

slide-30
SLIDE 30

30

Evaluation

Amazon EC2 Controlled Testbed Simulations

slide-31
SLIDE 31

31

Evaluation

Amazon EC2

  • 15 node Cassandra cluster
  • M1.xlarge
  • Workloads generated using YCSB (120 threads)
  • Read-heavy, update-heavy, read-only
  • 500M 1KB records dataset (larger than memory)
  • Compare against Cassandra’s Dynamic Snitching (DS)
slide-32
SLIDE 32

32

Lower is better

slide-33
SLIDE 33

33

2x – 3x improved 99.9 percentile latencies Also improves median and mean latencies

slide-34
SLIDE 34

34

2x – 3x improved 99.9 percentile latencies 26% - 43% improved throughput

slide-35
SLIDE 35

35

Takeaway: C3 does not tradeoff throughput for latency

slide-36
SLIDE 36

36

How does C3 react to dynamic workload changes?

  • Begin with 80 read-heavy workload generators
  • 40 update-heavy generators join the system after 640s
  • Observe latency profile with and without C3
slide-37
SLIDE 37

37

Latency profile degrades gracefully with C3

Takeaway: C3 reacts effectively to dynamic workloads

slide-38
SLIDE 38

38

Summary of other results

Higher system load Skewed record sizes SSDs instead of HDDs

> > 3x 3x better 99.9th percentile latency 50 50% higher throughput than with DS

slide-39
SLIDE 39

39

Ongoing work

  • Tests at SoundCloud and Spotify
  • Stability analysis of C3
  • Alternative rate adaptation algorithms
  • Token aware Cassandra clients
slide-40
SLIDE 40

40

?

Client Server Server Server

C3

Replica Ranking +

  • Dist. Rate Control

Summary